A free website that keeps the people informed about housing and evictions.

Overview

Eviction Tracker

Currently helping verify detainer warrant data for middle Tennessee - via Middle TN DSA - Red Door Collective

Features

  • Presents data visualizations for community awareness
  • Organizer Portal
    • Create direct action campaigns
    • Schedule events for campaigns
    • Record relevant information for phonebanking sessions
  • Admin Portal
    • Create and update detainer warrants

Development

Setup

You'll need to set a few secret environment variables before booting the app.

You can copy-paste the following lines into your terminal if you're using bash.

echo 'export TWILIO_ACCOUNT_SID=fake' >> ~/.bash_profile
echo 'export TWILIO_AUTH_TOKEN=fake' >> ~/.bash_profile

Then, you can either run source ~/.bash_profile or open a new terminal tab and proceed to the next step.

If you're wondering what the above code does: it defines two environment variables with the value "fake" and ensures they are available in all future (bash) shell sessions. TODO: make this process less painful.

Nix

Nix is a tool we use to ensure reproducable builds and deployments. Python's default build and deployment system is notoriously tricky. We've elected to use Nix instead to ease setup. Nix can be used for installing and deploying multiple languages, which may end up being useful for this project.

Install Nix

Please follow the instructions to install Nix on their website.

(Please ignore instructions regarding NixOS - that's a whole new operating system).

Using Nix

To work with the same python version as everyone else working on this project, you'll need to make sure you're in the nix-shell first.

  1. Check that you're in the same directory as shell.nix
  2. Type nix-shell and hit enter
  3. Wait for necessary dependencies to install from the internet to your machine

You'll get a new prompt that looks like:

[nix-shell:~/some/path/eviction-tracker]$ 

Now you're ready to run python and ipython! You can escape the nix-shell at any time by holding ctrl pressing the D key or typing exit and pressing enter.

Database

We use PostgreSQL for both development and production so that we can be assured working with the data locally will work exactly the same in production. Unfortunately, this requires a bit more manual setup for developers who don't already use Postgres.

First-time postgres

Install postgres with the most convenient installer for your OS.

The app doesn't care how you set up your postgres connection, so feel free to set up your postgres service in whatever method is convenient for you. By default, the SQLALCHEMY_DATABASE_URL is set to postgresql+psycopg2://[email protected]/eviction_tracker, which assumes that you've created a database called eviction_tracker, a user called eviction_tracker (without a password), and that you are using the default host and port of a postgres service running on the same host as the app.

Feel free to override this environment variable like SQLALCHEMY_DATABASE_URL="..." flask shell or create the same setup locally. If you're running your database on another host like a Docker container, you'll need to change localhost and possibly add a :port to your override.

Some postgres instructions on setting up for the default flow:

psql -U postgres # or, the superuser you've set up. -U postgres is default for modern installs

The above command will place you in this postgres shell:

postgres=# CREATE DATABASE eviction_tracker;
postgres=# CREATE USER eviction_tracker;
postgres=# GRANT ALL ON eviction_tracker to eviction_tracker;

These commands create the database, user, and assign all privileges for the user to the development database;

Migrations

Make sure to run flask db upgrade after setting up your database, or whenever a new migration is added to the migrations/ folder.

Google Spreadsheets

You'll need authentication to make requests to the online spreadsheet with the most current data.

Please install Keybase, a secure messaging platform. I will send any contributor who requests the authentication secrets the necessary file. When I send the file over Keybase, you'll need to download it, and move it somewhere you won't lose it.

By default, our scripts expect the file to be at the following path: ~/.config/gspread/service_account.json. In plain english: it should be in your home directory, under a hidden folder named .config and finally inside another folder called gspread. The file should be named service_account.json. If these are all true, you're good to go! If you'd like to save the file elsewhere or rename it, just run the script with your custom path under the optional argument: --service_account_key=/path/to/your/file.json.

Using a REPL

REPL (Read Eval Print Loop) is a concept implemented in many programming languages. If you've never written python before, we recommend spending an afternoon on these basics. You'll interact with a REPL in those courses.

While in a Nix Shell, launch the IPython shell like so:

ipython

And now, you can write python code with any of our libraries!

Running commands

While in a Nix Shell, run flask .

Bootstrap

You'll want to set up a small set of users and roles for development.

Run flask bootstrap to provision users and roles.

Afterwards, you'll be able to login to several demo users with varying roles.

  1. Superuser - [email protected]:123456
  2. Admin - [email protected]:123456
  3. Organizer - [email protected]:123456
  4. Defendant - [email protected]:123456

Sync database

To sync the data from our org's Google Spreadsheet, run flask sync .

If you want just a bit of data to work with locally, pass the --limit argument.

Example: flask sync --sheet-name 'detainer-warrants_15-02-2020' --limit 10 will populate the database with 10 detainer warrants from the spreadsheet titled detainer-warrants_15-02-2020.

Run app

To run the website locally, use flask run --no-reload within the nix-shell.

You'll be able to visit it at http://127.0.0.1:5000/

You can browse api endpoints like detainer warrants at http://127.0.0.1:5000/api/v1/detainer-warrants

Comments
  • Save a mock twilio response to the database

    Save a mock twilio response to the database

    Context

    Before we verify any phone numbers, we need to make sure we can save the verification data so we never lose what we've paid for. We are building a database of information: detainer warrants, attorneys, defendants, etc. There will also be a table for phone_number_verifications. We will then always be able to look up our data from Twilio in that database.

    Description

    This should serve as a test that our code will do the right thing when eventually validating a phone number.

    We should make sure that the JSON response documented by Twilio is able to be saved to the database.

    Example JSON response

    {
      "caller_name": {
        "caller_name": "Delicious Cheese Cake",
        "caller_type": "CONSUMER",
        "error_code": null
      },
      "carrier": null,
      "country_code": "US",
      "national_format": "(510) 867-5310",
      "phone_number": "+15108675310",
      "add_ons": null,
      "url": "https://lookups.twilio.com/v1/PhoneNumbers/+15108675310"
    }
    

    Modeling

    We need to build a database table to properly model this data before inserting it.

    I suggest:

    class PhoneNumberVerification(Base):
        __tablename__ = 'phone_number_verifications'
        id = Column(Integer, foreign_key=True)
        caller_name = Column(String)
        caller_type = Column(Integer) # smaller column than String
        error_code = Column(Integer)
        carrier = Column(String)
        country_code = Column(String)
        national_format = Column(String)
        phone_number = Column(String)
    

    This is actively under construction. Follow progress in #6.

    Inserting

    Please follow SQLAlchemy docs for how to insert this kind of data to the database

    Testing

    We don't yet have a test suite, i'll comment on this issue when we do. When that is set up, we can write a test to close out this issue:

    1. Create mock JSON data
    2. Test that the JSON can be inserted into the phone_number_verifications table
    opened by gregziegan 3
  • Test Twilio Phone Verfication Database Insertion

    Test Twilio Phone Verfication Database Insertion

    This creates a test to ensure a mock JSON Twilio response can be saved as a valid PhoneNumberVerification entry. A second test was added that checks the input of the mock Twilio response to what is saved in the database.

    opened by donald-mate 1
  • Set up SQLite database

    Set up SQLite database

    Binary files are bloated and terrible way to manage data over time. Let's use a simple database like SQLite for the time being.

    Checklist

    • [x] Set up SQLite for local development
    • [x] Create tables for detainer warrants and associated data
    • [ ] Create table for phone number verifications
    • [ ] Automate spreadsheet insertion into detainer warrant table
    opened by gregziegan 1
  • Separate Judgments from Hearings

    Separate Judgments from Hearings

    Context

    The site uses the concept of a Judgment to refer to both a hearing (scheduled court date) and its outcome. This is OK, but makes matching Judgment to hearings difficult as the only method of matching judgments to hearings is via court and file date. The judgment document is filed sometime after the court date. We can use this knowledge to guess which judgment corresponds to which hearing. This code was quite messy to work with when there were not two separate tables.

    Description

    Hearings and Judgments

    This PR introduces another table for hearings, where web scraping of the CircuitClerk will always be the sole author of that data. Users will not be able to edit hearing details. Judgment data is now connected to hearings via automatic PDF scraping, with an ability for users to fix details of the scraping.

    UI changes

    The detainer warrant is now merely a landing page for hearings and judgments. You must edit a judgement on a judgment page. There is now a Judgments table UI to help expedite auditing of judgments.

    Database changes

    I tried to get the migration scripts working for the data transition but got stuck on certain SQL transactions. After messing with it for a few hours i decided this was not worth the time since we always have a data backup since mid-November via google sheets and database dumps on the production server. I'll instead be executing sqlalchemy queries through the flask shell to fix data.

    opened by gregziegan 0
  • Convert to Paack-ui; remove extraneous fields; expand Judgement

    Convert to Paack-ui; remove extraneous fields; expand Judgement

    1. convert to paack-ui
    2. remove presiding judge from warrant
    3. move court date and court room to "judgements"
    4. switch to posix for all date and datetime columns
    opened by gregziegan 0
  • An organizer can record info from their phonebank into a form on the website

    An organizer can record info from their phonebank into a form on the website

    Phone Bank Script

    Let's add some text fields to capture notes.

    There's a lot of "If" statements in the script that would be great to automate. Say, click a button and another piece of the form shows up.

    opened by gregziegan 0
Releases(v314)
Height 2 LDraw With python

Height2Ldraw About This project aims to be able to make a full lego 3D model using the ldraw file format (.ldr) from a height and color map, currently

1 Dec 22, 2021
Decipher using Markov Chain Monte Carlo

Decipher using Markov Chain Monte Carlo

Science étonnante 43 Dec 24, 2022
A python package that adds "docs" command to disnake

About This extension's purpose is of adding a "docs" command, its purpose is to help documenting in chat. How To Load It from disnake.ext import comma

7 Jan 03, 2023
BlueBorne Dockerized

BlueBorne Dockerized This is the repo to reproduce the BlueBorne kill-chain on Dockerized Android as described here, to fully understand the code you

SecSI 5 Sep 14, 2022
디텍션 유틸 모음

Object detection utils 유틸모음 설명 링크 convert convert 관련코드 https://github.com/AI-infinyx/ob_utils/tree/main/convert crawl 구글, 네이버, 빙 등 크롤링 관련 https://gith

codetest 41 Jan 22, 2021
Data repo for one-among.us

Our Data Data repo for one-among.us File Structure Directory /people/userid/: Data for a specific person info.json5: Profile information page.md: Pr

Hykilpikonna 55 Dec 30, 2022
Web App for University Project

University Project About I made this web app to finish a project assigned by my teacher. It is written entirely in Python, thanks to streamlit to make

15 Nov 27, 2022
Percolation simulation using python

PythonPercolation Percolation simulation using python Exemple de percolation : Etude statistique sur le pourcentage de remplissage jusqu'à percolation

Tony Chouteau 1 Sep 08, 2022
Project issue to website data transformation toolkit

braintransform Project issue to website data transformation toolkit. Introduction The purpose of these scripts is to be able to dynamically generate t

Brainhack 1 Nov 19, 2021
A C-like hardware description language (HDL) adding high level synthesis(HLS)-like automatic pipelining as a language construct/compiler feature.

██████╗ ██╗██████╗ ███████╗██╗ ██╗███╗ ██╗███████╗ ██████╗ ██╔══██╗██║██╔══██╗██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝ ██████╔╝██║██████╔╝█

Julian Kemmerer 391 Jan 01, 2023
Combines power of torch, numerical methods to conquer and solve ALL {O,P}DEs

torch_DE_solver Combines power of torch, numerical methods and math overall to conquer and solve ALL {O,P}DEs There are three examples to provide a li

Natural Systems Simulation Lab 28 Dec 12, 2022
Student Enrollment Analysis System

SEAS Student Enrollment Analysis System Steps to start working: create a user name "seas", host name: local, password: seas, mark all checkbox - go C

Md. Zakaria Kabir 3 Jul 12, 2022
Tomador de ramos UC automatico para Windows, Linux y macOS

auto-ramos v2.0 Tomador de ramos UC automatico para Windows, Linux y macOS Funcion Este script de Python tiene como principal objetivo hacer que la to

Open Source eUC 13 Jun 29, 2022
This is an implementation of PEP 557, Data Classes.

This is an implementation of PEP 557, Data Classes. It is a backport for Python 3.6. Because dataclasses will be included in Python 3.7, any discussio

Eric V. Smith 561 Dec 06, 2022
A Way to Use Python, Easier.

PyTools A Way to Use Python, Easier. How to Install Just copy this code, then make a new file in your project directory called PyTools.py, then paste

Kamran 2 Aug 15, 2022
This is the Quiz that I made using Python Programming Language. This can only run in the Terminal

This is the Quiz that I made using Python Programming Language. This can only run in the Terminal

YOSHITHA RATHNAYAKE 1 Apr 08, 2022
Make after-work Mending More flexible In Python

Mending Make after-work Mending More flexible In Python A Lite Package focuses on making project's after-post mending pythonic and flexible. Certainly

2 Jun 15, 2022
Python for Microscopists and other image processing enthusiasts

The YouTube channel associated with this code walks you through the entire process of learning to code in Python; all the way from basics to advanced machine learning and deep learning.

Dr. Sreenivas Bhattiprolu 2.3k Jan 01, 2023
An OBS script to fuze files together

OBS TEXT FUZE Fuze text files and inject the output into a text source. The Index file directory should be a list of file directorys for the text file

SuperZooper3 1 Dec 27, 2021
NExT-Ford-aula4 - NExT Ford aula4

Questão 1: vocês deveram fazer o passo a passo de como ficará as pilhas(Stack) e

Gerson 1 Jan 06, 2022