squote

A semantic search engine that takes some input text and returns some (questionably) relevant (questionably) famous quotes.

Built with:

• bert-as-a-service
• faiss (actually, faiss_prebuilt)
• streamlit

Quotes from https://thewebminer.com/.

setup

First, install the necessary dependencies into a python 3 environment of your choice. For instance, to install the deps into a venv, run

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

There are additional native dependencies for FAISS: libomp and libopenblas must be available (see the FAISS repo for install instructions). All other commands should be run from within the virtual environment.

A Makefile is provided to make things nice and easy.

make dirs
make data   # downloads the raw quote data
make model  # downloads ~350MB of BERT weights

running

Before we can run the app, we need embeddings of the quotes. To generate the embeddings and save them in a pickled pandas DataFrame, run the commands below. This will take some time (couple of hours) on CPU.

make serve  # this runs bert-as-a-service
make embed  # this computes the embeddings

Once the embeddings exist, we can run the streamlit app with:

make serve  # not needed if still running from above
make app

Have fun!