LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

Pytorch implementation of "M-LSD: Towards Light-weight and Real-time Line Segment Detection"

PyTorch evaluation code for Delving Deep into the Generalization of Vision Transformers under Distribution Shifts.

An Open Source Machine Learning Framework for Everyone

A novel framework to automatically learn high-quality scanning of non-planar, complex anisotropic appearance.

Automatic Idiomatic Expression Detection

Instance-based label smoothing for improving deep neural networks generalization and calibration

A simple, fast, and efficient object detector without FPN

The PASS dataset: pretrained models and how to get the data - PASS: Pictures without humAns for Self-Supervised Pretraining

Generating retro pixel game characters with Generative Adversarial Networks. Dataset "TinyHero" included.

Mengzi Pretrained Models

The Body Part Regression (BPR) model translates the anatomy in a radiologic volume into a machine-interpretable form.

ChainerRL is a deep reinforcement learning library built on top of Chainer.

Graph Convolutional Networks for Temporal Action Localization (ICCV2019)

[Machine Learning Engineer Basic Guide] 부스트캠프 AI Tech - Product Serving 자료

Framework for abstracting Amiga debuggers and access to AmigaOS libraries and devices.

This is the repository for the NeurIPS-21 paper [Contrastive Graph Poisson Networks: Semi-Supervised Learning with Extremely Limited Labels].

Improving Query Representations for DenseRetrieval with Pseudo Relevance Feedback:A Reproducibility Study.

RL algorithm PPO and IRL algorithm AIRL written with Tensorflow.

Object tracking using YOLO and a tracker(KCF, MOSSE, CSRT) in openCV

'Solving the sampling problem of the Sycamore quantum supremacy circuits