Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Last update: Jan 08, 2023

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Model	STS12	STS13	STS14	STS15	STS16	STSb	SICK-R	Avg.
unsup-prompt-bert-base Download	71.98	84.66	77.13	84.52	81.10	82.03	70.64	78.87
unsup-prompt-roberta-base Download	73.98	84.73	77.88	84.93	81.89	82.74	69.21	79.34
sup-prompt-bert-base Download	75.48	85.59	80.57	85.99	81.08	84.56	80.52	81.97
sup-prompt-roberta-base Download	76.75	85.93	82.28	86.69	82.80	86.14	80.04	82.95

Download Data

cd SentEval/data/downstream/
bash download_dataset.sh
cd -
cd ./data
bash download_wiki.sh
bash download_nli.sh
cd -

Static token embedding with removing embedding biases

robert-base, bert-base-cased and robert-base-uncased

./run.sh roberta-base-embedding-only-remove-baises
./run.sh bert-base-cased-embedding-only-remove-baises
./run.sh bert-base-uncased-embedding-only-remove-baises

Non fine-tuned BERT with Prompt

bert-base-uncased with prompt

./run.sh bert-prompt

bert-base-uncased with optiprompt

./run.sh bert-optiprompt

fine-tuned BERT with Prompt

unsupervised

SEED=0
./run.sh unsup-roberta $SEED

SEED=0
./run.sh unsup-bert $SEED

supervised

./run.sh sup-roberta

./run.sh sup-bert

Our Code is based on SimCSE

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Download Data

Static token embedding with removing embedding biases

Non fine-tuned BERT with Prompt

fine-tuned BERT with Prompt

unsupervised

supervised

Owner

Code Impementation for "Mold into a Graph: Efficient Bayesian Optimization over Mixed Spaces"

Unofficial Tensorflow 2 implementation of the paper Implicit Neural Representations with Periodic Activation Functions

ICON: Implicit Clothed humans Obtained from Normals

SGPT: Multi-billion parameter models for semantic search

Matching python environment code for Lux AI 2021 Kaggle competition, and a gym interface for RL models.

FCOSR: A Simple Anchor-free Rotated Detector for Aerial Object Detection

Point Cloud Registration using Representative Overlapping Points.

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)

Filtering variational quantum algorithms for combinatorial optimization

Anchor-free Oriented Proposal Generator for Object Detection

Learn about Spice.ai with in-depth samples

Real-Time and Accurate Full-Body Multi-Person Pose Estimation&Tracking System

An open source python library for automated feature engineering

Sample code from the Neural Networks from Scratch book.

Face and Body Tracking for VRM 3D models on the web.

Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds (Local-Lip)

Code for the paper "Query Embedding on Hyper-relational Knowledge Graphs"

Air Quality Prediction Using LSTM

Official Pytorch Code for the paper TransWeather

NAS Benchmark in "Prioritized Architecture Sampling with Monto-Carlo Tree Search", CVPR2021