Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Last update: Jan 08, 2023

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Model	STS12	STS13	STS14	STS15	STS16	STSb	SICK-R	Avg.
unsup-prompt-bert-base Download	71.98	84.66	77.13	84.52	81.10	82.03	70.64	78.87
unsup-prompt-roberta-base Download	73.98	84.73	77.88	84.93	81.89	82.74	69.21	79.34
sup-prompt-bert-base Download	75.48	85.59	80.57	85.99	81.08	84.56	80.52	81.97
sup-prompt-roberta-base Download	76.75	85.93	82.28	86.69	82.80	86.14	80.04	82.95

Download Data

cd SentEval/data/downstream/
bash download_dataset.sh
cd -
cd ./data
bash download_wiki.sh
bash download_nli.sh
cd -

Static token embedding with removing embedding biases

robert-base, bert-base-cased and robert-base-uncased

./run.sh roberta-base-embedding-only-remove-baises
./run.sh bert-base-cased-embedding-only-remove-baises
./run.sh bert-base-uncased-embedding-only-remove-baises

Non fine-tuned BERT with Prompt

bert-base-uncased with prompt

./run.sh bert-prompt

bert-base-uncased with optiprompt

./run.sh bert-optiprompt

fine-tuned BERT with Prompt

unsupervised

SEED=0
./run.sh unsup-roberta $SEED

SEED=0
./run.sh unsup-bert $SEED

supervised

./run.sh sup-roberta

./run.sh sup-bert

Our Code is based on SimCSE

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Download Data

Static token embedding with removing embedding biases

Non fine-tuned BERT with Prompt

fine-tuned BERT with Prompt

unsupervised

supervised

Owner

Lipschitz-constrained Unsupervised Skill Discovery

Fully Convlutional Neural Networks for state-of-the-art time series classification

A library for augmentation of a YOLO-formated dataset

Pytorch implementation of MixNMatch

FS-Mol: A Few-Shot Learning Dataset of Molecules

Source code for the paper "PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction" in ACL2021

The King is Naked: on the Notion of Robustness for Natural Language Processing

Code for Blind Image Decomposition (BID) and Blind Image Decomposition network (BIDeN).

An example of semantic segmentation using tensorflow in eager execution.

A Pytorch Implementation of Domain adaptation of object detector using scissor-like networks

Implementation of Monocular Direct Sparse Localization in a Prior 3D Surfel Map (DSL)

2021搜狐校园文本匹配算法大赛分比我们低的都是帅哥队

Convert openmmlab (not only mmdetection) series model to tensorrt

A boosting-based Multiple Instance Learning (MIL) package that includes MIL-Boost and MCIL-Boost

PixelPyramids: Exact Inference Models from Lossless Image Pyramids (ICCV 2021)

Pytorch implementation of the paper Improving Text-to-Image Synthesis Using Contrastive Learning

House_prices_kaggle - Predict sales prices and practice feature engineering, RFs, and gradient boosting

Galactic and gravitational dynamics in Python

An implementation of the BADGE batch active learning algorithm.

Pointer-generator - Code for the ACL 2017 paper Get To The Point: Summarization with Pointer-Generator Networks

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Download Data

Static token embedding with removing embedding biases

Non fine-tuned BERT with Prompt

fine-tuned BERT with Prompt

unsupervised

supervised

Owner

Lipschitz-constrained Unsupervised Skill Discovery

Fully Convlutional Neural Networks for state-of-the-art time series classification

A library for augmentation of a YOLO-formated dataset

Pytorch implementation of MixNMatch

FS-Mol: A Few-Shot Learning Dataset of Molecules

Source code for the paper "PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction" in ACL2021

The King is Naked: on the Notion of Robustness for Natural Language Processing

Code for Blind Image Decomposition (BID) and Blind Image Decomposition network (BIDeN).

An example of semantic segmentation using tensorflow in eager execution.

A Pytorch Implementation of Domain adaptation of object detector using scissor-like networks

Implementation of Monocular Direct Sparse Localization in a Prior 3D Surfel Map (DSL)

2021搜狐校园文本匹配算法大赛 分比我们低的都是帅哥队

Convert openmmlab (not only mmdetection) series model to tensorrt

A boosting-based Multiple Instance Learning (MIL) package that includes MIL-Boost and MCIL-Boost

PixelPyramids: Exact Inference Models from Lossless Image Pyramids (ICCV 2021)

Pytorch implementation of the paper Improving Text-to-Image Synthesis Using Contrastive Learning

House_prices_kaggle - Predict sales prices and practice feature engineering, RFs, and gradient boosting

Galactic and gravitational dynamics in Python

An implementation of the BADGE batch active learning algorithm.

Pointer-generator - Code for the ACL 2017 paper Get To The Point: Summarization with Pointer-Generator Networks

2021搜狐校园文本匹配算法大赛分比我们低的都是帅哥队