PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers

Last update: Jan 08, 2022

Related tags

Overview

code and model are coming soon...

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers

Related tags

Overview

Contributing

Trademarks

Owner

Microsoft

A CRM department in a local bank works on classify their lost customers with their past datas. So they want predict with these method that average loss balance and passive duration for future.

Code for papers "Generation-Augmented Retrieval for Open-Domain Question Answering" and "Reader-Guided Passage Reranking for Open-Domain Question Answering", ACL 2021

IMDB film review sentiment classification based on BERT's supervised learning model.

Basic yet complete Machine Learning pipeline for NLP tasks

PUA Programming Language written in Python.

CoNLL-English NER Task (NER in English)

OceanScript is an Esoteric language used to encode and decode text into a formulation of characters

Traditional Chinese Text Recognition Dataset: Synthetic Dataset and Labeled Data

Multilingual text (NLP) processing toolkit

Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding

The model is designed to train a single and large neural network in order to predict correct translation by reading the given sentence.

pytorch implementation of Attention is all you need

Simple Text-Generator with OpenAI gpt-2 Pytorch Implementation

Machine learning models from Singapore's NLP research community

End-to-end text to speech system using gruut and onnx. There are 40 voices available across 8 languages.

novel deep learning research works with PaddlePaddle

Sentello is python script that simulates the anti-evasion and anti-analysis techniques used by malware.

I can help you convert your images to pdf file.

Framework for fine-tuning pretrained transformers for Named-Entity Recognition (NER) tasks

Transcribing audio files using Hugging Face's implementation of Wav2Vec2 + "chain-linking" NLP tasks to combine speech-to-text with downstream tasks like translation and summarisation.