RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Last update: Dec 22, 2022

Overview

RITA: a Study on Scaling Up Generative Protein Sequence Models

RITA is a family of autoregressive protein models, developed by a collaboration of Lighton, the OATML group at Oxford, and the Debbie Marks Lab at Harvard.

Model	#Params	d_model	layers	lm loss uniref-100
Small	85M	768	12	2.31
Medium	300M	1024	24	2.01
Large	680M	1536	24	1.82
XLarge	1.2B	2048	24	1.70

Results

For full results see our preprint: https://arxiv.org/abs/2205.05789

Usage

Instantiate a model like so:

from transformers import AutoModel, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("lightonai/RITA_s, trust_remote_code=True")
tokenizer = AutoTokenizer.from_pretrained("lightonai/RITA_s")

for generation we support pipelines:

from transformers import pipeline
rita_gen = pipeline('text-generation', model=model, tokenizer=tokenizer)
sequences = rita_gen("MAB", max_length=20, do_sample=True, top_k=950, repetition_penalty=1.2, 
                     num_return_sequences=2, eos_token_id=2)
for seq in sequences:
    print(f"seq: {seq['generated_text'].replace(' ', '')}")

Or see example.py

How to cite

@article{hesslow2022rita,
  title={RITA: a Study on Scaling Up Generative Protein Sequence Models},
  author={Hesslow, Daniel and Zanichelli, Niccol{\'o} and Notin, Pascal and Poli, Iacopo and Marks, Debora},
  journal={arXiv preprint arXiv:2205.05789},
  year={2022}
}

RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Related tags

Overview

RITA: a Study on Scaling Up Generative Protein Sequence Models

Results

Usage

How to cite

Owner

LightOn

Codebase for the self-supervised goal reaching benchmark introduced in the LEXA paper

Detection of drones using their thermal signatures from thermal camera through YOLO-V3 based CNN with modifications to encapsulate drone motion

Code for our ICCV 2021 Paper "OadTR: Online Action Detection with Transformers".

CURL: Contrastive Unsupervised Representations for Reinforcement Learning

AI Virtual Calculator: This is a simple virtual calculator based on Artificial intelligence.

Cross-Modal Contrastive Learning for Text-to-Image Generation

Simple improvement of VQVAE that allow to generate x2 sized images compared to baseline

The code for 'Deep Residual Fourier Transformation for Single Image Deblurring'

The "breathing k-means" algorithm with datasets and example notebooks

Simple data balancing baselines for worst-group-accuracy benchmarks.

A PyTorch Implementation of Single Shot MultiBox Detector

Hierarchical User Intent Graph Network for Multimedia Recommendation

Semantic Image Synthesis with SPADE

Sequence-tagging using deep learning

Massively parallel Monte Carlo diffusion MR simulator written in Python.

State-of-the-art language models can match human performance on many tasks

Collection of tasks for fast prototyping, baselining, finetuning and solving problems with deep learning.

WSDM2022 "A Simple but Effective Bidirectional Extraction Framework for Relational Triple Extraction"

Explicable Reward Design for Reinforcement Learning Agents [NeurIPS'21]

Stroke-predictions-ml-model - Machine learning model to predict individuals chances of having a stroke