RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Last update: Dec 22, 2022

Overview

RITA: a Study on Scaling Up Generative Protein Sequence Models

RITA is a family of autoregressive protein models, developed by a collaboration of Lighton, the OATML group at Oxford, and the Debbie Marks Lab at Harvard.

Model	#Params	d_model	layers	lm loss uniref-100
Small	85M	768	12	2.31
Medium	300M	1024	24	2.01
Large	680M	1536	24	1.82
XLarge	1.2B	2048	24	1.70

Results

For full results see our preprint: https://arxiv.org/abs/2205.05789

Usage

Instantiate a model like so:

from transformers import AutoModel, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("lightonai/RITA_s, trust_remote_code=True")
tokenizer = AutoTokenizer.from_pretrained("lightonai/RITA_s")

for generation we support pipelines:

from transformers import pipeline
rita_gen = pipeline('text-generation', model=model, tokenizer=tokenizer)
sequences = rita_gen("MAB", max_length=20, do_sample=True, top_k=950, repetition_penalty=1.2, 
                     num_return_sequences=2, eos_token_id=2)
for seq in sequences:
    print(f"seq: {seq['generated_text'].replace(' ', '')}")

Or see example.py

How to cite

@article{hesslow2022rita,
  title={RITA: a Study on Scaling Up Generative Protein Sequence Models},
  author={Hesslow, Daniel and Zanichelli, Niccol{\'o} and Notin, Pascal and Poli, Iacopo and Marks, Debora},
  journal={arXiv preprint arXiv:2205.05789},
  year={2022}
}

RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Related tags

Overview

RITA: a Study on Scaling Up Generative Protein Sequence Models

Results

Usage

How to cite

Owner

LightOn

Pytorch implementation for "Implicit Semantic Response Alignment for Partial Domain Adaptation"

Main repository for the HackBio'2021 Virtual Internship Experience for #Team-Greider ❤️

Medical-Image-Triage-and-Classification-System-Based-on-COVID-19-CT-and-X-ray-Scan-Dataset

Greedy Gaussian Segmentation

Author's PyTorch implementation of Randomized Ensembled Double Q-Learning (REDQ) algorithm.

[CVPR 2022 Oral] TubeDETR: Spatio-Temporal Video Grounding with Transformers

Few-shot Relation Extraction via Bayesian Meta-learning on Relation Graphs

Code for KHGT model, AAAI2021

Space Time Recurrent Memory Network - Pytorch

Run Effective Large Batch Contrastive Learning on Limited Memory GPU

Pytorch implementation of SimSiam Architecture

A Japanese Medical Information Extraction Toolkit

A simple editor for captions in .SRT file extension

A collection of resources, problems, explanations and concepts that are/were important during my Data Science journey

S-attack library. Official implementation of two papers "Are socially-aware trajectory prediction models really socially-aware?" and "Vehicle trajectory prediction works, but not everywhere".

Additional functionality for use with fastai’s medical imaging module

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems

Compact Bilinear Pooling for PyTorch

An implementation of the AdaOPS (Adaptive Online Packing-based Search), which is an online POMDP Solver used to solve problems defined with the POMDPs.jl generative interface.