Code for Massive-scale Decoding for Text Generation using Lattices

Last update: Dec 18, 2022

Related tags

Overview

Massive-scale Decoding for Text Generation using Lattices

TL;DR: a new search algorithm to construct lattices encoding many generation options; two key technical contributions: (1) best-first search, (2) path recombination.

Visualization

We provide a few examples in the vis folder and on my homepage. You need to download the html files to view and interact with the model outputs.

The complete set of outputs are available on Box.

Getting started

model contains all of the methods, including baselines like beam search, nucleus sampling, and our methods.
evaluation contains scripts for evaluation.
command are the prompts and shells we use to run the experiment.

Beam Search:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4  -beam_size 16 -min_len 10 -max_len 35   -model bs

Best-first Search:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4  -beam_size 16 -min_len 10 -max_len 35   -model astar_baseline

Best-first Search with Recomb:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4 -beam_size 16 -min_len 10 -max_len 35 -model astar -merge imp  -avg_score 0.75  -adhoc

Best-first Search with Zip:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4 -beam_size 16 -min_len 10 -max_len 35 -model astar -merge zip  -avg_score 0.75  -adhoc

More detailed instructions coming soon!

Citation

@misc{xu-durrett-2021-massive,
    title={Massive-scale Decoding for Text Generation using Lattices},
    author={Jiacheng Xu and Greg Durrett},
    year={2021},
    eprint={2112.07660},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

Contact

[email protected]

Code for Massive-scale Decoding for Text Generation using Lattices

Related tags

Overview

Massive-scale Decoding for Text Generation using Lattices

Visualization

Getting started

Citation

Contact

Owner

Jiacheng Xu

This repo holds code for TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

Code release for NeurIPS 2020 paper "Co-Tuning for Transfer Learning"

Differentiable molecular simulation of proteins with a coarse-grained potential

🎁 3,000,000+ Unsplash images made available for research and machine learning

A real world application of a Recurrent Neural Network on a binary classification of time series data

A python3 tool to take a 360 degree survey of the RF spectrum (hamlib + rotctld + RTL-SDR/HackRF)

Implementation of FitVid video prediction model in JAX/Flax.

A simple baseline for the 2022 IEEE GRSS Data Fusion Contest (DFC2022)

ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction

Data and analysis code for an MS on SK VOC genomes phenotyping/neutralisation assays

(EI 2022) Controllable Confidence-Based Image Denoising

S2s2net - Sentinel-2 Super-Resolution Segmentation Network

We present a regularized self-labeling approach to improve the generalization and robustness properties of fine-tuning.

Personal thermal comfort models using digital twins: Preference prediction with BIM-extracted spatial-temporal proximity data from Build2Vec

PyTorch implementation code for the paper MixCo: Mix-up Contrastive Learning for Visual Representation

Source code for models described in the paper "AudioCLIP: Extending CLIP to Image, Text and Audio" (https://arxiv.org/abs/2106.13043)

Pre-trained Deep Learning models and demos (high quality and extremely fast)

Computer Vision and Pattern Recognition, NUS CS4243, 2022

Patch2Pix: Epipolar-Guided Pixel-Level Correspondences [CVPR2021]

[CVPR2021 Oral] UP-DETR: Unsupervised Pre-training for Object Detection with Transformers