Simple Annotated implementation of GPT-NeoX in PyTorch

This is a simpler implementation of GPT-NeoX in PyTorch. We have taken out several optimizations in GPT-NeoX for simplicity.

Annotated implementation

Sample usages

• Notebook to download checkpoints and test the model
• Fine-tuning example
• Text generating on multi GPU
• 🚧 Text generating on a single GPU