Build-GPT-2

Build-GPT-2 is a compact engineering codebase for training the GPT-2 124M model from scratch in PyTorch. It features explicit data sharding, multi-GPU training with DistributedDataParallel, resumable checkpoints, integrated evaluation on HellaSwag, and optional Weights & Biases logging, all implemented in around 1,600 lines of code. It is designed as a learning and development tool for understanding GPT-2 training workflows rather than a high-level trainer.