TurboGPT

TurboGPT is an open-source tool that enables training of tiny GPT transformer models quickly using NVIDIA CUDA GPUs. It is implemented in CUDA C++ and supports Linux, NixOS, and Windows platforms. The tool provides features like checkpointing, TensorBoard-compatible logging, and is designed for rapid training on small-scale GPT models.