Spectra-FFT

Spectra-FFT is a drop-in replacement optimizer for PyTorch models that reduces the memory required for training large language models by over 50%, allowing full-parameter training on memory-constrained GPUs. It replaces the AdamW optimizer state with a compact frequency-domain representation, enabling training on smaller or existing hardware without compromising convergence or model fidelity.