SpectraAdamW

SpectraAdamW is a hybrid optimizer that halves the optimizer state memory required for fine-tuning large language models by using factorized variance approximations and frequency-domain momentum compression while preserving convergence behavior. It serves as a drop-in replacement for torch.optim.AdamW and aims to enable efficient LLM training on constrained VRAM hardware.