LoongForge

LoongForge is a software framework that improves the training speed and efficiency of Vision-Language Agent models such as Pi0.5. It applies various optimization techniques including dynamic token pruning, partial compilation with torch.compile, custom attention (FlexAttention), selective recomputation, and layered compilation to overlap communication with computation, achieving up to 3x acceleration in training throughput without loss in model quality.