GridCore

GridCore is a scheduler that manages a single GPU’s memory and usage among multiple local large language models (LLMs) running concurrently, prioritizing interactive requests over background tasks, estimating VRAM needs precisely, and controlling model loading/unloading to avoid crashes and improve responsiveness on real hardware.