Narwhal

Narwhal is an adaptive inference framework that dynamically reallocates GPU roles between prefill and decode in real-time to optimize large language model serving. It enables scalable serving from a single GPU to multi-node clusters with role hot-swapping without reloading model weights, latency-aware admission, and fault tolerance, targeting efficient LLM deployment.