Phobos

Phobos is an open-source LLM inference stack and tiny kernel language that compiles kernels at runtime to run large language models efficiently on GPU hardware, including older GPUs like the RTX 2080 SUPER. It supports quantized models and competes closely with other inference frameworks such as llama.cpp, focusing on speed optimizations and caching strategies to maximize memory bandwidth and execution efficiency.