BigMoeOnEdge

BigMoeOnEdge is an adaptive streaming optimizer that enables running large Mixture-of-Experts AI models exceeding device RAM limits on phones, laptops, and edge devices without loss of output fidelity. It runs on any hardware and OS, supporting CPU, GPU, and NPU backends, any model format compatible with llama.cpp, and any quantization. It acts as a local OpenAI-compatible server allowing AI chat clients and coding agents to operate fully locally.