PicoLM

PicoLM is a lightweight, cross-platform inference engine implemented in C and GPU backends, supporting numerous large language models with various quantization techniques. It aims for portability across many systems from MS-DOS to RTX GPUs, optimized CPU and GPU performance, and supports advanced features like HTTP server for OpenAI-compatible APIs, persistent threading, and model streaming from disk to minimize RAM usage.