Lifeboat

Lifeboat lets users run large language models on their own hardware (macOS, Windows, Linux, Docker, Kubernetes) behind an OpenAI- and Anthropic-compatible API. It supports GPU acceleration and offers a desktop app as well as Docker and Kubernetes install methods, enabling control of ML models without sending data off-network.