LLMxRay

LLMxRay is an open-source observability and benchmarking tool for local large language models (LLMs) running with Ollama. It provides real-time token streaming with confidence coloring, performance profiling, response quality analysis, multi-model comparison, embeddings visualization, cost tracking, and a knowledge base for retrieval-augmented generation. It enables developers and researchers to inspect and analyze their local LLM behavior without cloud or API keys.