HoldemBench

HoldemBench is a benchmark tool and dataset for evaluating decision-making AI models in heads-up no-limit Texas hold’em poker. It enables side-by-side comparisons of various AI agents, including decision models, chat LLMs, scripts, and local models, by playing thousands of poker hands with standardized prompts and scoring methods that remove luck. It provides insights into AI strategic play, reasoning accuracy, and cost-effectiveness in poker decision-making.