Dungeon Master

AI Dungeon Master runs an interactive fantasy adventure where Claude narrates the story, but every action in the world goes through a deterministic Python game engine that owns the rules, the dice and the state. The model can only propose structured actions through tool calls; the engine validates them, decides what is legal, and returns the ground truth for the model to narrate back. That separation keeps the AI from hallucinating outcomes and makes the system testable, which is the part most agentic AI projects skip. The backend is Python and FastAPI with Claude tool use and Pydantic state, and the frontend is a Next.js and Tailwind play surface. There is no live demo yet; deployment is on the roadmap.

Anyone can wire an LLM to a chat box, but a Dungeon Master is a non-deterministic, multi-turn, stateful agent, and letting the model own the world means it will happily fake outcomes: resurrect a slain NPC, walk through a locked door, or invent a dice roll. The harder problem is that an agent like this has no single correct output, so most projects never really evaluate it. I wanted to build one that was both grounded in a real ruleset rather than the model's imagination, and verifiable through an evaluation suite I could actually trust.

I kept the game engine completely LLM-free: rules, dice, combat and constraints live in plain Python, world state is modelled with Pydantic, and adventures are authored as data the engine never mutates. The DM layer is a Claude tool-use loop that turns free text like "grab the key, then open the door" into validated engine calls, then narrates whatever the engine reports back. Illegal actions are refused by the engine and narrated as refusals, not faked. FastAPI exposes start-a-game and take-a-turn endpoints, and the Next.js frontend renders a scrolling story log with a live status panel driven straight off the game state.

The key decision was keeping the engine deterministic and LLM-free so the rules are unit-testable, the dice are seedable, and the evals are reproducible, none of which holds if the model owns the state. That let me build the eval harness in two buckets. Objective checks drive scripted playthroughs and assert on things with a genuine right answer: taken items persist, slain NPCs stay dead, locked doors block, and every emitted state validates against the model, all running deterministically in CI with no API key. Prose quality is scored separately by an LLM-as-judge on a rubric, kept advisory so it never gates CI.

This project taught me how to evaluate a non-deterministic system without pretending it is deterministic: separate what has a right answer from what is a matter of taste, check the former with hard assertions, and keep the latter advisory. Drawing a firm line between a deterministic core and the model, where the engine owns state and the LLM only proposes, is what made the whole thing testable, and seedable dice were the small detail that made the evals reproducible. I also got a clearer feel for designing a tool-use loop, where the model's job is to interpret intent and narrate ground truth rather than decide outcomes.