MTG Commander AI

A conversational agent that builds and tunes full 100-card Commander decks by calling a live MCP tool server over 33,000+ constantly-changing Magic cards, tournament results, and 10+ podcast and YouTube shows.

Live site

Why this stack

The backend and knowledge pipeline are Python: the raw material is 33K+ cards from Scryfall plus tens of thousands of podcast and YouTube transcript chunks, and Python's ecosystem for batch LLM calls, embeddings, and direct Postgres access fit that better than anything else considered. The web frontend is SvelteKit with the Vercel AI SDK, not Next.js. The project's own feature-branch folder is literally named 026-nextjs-frontend, a name left over from the initial plan before SvelteKit's smaller client bundle and native streaming primitives won the actual build. The 'Vercel' in the SDK's name is just a library brand, not the deploy target: both the SvelteKit frontend and the Python backend deploy to Railway via a Node adapter, not to Vercel. Supabase Postgres with pgvector holds the 33K-card embedding index and the structured-insight knowledge base in the same database that already handles auth and conversation storage, instead of standing up a separate vector store for 1 extra table. FastMCP exposes the deck-building tool surface (search_cards, find_similar_cards, build_commander_deck, and more) as the same interface any MCP client, not just this web app, could connect to directly.

AI-assist note

It's spec-driven, built with Claude Code across 164 specs (spec, plan, tasks, and research docs per feature, with a dedicated research pass on the trickier ones like the deck-construction engine and the fabrication gate). I wrote every spec and reviewed every diff; Claude Code wrote most of the implementation and tests. The thesis-aware deck engine, the color-identity validation, the fabrication gate, and the tool-routing audit below all came out of that same spec-plan-implement-review loop, not a rewrite from scratch.

Stack

Python 3.12
FastMCP
Anthropic Claude (Opus for extraction, Sonnet Batch API for validation)
OpenAI text-embedding-3-small
Supabase (Postgres + pgvector + Auth)
SvelteKit + Vercel AI SDK v6
Mana Pool (affiliate) + Patreon + Ko-fi
PostHog (product analytics)
Railway

Domains

Agentic AI
Constraint-Based Deck Construction
RAG & Knowledge Systems
AI Safety & Grounding
Applied AI Evaluation

Live4 mo

Users570 registered users, 2,271 decks built, 2,490 chat conversations (Supabase, Jul 2026)

PaymentsFree forever; Patreon + Ko-fi + Mana Pool affiliate (1 paying patron at $20/mo)

InfraPython 3.12, Supabase Postgres + pgvector, FastMCP, SvelteKit + Vercel AI SDK v6, Railway, Axiom + Discord ops alerts

AuthSupabase Auth (Google, Discord, Twitch, email) + RLS per user

CommunitySmall Discord community, seeded from the Mana Pool partnership

Specs164

Why this exists

A friend of mine, Wes, was building a homebrew Magic: The Gathering cube by hand. He had 10 defined archetypes and a card list, but no easy way to check whether his black-green reanimator support was actually deep enough or just felt deep. Commander deckbuilding has the same problem at a bigger scale. EDHREC gives you popularity data, not the reasoning a good player would give you for why a specific card fits your specific 99. mtgcommander.ai is a conversational assistant that reads real card data (33K+ Scryfall cards) and real expert reasoning (transcript chunks from 10+ podcast and YouTube shows), builds full Commander decks against a stated budget and bracket, and hands the actual purchase off to Mana Pool through an affiliate link. It runs as a live, free product today, not a prototype. I tried a paid subscription tier first, couldn’t find a version of it worth charging for, and pivoted to Patreon after a couple of months, so it’s funded now by Patreon and Ko-fi support instead of a paywall.

Architecture

One box-wall diagram can’t hold this system honestly, so here it is as the four flows it actually runs, plus the operations layer that keeps a one-person product alive: a chat turn, the inside of a deck build, the handoff from deck to real cart, the knowledge pipeline behind it all, and how the whole thing watches itself in production.

The chat agent loop

A chat turn is an agent loop: the SvelteKit server runs the Vercel AI SDK with Claude Sonnet as the chat model (a failover model and a per-request bake-off override are each a single env flip), and the model works the FastMCP tool surface, the same tools any MCP client could call directly. Which tools the model even sees is decided per turn by a workflow state machine informed by the 293-call usage audit: paste in a 30-card list and it detects a net-deck import and swaps in the orient/swap/buylist tools; ask a fresh question and those stay out of the model’s way. Every card name in the streamed answer is resolved against the real catalog before the reader sees it: the fabrication gate from the skill stories below.

Inside a deck build

build_commander_deck is a staged engine, not one model call. The LLM owns the two genuinely creative stages, reading the thesis and choosing the specific cards that serve it, and deterministic code owns the solved problems: structural role counts from YAML package templates, declared-minimum conformance enforced through an explicit swap plan, a mana base audited pip-by-pip against the colors the deck actually needs, and a fabrication gate that refuses any card name that doesn’t resolve in the catalog. Every stage reports its own status into the build metadata, so a degraded stage is visible in the result instead of silently swallowed.

From deck to cart: the Mana Pool handoff

When a reader clicks Buy, the deck goes to Mana Pool’s cart optimizer for live pricing and availability (if a printing is out of stock, the request retries with it dropped instead of failing the whole cart), then through deck validation (legality, color identity, quantities) and comes back as a pre-filled cart URL carrying my affiliate ref. Payment completes on Mana Pool; there is no payment surface in this app at all. The engine can also build against a constrained pool, a single seller’s inventory or the user’s own collection, so the deck it hands over is actually fulfillable. That affiliate cut, plus Patreon and Ko-fi support, is the entire business model: the product is free forever, and the support links live on a support page, not in this pipeline.

The knowledge pipeline (offline)

Everything the agent knows arrives through a batch pipeline, deliberately not real time: validating a new insight against a card’s oracle text before a user can see it matters more than shipping it a few minutes faster. Scryfall card data refreshes on a nightly cron and the ops audit auto-heals any gaps it finds; transcripts go through Whisper, then a Claude extraction pass that discards noise and pulls card-linked insights, then the oracle-text validator that flagged 8,380 hallucinated mechanic claims across the backfill and now gates every new insight the same way. It all lands in the one Supabase Postgres database: cards, insights, embeddings, decks, and conversations together, not a separate vector store.

Production operations

This is the part that doesn’t demo well but matters most: the product runs itself like a team several times its size. Every PR passes CodeRabbit’s assertive review profile and CI before Railway deploys the two services, and production ships structured logs to Axiom with a dead-man heartbeat so even a dead logger gets noticed. Then the watchers take over. A nightly ops audit checks catalog completeness against Scryfall and auto-heals gaps, has a small model read the previous day’s conversations and file deduplicated GitHub issues for real failures (that audit is where the fabrication-gate work came from), and scans Axiom for error clusters. An hourly synthetic probe runs a scripted conversation end to end, and deck-quality evals run nightly with an LLM-judged scorecard three times a week. Everything reports to a Discord alerts channel I actually read, every cron pings Healthchecks.io, and the issues the audit files feed straight back into the spec queue.

What shipped

By spec 164 the platform had grown from a favor for Wes’s cube into a real free product: a Mana Pool affiliate checkout handoff, a small Discord seeded from a card-marketplace partner, and a knowledge base that validates itself before storing anything new. The hallucination cleanup alone put 33,040 insight-card pairs through oracle-text validation, flagging 8,380 hallucinated mechanic claims, and left the extraction pipeline validating every new insight against oracle text before storage, not just the backfill. The tool-routing audit replaced guesswork with 293 real tool calls across 50 conversations, consolidating or rerouting 12 tools that had 0 calls in the sample. All of these fixes came out of the same spec, plan, tasks, and research loop with Claude Code, not a rewrite from scratch.

The extraction pipeline still runs as a batch process, not real time. That’s deliberate: validating a new insight against a card’s oracle text before it ever reaches a user matters more than shipping it a few minutes faster.

Skill stories

Each card below opens the engineering story behind one skill: the decision I made, what broke, how I measured it, and how I fixed it. Click any card to read it.

Applied AI & Constraint SolvingThesis-Aware Deck Construction Engine
Applied AI & Domain ModelingColor-Identity and Mana-Base Correctness
AI Safety & GroundingCard-Fabrication Gate (Catalog-as-Arbiter)
Agentic AIMCP Tool Server and Usage-Driven Routing
RAG & Knowledge SystemsRAG Knowledge Base and Insight Validation
Applied AI & PersonalizationUser Memory and Personalization
Applied AI & EvaluationModel Bake-Offs and Failover
Production Cost GovernancePer-User Cost Caps for a Free Product
Testing & CI ReliabilityTest-Suite Economics: Recorded LLM Calls in CI

Applied AI & Constraint Solving

Thesis-Aware Deck Construction Engine

Decision: EDHREC gives you a commander's most popular cards, not a deck built for your actual strategy. So a build isn't 1 LLM call, it's a multi-stage engine: it reads the user's thesis and any must-include cards, reasons about the strategy, assembles a candidate pool, fills structural roles (ramp, removal, card draw, board wipes, tutors, protection) from package templates I borrowed from my Ansible and network-automation background, then builds a real mana base, all gated by a conformance validator before the deck is ever shown.
What broke: Letting the model generate a whole 100-card list from scratch produced invalid or incoherent decks about 20% of the time: cards that don't exist, off-strategy popular picks, and slot-count violations. For a request like 'Atraxa focused on life drain with Exsanguinate and Vito,' pure EDHREC synergy returned the generic blink-value Atraxa pile and ignored the stated drain plan.
How I measured it: I split infrastructure correctness from strategic quality, the way you separate 'is this a valid router config' from 'is this a good one.' Structural packages, land base, and slot counts get deterministic validation, a commit-dry-run gate; strategic fit gets the AI reasoning. Data enrichment happens before the model is involved: canonical card data plus community tags plus usage context, with AI classification of the top 10,000 cards done once for about $112.
How I fixed it: Deterministic templates own the solved problems (every deck gets its ramp, removal, and draw counts and a mana base that fits) and the AI owns the creative call of which specific cards serve the thesis. The result respects the stated strategy and include-cards instead of defaulting to popularity, and it can build from a constrained pool too, a single seller's inventory or the user's own collection, so the deck is actually buyable. I wrote up the template-engine design here: https://sierracodeco.com/blog/jinja-templates-ai-agents/

Applied AI & Domain Modeling

Color-Identity and Mana-Base Correctness

Decision: A deck can look great and still be unplayable if its lands can't cast its spells. Before returning a deck, the engine counts the color pips the deck actually needs, checks the land base against them, and fixes any mismatch by swapping lands for the right colors.
What broke: A generated Isshin deck (white, red, black) was heavy on black pips, double-black costs like Necropotence and Phyrexian Arena, but its land base had grabbed mostly red-white duals and colorless utility lands: 25 black pips against only 8 black sources when it needed roughly 14. It's invisible to a casual player and immediately obvious to an experienced one, which is exactly the audience the product is for.
How I measured it: The check is a real count, not a vibe: pips required per color against sources available per color, with a target ratio for each color's presence in the deck.
How I fixed it: When the land base underserves a color, the engine swaps colorless utility lands for basics of that color and tapped duals for untapped duals in the right colors, so the deck can cast its spells on curve. Color-identity correctness is the difference between a deck that reads well and a deck that actually functions.

AI Safety & Grounding

Card-Fabrication Gate (Catalog-as-Arbiter)

Decision: The single biggest cluster in the nightly ops audit was the chat assistant confidently naming Magic cards that do not exist, often with fabricated rules text. Every card the product mentions has to resolve to a real card, so I built a self-correcting fabrication gate that treats the actual card catalog, not an LLM judge, as the arbiter of what is real.
What broke: The model would produce confident, uncorrected fabrications: names like Turn Krenko, Seri Inseparable, Avatar Aang, and Dragon of Mount Gulg, sometimes with invented abilities. The best the old system did was a muted could-not-verify badge, and only when a detector happened to catch the invented span. A wholesale invented name the detector never saw shipped straight to the user.
How I measured it: The audit surfaced fabricated card names as the largest recurring failure cluster, so I made the fix catalog-driven and deterministic instead of asking a second model to grade the first: entity-link every card the assistant names against the real catalog, and treat anything that doesn't resolve as a fabrication to correct, not a judgment call.
How I fixed it: The gate self-corrects a fabricated card in the live answer by resolving names against the catalog, so an invented card gets caught and replaced with a real one instead of shipping a confident hallucination. Grounding to the catalog, not to a second model's opinion, keeps the check cheap, deterministic, and hard to argue with.

Agentic AI

MCP Tool Server and Usage-Driven Routing

Decision: The conversational agent is a FastMCP tool server exposing the deck-building surface (search_cards, get_card, find_similar_cards, build_commander_deck, search_expert_knowledge, and more) as the same interface Claude Desktop or any other MCP client could connect to directly, not a bespoke API only the web chat can call. After launch, instead of guessing which tools the model actually needed, I audited real usage: 293 tool calls across 50 logged conversations.
What broke: The audit showed a long tail of dead weight: 12 of the 30 exposed tools recorded 0 calls across all 50 conversations, while search_cards and get_card alone accounted for 160 of the 293 calls (55%). A flat, always-available tool list makes tool selection noisier for the model and gives no signal about which capability is actually doing anything.
How I measured it: Built a frequency table (calls and percentage per tool) directly from the 50-conversation sample, then classified each tool as keep, merge, internalize, or needs-routing based on that data instead of intuition. search_expert_knowledge had 16 calls but got reclassified INTERNALIZE, since it turned out to be called by build_commander_deck and brainstorm_cards internally, never directly by the model in response to a user ask.
How I fixed it: 0- and near-0-call tools got consolidated by their real verdict: explore_commanders (4 calls) merged into search_commanders_by_strategy, get_card_price (1 call) merged into price_card_list, and suggest_creator_decks (0 calls) got removed as a standalone tool since brainstorm_cards already baked it in. The remaining 0-call tools (rules lookups, post-build analysis) moved behind precondition-based routing (today a workflow state machine that detects intent and gates which tools are live each turn), so they only surface once their precondition step has actually run instead of sitting in front of the model on every single turn.

RAG & Knowledge Systems

RAG Knowledge Base and Insight Validation

Decision: The expert knowledge base started as 32,807 raw podcast and YouTube transcript chunks retrieved by plain similarity search. I added a structured extraction layer, a Claude pipeline that discards noise (sponsor reads, intros, tangents) and pulls card-linked insights tagged with color identity, archetype, and a recency era, then validated every extracted insight against the card's real oracle text before it could be trusted.
What broke: Two failures. Raw-chunk similarity returned fragments about the wrong commander, a sponsor read, and a personal anecdote for a specific-commander query. And the extracted insights themselves included hallucinations, an insight claiming a card had an ability it doesn't have, which an earlier regex validator couldn't catch because it matched a mechanic keyword anywhere near a card name (it flagged 'ward' inside 'reward').
How I measured it: I replaced the regex check with a Claude Sonnet Batch validator that reads each insight next to the card's actual oracle text and labels it VALID, HALLUCINATION, or OPINION. Submitting 33,040 insight-card pairs cost about $15, then I manually sampled the flags to separate real mechanic errors from stat claims that were merely outside the oracle text.
How I fixed it: Extraction turned 32,807 raw chunks into 41,790 structured insights. The validation pass over 33,040 insight-card pairs came back 20,051 valid, 8,380 flagged as hallucinated mechanic claims, and 4,609 set aside as subjective opinion rather than rules fact, and a follow-up review pass separated the genuine mechanic errors from false positives before anything was purged from retrieval. Recency weighting ranks post-2024 bracket-era content above older advice, and every new insight now runs the same validation before it is stored.

Applied AI & Personalization

User Memory and Personalization

Decision: A Patreon supporter asked whether the AI could learn from his interactions and remember his personal deck-building style. I built a per-user memory layer: explicit facts that survive across chat sessions, surfaced to the model on each turn by semantic retrieval, and managed by the AI itself through 3 tools, remember, forget, and list_memories.
What broke: Without memory, every session started cold. A user who always builds on a budget, avoids a particular card, or prefers a certain archetype had to re-explain himself every time, and the assistant couldn't act like it knew him.
How I measured it: The design centered on the one outcome that mattered: tell the AI a preference in 1 session and see it applied in the next. Memories are retrieved semantically so a relevant preference surfaces even when the user doesn't restate it, and the AI decides when to record, update, or drop a fact rather than storing everything.
How I fixed it: The assistant now remembers per-user preferences across sessions and applies them without being re-told. This was exactly what that Patreon supporter's first feature request pointed at, and after it shipped he raised his pledge from $10 to $20 a month.

Applied AI & Evaluation

Model Bake-Offs and Failover

Decision: Picking the chat model shouldn't be a preference. I built a bake-off that replays the same real production prompts through staging against each candidate I hold keys for (Gemini, DeepSeek, GPT), measures quality, speed, and cost on identical inputs, and produces a scorecard I read to set the default, with the swap itself de-risked down to a single env flip.
What broke: The chat orchestrator ran on 1 model by default, with no evidence base for whether a cheaper or faster provider would hold quality or what each provider's latency-versus-quality knob actually bought. Separately, a hard dependency on any single provider meant a deprecation or outage could take the assistant down.
How I measured it: The bake-off runs the same prompts through each candidate and scores them with Claude as the judge, unbiased precisely because Claude is not a contestant in this comparison. It reports quality, speed, and cost per candidate on identical inputs and explores each provider's latency-quality knob, so the decision is evidence, not vibes.
How I fixed it: The output is a scorecard, not an automatic production change, so acting on it is a deliberate env flip rather than a silent swap. Paired with a model-deprecation failover path, the assistant can move off a provider that gets deprecated or degrades without a rewrite.

Production Cost Governance

Per-User Cost Caps for a Free Product

Decision: A free product with real LLM costs behind every chat turn needs a way to be generous without being abusable. Every user's actual model spend is tracked in cents as it happens, and a flat monthly cost cap is enforced server-side before the model runs, for every user, after I retired the paid tiers. Free doesn't mean unmetered.
What broke: The cap fired in production twice for the wrong reasons. First, the monthly reset for the cost counter was never actually implemented, so the '$5/month' cap silently behaved as a $5 lifetime cap: any loyal user would eventually cross it and be permanently rate-limited, with no path back. Then, after a model swap, a mispriced entry in the cost table over-billed real usage, and my one paying Patreon supporter hit the cap even though his genuine usage shouldn't have come anywhere near it.
How I measured it: The fix was sized from the ledger, not a guess: the maximum observed genuine single-user monthly spend was about $1.72, so the cap moved to $20, roughly 11.6 times the observed max. A real power user should effectively never feel it, while a runaway loop or a deliberate abuser still hits a hard wall.
How I fixed it: The monthly reset now actually resets (UTC calendar rollover, with build counts and cost resetting independently and legacy rows handled), the pricing table got corrected after the bad swap, and the patron got a personal follow-up, an explanation of exactly what happened, and my assurance he'll never be capped again. Cost governance is a user-experience problem too: the person funding the product should never be the one throttled by its safety rail.

Testing & CI Reliability

Test-Suite Economics: Recorded LLM Calls in CI

Decision: AI-assisted development writes tests faster than any team I've worked on, and the suite and CI wall-clock balloon accordingly. I've done several rounds of test-diet work across this project and NovelFlame, and the structural fix here was record/replay: the deck-quality and MCP integration tests run against recorded HTTP cassettes of real LLM and data-API traffic, so CI replays captured responses with dummy keys instead of paying for live calls.
What broke: Early on, the integration and smoke tests hit live LLM and data APIs on every change: real dollars and minutes of wall-clock per run, multiplied by how fast the spec-driven loop pushes changes, and flaky on top because live model output is nondeterministic. The cost of validating a change was quietly taxing every single iteration.
How I measured it: CI now defaults to replay: deterministic, free, and fast, against 11 cassettes of captured traffic. Matching stays stable because a custom matcher pins the recorded OpenAI tool-forcing calls, so a prompt refactor doesn't silently invalidate the cassette library.
How I fixed it: Live API coverage became a decision instead of a tax: adding a run-live label to a PR (or a manual dispatch) flips the recorder into rewrite mode and re-records the cassettes against the real APIs. Day-to-day pushes validate against reality's last known recording; spending money on the real thing happens when a change actually warrants it.