Data-Driven Over LLM-Driven: Where AI Belongs in Production
LLMs are powerful and unreliable. Keep critical paths deterministic and put AI only at the boundaries.
AI is genuinely useful, but the fastest way to ship an unreliable product is to put a large language model on a critical path. LLMs are non-deterministic, occasionally wrong with total confidence, and hard to test. My rule: keep critical paths data-driven, and use LLMs only at the boundaries.
On a betting-analytics platform I built, the prediction engine is entirely deterministic — Poisson expected goals, Elo ratings, odds de-overrounding, and a consensus layer. Every number is reproducible and unit-testable. There is no LLM anywhere near a figure a user might act on.
Where LLMs earn their place is at the edges: turning a messy free-text inquiry into a structured route, summarizing, or drafting copy a human reviews. Even there, I force structured output — a strict schema the model must fill — so the rest of the system receives typed data, not prose to parse.
Guard against hallucination and cost. Validate every model response against a schema and reject what does not fit; cache aggressively; set timeouts and fallbacks so a slow or failed call degrades gracefully instead of breaking the request. Treat the model as an unreliable external service, because that is what it is.
The test is simple: if a wrong answer would corrupt data or mislead a user, it does not belong to the model — it belongs to deterministic code. Use AI to improve the experience around the edges, not to make decisions you cannot verify.