A client asks for “AI” in a feature more often than they ask for a specific capability. That’s backwards. AI is a technique with real costs, in latency and in a bill that scales with usage. Before we let a model near a feature, we run the same five questions, in the same order, on our AI work and everyone else’s. Fail one, and the model isn’t ready yet, whatever the sales pitch says.
Is this actually probabilistic, or is it deterministic wearing a costume?
A lot of “can you make this AI-powered” requests turn out to be a lookup or a calculation someone hasn’t written down as a rule yet. If the same input should always produce the same correct output, a model is the wrong tool for it, because even a good one won’t reliably reproduce that. Anthropic’s own Messages API documentation says as much: even at a temperature of 0.0, the results are not fully deterministic. That’s inherent to how a model samples text, regardless of vendor.
A shipping ETA based on zone and weight, or a discount that applies once a cart total crosses a threshold: both of those are a function, plain code with no model involved.
function shippingEta(zip: string, weightKg: number) {
const zone = zoneForZip(zip)
const transitDays = ZONE_DAYS[zone] ?? 5
return transitDays + Math.ceil(weightKg / 5)
}The background worker that keeps FurnitureAxis’s vendor catalog in sync with Shopify and Algolia runs on a cron schedule and matches records deterministically. Handing that job to a model would make it slower and harder to debug at 2am, for no gain. Plain code is the right choice here.
What does a wrong answer cost?
If the task clears the first question, the next one is about consequences. A model that occasionally writes an awkward product description costs you an edit, a few minutes, nothing more. A model that auto-approves a refund or deletes the wrong record costs you something you can’t get back, and the fact that the model is usually right doesn’t help the one customer it gets wrong.
The fix is usually matching the amount of human oversight to the size of the mistake. A drafting task can run unsupervised because a person reviews the output before it goes anywhere. An action with financial or irreversible consequences needs a human in the loop, or at minimum a confirmation step, until the model has a track record that justifies removing one.
Is there real data to ground the answer in?
A model answering from data you handed it is a different product than a model answering from whatever it absorbed during training. The first is retrieval: you give it your actual product catalog and your actual support history, and it answers from those. The second is a guess dressed up as an answer. Ask an ungrounded model about a return policy or a delivery estimate and it will produce something that reads like a fact, with no connection to the policy you actually run. A confident-sounding guess is worse than no answer at all, because the person reading it can’t tell the difference.
If there’s no clean source of truth for the feature to draw on, that’s the real project. Building the retrieval pipeline and the data hygiene that supports it is most of what we describe on the AI integration page, and it usually takes longer than wiring up the model call itself.
What’s the fallback?
Every model call can fail: the API can time out, or the answer can come back confidently wrong. A feature needs a defined fallback for the moments the model doesn’t behave, a simpler rule-based response or a queue for human review, instead of a wrong answer delivered with total confidence.
This matters more as a feature moves from demo to production traffic, which is most of what separates a working prototype from something we’d put our name on for AI SaaS work. A demo can go quiet when the API hiccups. A product a client’s customers depend on can’t.
Can you actually score it?
If you can’t write a test that says whether an answer was right, you can’t tell whether a prompt change made things better or worse, and you’re shipping changes on vibes. Anthropic’s evaluation tooling exists for exactly this: build a small set of real input examples, run them against a prompt, and grade the output automatically or by hand before you ship a change, so a regression shows up in the eval set instead of in a customer complaint.
That eval set doesn’t need to be large to be useful. What it needs is to reflect the actual inputs the feature will see, edge cases included, so a change that looks like an improvement on three cherry-picked prompts doesn’t quietly break the tenth.
The verdict
Five questions, and any one of them failing is a reason to wait, not a reason to give up on the feature. A “no” on determinism means write the function instead. A “no” on cost means add a human checkpoint. A “no” anywhere else means the missing piece is the work around the model call, and that’s what needs building before the model call does anything useful.
Most features that get pitched to us as “AI-powered” turn out to need less AI than the pitch implies, and the ones that genuinely need it need more scaffolding around the model than the demo suggested. If you’ve got a feature and you’re not sure which side of that line it’s on, tell us what you’re building and we’ll walk through the checklist with you before any of it gets built.