Testing non-deterministic AI apps: assert on the invariants, not the output
You can't write expect(output).toEqual(...) against a product whose output changes every run. Testing non-deterministic AI apps means finding the things that DON'T change — structure, semantics, behavior — and asserting on those. A field guide.
Testing non-deterministic AI apps: assert on the invariants, not the output
Here's a question that breaks most test suites in half: what does it mean for a feature to "pass" when the correct answer is different every time you run it?
That used to be a philosophy-seminar question. Now it's most of your product. The summarizer, the "ask your data" box, the support agent, the code assistant, the thing that drafts the email — none of them return the same bytes twice, and all of them are shipping to production with tests that either assert nothing meaningful or don't exist. I've watched good teams with disciplined testing cultures wave a whole AI feature through review with a single test that checks the response isn't empty, because they genuinely didn't know what else to assert. The output is non-deterministic, toEqual is dead on arrival, so they gave up and checked for a pulse.
This post is the argument against giving up, and a concrete method for what to do instead. The short version: you are not testing the output. You never were. You're testing a set of invariants — the things that must hold true regardless of what the model says — and there are far more of those than the exact-match reflex lets you see.
First, a category error to clear out of the way
Search "testing non-deterministic AI" and you'll drown in content about model evaluation: eval harnesses, LLM-as-judge scoring, BLEU and ROUGE and semantic-similarity metrics, golden datasets, offline benchmarking. That work is real and it matters. It is also not what most people building products need, and conflating the two is why the topic feels so much harder than it is.
Model evaluation asks: is this model good at this task? You run it over a labeled dataset, you score the outputs, you compare model A to model B. It's a data-science activity, it happens offline, and its unit of analysis is the model.
Product QA asks a completely different question: does my application behave correctly when it wraps this model? Does the answer render without breaking the layout? Does the citation link actually go somewhere? Does the "regenerate" button work? Does the app degrade sanely when the model returns garbage, or does it show a stack trace to the user? None of that is about whether the model is smart. It's about whether the software around the model is correct — and the software around the model is deterministic, testable, and, in my experience, where the actual bugs are.
The mistake is thinking that because the model is non-deterministic, the whole application is untestable. It isn't. A non-deterministic core is surrounded by a large, boring, entirely deterministic shell, and most of what users experience as "the AI is broken" is actually the shell being broken. Test the shell like you'd test any other software, and treat the model's output as an input you don't control — because that's what it is.
Exact match was never coming back
Let me kill the hope directly, because some part of you is still wondering whether you can pin the output down with the right settings.
You can't, and not for lack of trying. OpenAI added a seed parameter specifically so developers could get reproducible outputs, and their own documentation is refreshingly blunt about the ceiling: even with the seed fixed and all other parameters held constant, "Determinism is not guaranteed, and you should refer to the system_fingerprint response parameter to monitor changes in the backend... there is a small chance that responses differ even when request parameters and system_fingerprint match, due to the inherent non-determinism of our models."
Sit with that. The vendor built a feature whose entire purpose is reproducibility, and the best they can promise is mostly. The infrastructure — floating-point non-associativity across GPUs, batching, mixture-of-experts routing, model updates "a few times a year" that silently change the fingerprint — makes byte-identical output an unfulfillable dream even before you get to temperature and sampling, which most products deliberately turn up because users want variety.
So exact match is out. Good. It was a bad test anyway — it couples your suite to one specific string, and the moment you upgrade the model or tweak the prompt, every assertion breaks even though nothing is actually wrong. This is the same disease that makes selector-based E2E tests so miserable, just in a new location. If you've read why your E2E tests are flaky, you already know the shape of the trap: you bet on a brittle, implementation-level detail, and reality drifts out from under the bet. An exact-match assertion on model output is that bet turned up to eleven.
The reframe: three layers of things that don't change
Stop asking "is the output correct" (unanswerable) and start asking "what must be true about the output" (very answerable). Those are the invariants, and they come in three layers, from cheapest and most reliable to most expensive and most judgment-laden.
Layer 1 — structural invariants. The properties of the output that have nothing to do with meaning. If you ask for JSON, is it valid JSON that parses? If you ask for a table, does it have the right columns? If the model is supposed to return a list of exactly five suggestions, are there five? Did the answer render inside the chat bubble instead of overflowing it? Are there exactly as many citation footnotes as there are citation links? These are deterministic assertions about a non-deterministic payload, and they catch an astonishing share of real bugs — truncated responses, malformed structured output, the streaming that dropped the last token, the markdown that didn't parse and dumped raw backticks on the user. You can and should assert these hard.
Layer 2 — semantic invariants. The properties of the meaning that must hold regardless of phrasing. If a user asks "what's the capital of France," the word "Paris" must appear, in whatever sentence the model wraps around it. If they ask the support bot to cancel their subscription, the response must be about cancellation, not about upgrading. If you feed a document and ask for a summary, key entities from the source should survive into the summary. You're not checking the words; you're checking that the intent landed. This is where "assert on intent, not on string" becomes the whole game — you verify that a required concept is present, a forbidden one is absent, or the response is on-topic, and you tolerate infinite variation in how it's said.
Layer 3 — behavioral invariants. The properties of what the application does around the answer. The stop button halts the stream. The copy button copies this message, not the last one — the exact class of state bug we pulled apart for chat interfaces in how to test an AI chat UI, here generalized to any AI surface. A refusal renders as a calm message, not a 500. A too-long answer scrolls instead of breaking the page. Rate limiting shows a real message instead of a spinner forever. These are pure application behavior — the deterministic shell — and they're testable exactly like any other UI, because the model's specific words are irrelevant to whether the button works.
Almost every "we can't test this" feature is fully covered by layers 1 and 3 plus a light touch of layer 2. The output varies; the invariants don't.
What's actually deterministic in your "non-deterministic" app
It's worth being concrete about how much of an AI feature is not, in fact, random, because the fog around this topic makes people overestimate the chaos.
The input handling is deterministic. The prompt assembly is deterministic. The routing, the auth, the rate limiting, the retry logic, the fallback when the provider 429s, the token accounting, the way the response gets sanitized and rendered — all deterministic. The error states are deterministic: a timeout should always produce the same UI, a content-filter refusal should always produce the same UI, an empty completion should always produce the same UI. The persistence is deterministic: what gets written to the conversation history, what survives a reload, what syncs across devices.
The only non-deterministic thing in the entire pipeline is the specific sequence of tokens the model samples. That's one component. It's an important one, and it's the one everyone fixates on, but it's surrounded on all sides by ordinary software that ordinary testing handles fine — if you're willing to treat the model output as an untrusted, variable input rather than as a value to assert against.
I'd go further: the model's variability is a fuzzing feature, not just a problem. Every run feeds your rendering, parsing, and layout code a slightly different payload — different length, different structure, different edge characters. A suite that runs the same AI feature ten times is quietly running ten different inputs through your deterministic shell, and that's exactly the kind of variation that surfaces the "it breaks on responses over 2,000 characters" bug you'd never hit with a fixed fixture.
The judge problem, and how to keep it honest
Here's the part the vendors selling you LLM-as-judge won't lead with: if you use a model to evaluate a model's output, your test is now non-deterministic too. You've moved the problem, not solved it. The judge that says "yes, this summary is faithful" today might say "no" tomorrow on the same input, and now you have flaky tests with a machine-learning excuse.
This is real, and the answer is not to throw out the judge — it's to give the judge deterministic-enough questions. There's a spectrum of how much you're trusting judgment, and you want to sit as far toward the reliable end as your feature allows:
- "Does the string 'Paris' appear?" — not a judgment at all. Perfectly reliable. Use this whenever the invariant is a concrete token.
- "Is this response about cancelling a subscription, yes or no?" — a judgment, but a coarse, binary, low-variance one. Two different competent humans would agree every time, which is the sign that a model judge will too, run after run.
- "Rate the empathy of this response from 1 to 10." — high-variance judgment. Humans disagree, so the model will be flaky. Avoid this in a pass/fail gate; it belongs in offline eval with aggregate scoring, not in the check that blocks your deploy.
The discipline is: push every assertion down toward the concrete end of that spectrum. Turn "is this answer good" (unanswerable, flaky) into a bundle of narrow questions — "does it name the right product," "does it avoid promising a refund," "is it under the length limit," "does it render as clean markdown" — each of which is close enough to deterministic that a judge answers it the same way every time. You're not eliminating judgment; you're rationing it, and spending it only where a human would give the same verdict twice.
And when you genuinely can't get below a certain variance, borrow the statistician's move: run it N times and assert on the rate. "The cancellation intent is correctly detected in at least 9 of 10 runs" is a legitimate, stable test even though any single run might wobble. That's the same math that governs why flaky suites rot as they grow, used deliberately and in your favor — you're setting a threshold you understand instead of pretending the underlying process is deterministic when it isn't.
Why a browser agent is the right shape for this
Everything above is doable in principle with a hand-rolled harness. In practice it's brutal, because layers 1, 2, and 3 live in three different places — the JSON in the network response, the meaning in the rendered text, the behavior in the DOM — and stitching assertions across all three from a scripted test is exactly the kind of multi-surface, timing-dependent work that scripts are worst at. This is the same reason scripted tests miss whole classes of bugs: they look at the app through a keyhole, one selector at a time.
An agent-based check reads the rendered page the way a person does and judges it against the invariants you stated in plain English. It doesn't need to know what the model will say, because you didn't ask it to check what the model said — you asked it to check that the answer was on-topic, rendered cleanly, and the buttons worked. That's a description of intent, and describing intent instead of scripting steps is the whole premise of natural-language test automation. The non-determinism of the content stops mattering the moment your assertion is about properties instead of strings.
It also sidesteps the maintenance tax. A model upgrade that changes every word of every response breaks zero invariant-based checks and every exact-match one. That resilience is the same property that makes self-healing tests attractive, arriving through the front door instead of as a bolt-on: if your assertion never referenced the specific output, there's nothing to heal.
Putting it together: prompts against a real AI feature
Here's the method as runnable checks. Assume a staging build of a product with an "ask your data" feature — a box where a user asks a question and gets an answer grounded in their uploaded documents.
The structural-and-behavioral check (layer 1 + 3). The highest-value, most-reliable test, and the one to write first.
None of that asserts what the summary says. All of it catches the bugs that actually ship: the fourth bullet, the broken citation, the raw markdown, the layout overflow on a long answer.
The semantic-invariant check (layer 2), kept binary. Assert on intent, at the concrete end of the judgment spectrum.
Each check is coarse enough that a second run — or a second human — reaches the same verdict. That's what keeps the test itself from becoming flaky.
The degradation check (layer 3). The part everyone forgets: what the app does when the model doesn't cooperate.
Where this leaks — the honest limits
I'm not going to pretend invariant testing is a complete theory of AI quality, because it isn't, and the posts that oversell their method are the ones you stop trusting.
It won't tell you the model got smarter or dumber in a subtle way — that a summary is technically on-topic but quietly less accurate than last month's. Drift in answer quality, as opposed to answer correctness on invariants, is a job for offline evaluation against a labeled set, and you should run that too; it's a different tool for a different question. Invariant checks also can't judge genuinely subjective properties — tone, tact, brand voice — beyond the crudest binary. And the harder your feature leans on nuanced correctness (a medical or legal assistant where "mostly right" is dangerous), the more you need human review in the loop and the less any automated gate should be trusted alone.
What invariant testing does give you is the thing you're currently missing entirely: a continuous, every-deploy proof that the deterministic 90% of your AI feature — the rendering, the structure, the behavior, the failure handling, the on-topic-ness — still works, run after run, without coupling a single assertion to a string the model will never repeat. That's not everything. It's most of what breaks, and right now most teams are shipping it with a test that checks for a pulse.
The one to run first
If you take one thing from this, make it the structural-and-behavioral check — it's the cheapest, the most reliable, and it finds the most real bugs per credit. Save it as a Test Scenario, point it at your AI feature, and run it on every preview deploy so a regression in the shell around the model fails the PR that caused it. Each run is roughly 8–13 credits (about $0.08–$0.13), and every failed run returns a full Monito Session — screenshots, network log, and the agent's reasoning — so you can see exactly which invariant broke.
Here's the one to paste in:
Your first run is free — point it at the AI feature you've been afraid to test and find out how much of it was deterministic all along.