Testing in production is real. It is not a QA strategy.
Testing in production catches things staging never will. It also catches them after a real user already hit the bug. Why testing in production is a supplement to pre-merge verification, not a replacement for it — and the specific bug classes each one owns.
Testing in production is real. It is not a QA strategy.
I want to defend testing in production and then take one thing away from it, because the way the phrase gets used has quietly turned a good practice into an excuse. Testing in production is real engineering. Canaries, feature flags, progressive rollouts, synthetic monitoring, dogfooding — these catch failures that no staging environment can, because staging is a scale model and production is the territory. I'm not here to talk anyone out of it. I'm here to argue with the sentence it's increasingly used to justify, which is some version of: "we test in production, so we don't really need the pre-merge stuff." That sentence is where a deployment technique gets promoted into a testing strategy it was never built to be.
The tell is in the timing, and it's the whole argument, so let me put it first: every form of testing in production catches the bug after a real user could hit it. That's not a knock — it's the definition. You ship the code to production, then you watch. The canary group is real customers. The flagged feature is on for real accounts. The synthetic check runs against the live site after the deploy. In every case the code is already out there when the signal arrives. Pre-merge verification catches the bug before the code merges — before any user, canary or otherwise, can reach it. Those are different jobs at different moments, and no amount of production observability collapses the gap between "we noticed" and "it never shipped."
The advocates already concede the point
Here's the part that should settle it, and it comes from the people who sell testing in production, not from a skeptic. LaunchDarkly — whose entire business is feature flags and progressive delivery — opens their case for testing in production by drawing the line themselves: "Testing in production is not a substitute for quality assurance (QA), or a shortcut to eliminating unit testing or integration testing. Instead, it is an extension of testing and QA control points into the most realistic environment possible—the real-world." An extension of QA control points. Not a replacement for them. And when they ask the question directly — does production testing replace other testing? — the answer is a flat "No," followed by a list of the tests they expect you to have already run before the code gets near production: unit, integration, performance, regression, functional, usability.
Read that list again, because it's an advocate's list, not mine. The company most invested in you testing in production is telling you to run your regression and functional and integration suites in a pre-production environment first, and to treat the production layer as the thing that sits on top. The people who'd most benefit from "just test in prod" being sufficient are the ones telling you it isn't. When both the critics and the champions of a practice agree on its boundary, the boundary is real, and the only people ignoring it are the ones using the phrase as cover.
What each layer actually catches
The reason both jobs exist is that they catch genuinely different bug classes, and confusing them is how teams end up exposed. Testing in production is uniquely good at the failures that only exist at real scale, with real data, real traffic, and real user weirdness. LaunchDarkly's own framing names it: you test in production because "it's difficult to create synthetic users that have the full range of organic users," and because of "weirdness" — the user who "names an object with an emoji, or runs a three-year-old unpatched browser." That's true and it's important. A load pattern you can't reproduce, a third-party integration that behaves differently under production credentials, a data shape only your oldest customer has — those are found in production because they only exist in production.
But look at what that list has in common: every item is an environmental surprise — scale, data, traffic, a weird client. None of them is "the checkout button submits the wrong total," or "the new signup flow drops the email field on mobile," or "the redirect after login loses the query param." Those are logic bugs, and they're deterministic. They break for the first user exactly the way they break for the millionth. There is no reason on earth to discover a broken checkout by watching it fail for a paying customer in a canary when you could have driven the same flow, in a real browser, against the pull request, and never merged it. Using production to catch a deterministic logic bug is using the most expensive possible detector — one that bills you in real users — for a bug a pre-merge run would have caught for free. This is the same reason I keep arguing that coverage percentages measure the wrong thing: the number, like the canary, tells you something, but not the thing you actually needed to know before you shipped.
The failure mode: "test in prod" as a synonym for "skip the hard part"
The honest version of testing in production is a lot of work. You need feature flags wired correctly, a canary population defined, observability that can actually distinguish a regression from noise, and a kill switch you've tested. LaunchDarkly is careful to note the safety net isn't free — "take advantage of test automation, don't bypass running tests in staging because you will be testing in production," they warn, repeating it because teams do exactly that. The failure mode isn't teams doing rigorous progressive delivery and skipping E2E. It's teams saying "we test in prod" while doing neither the rigorous production part nor the pre-merge part — using the phrase to describe the absence of a strategy rather than the presence of one.
I've watched this happen. The unit tests are green because they test pure functions. There's no end-to-end coverage of the critical flows because those tests were flaky and got deleted — and E2E tests are flaky for reasons that feel like they justify giving up. Monitoring exists, so the team tells itself it's "testing in production." What's actually happening is that the first full exercise of the checkout flow on any given deploy is performed by a customer, and the monitoring's job is to notice when that customer has a bad time. That's not testing in production. That's finding out in production, which is the thing testing was supposed to prevent, dressed up in the vocabulary of a discipline the team isn't practicing.
The distinction I'd hold onto: production testing done right is proactive — Cindy Sridharan's framing, which LaunchDarkly cites, splits it into deploy, release, and post-release phases, each with deliberate checks. Production testing done as an excuse is reactive — you deploy and wait for a complaint. Same production environment, opposite intent. One is a layer on a strategy. The other is the absence of one with a good alibi.
Where the pre-merge layer got easy enough to have no excuse
Part of why "we'll catch it in prod" became tempting is that the pre-merge alternative used to be genuinely painful. Full end-to-end coverage of your critical flows meant a Playwright or Cypress suite that took a person-week to write and broke every time a selector moved. When the pre-merge option is a brittle suite you're maintaining at the cost of shipping features, punting to production monitoring is a rational-feeling trade — a bad one, but rational. That trade is what changed, and it's why I think the "test in prod instead" argument is weaker now than it was three years ago.
You can describe a critical flow in plain English — "log in, add two items, check out with a test card, confirm the order total and that it appears in history" — and have an agent drive that flow in a real browser against the pull request, before merge, returning an evidenced verdict instead of a red selector. No suite to maintain, no scripts to heal when the checkout gets redesigned. That's the argument I made in the case against writing your own Playwright tests and in what an AI QA engineer actually replaces: the reason to skip pre-merge E2E was the cost of pre-merge E2E, and that cost is mostly gone. Wire the run into the pull-request check so a red result blocks the merge, or against the preview deployment before it's promoted, and the deterministic logic bugs never reach the canary in the first place. Then testing in production goes back to its actual job — catching the environmental surprises only production has — instead of moonlighting as the QA strategy that isn't there.
The point, stated plainly
Keep testing in production. Flags, canaries, progressive rollout, synthetic checks, dogfooding — all of it, done deliberately. It catches a class of bug nothing else can, and turning it off would be a mistake. Just don't let the phrase do work it can't do. It is a supplement to pre-merge verification, not a substitute for it, and the moment it becomes the only place a flow gets exercised, you've stopped testing and started hoping the monitoring is fast. The strategy is both layers: verify the deterministic flows before merge so users never meet those bugs, and watch production for the surprises that only live at scale. Anyone selling you either one as the whole answer — including the vendors who sell testing in production — will, if you read their own words, tell you it's half.
Try the pre-merge half on your riskiest flow
Pick the flow you'd least want a customer to find broken in a canary. Here's the pre-merge check that keeps it out of production in the first place. Save it as a Test Scenario and point it at a staging or preview build:
Each run returns a Monito Session — screenshots, the network log, and the agent's reasoning at each step — so a failure is something you read on the pull request, not something a customer reports from the canary. Wire it into CI on every deploy and the deterministic bugs stop before merge; let production monitoring do what it's actually good at with the rest. A run is roughly 8–13 credits — about $0.08–$0.13 — and your first run is free. Test the flows that would hurt before they ship, and let "testing in production" go back to being a supplement instead of an alibi.