Test Stimulus & Coverage Generator: A Day in Practice

I run security and deployment at StudioX, which means my job is to be the person who asks "what happens when this touches a real team, on a real Thursday, inside a real perimeter?" Mark's pieces make the case and open the hood. Mine is the field report. I want to walk you through one change, from PR to merged, the old way and the new way, with the numbers I actually saw when we ran the Test Stimulus & Coverage Generator against a mid-size verification team's flow.

Names are changed; the diff is real in shape. A memory-controller team, mixed-signal SoC, roughly forty PRs a week hitting the RTL that their verification group has to check.

The old Thursday

Devon, the design engineer, opens a PR at 2:10pm: a new low-power arbitration mode. It changes how requests get prioritized when the controller is draining a queue before a refresh. Two hundred lines, clean, tests-pass-locally, moves on.

The PR sits in the verification queue until 3:30, when Maya picks it up. Here's how her afternoon goes:

• 3:30–4:20 — Ramp-up. She re-reads the arbitration section of the spec, diffs it against what the code actually does, and reconstructs Devon's intent. This is the blank page. Fifty minutes before she writes a single line of stimulus.
• 4:20–5:10 — She drafts directed tests for the obvious paths: normal arbitration, the new low-power mode in isolation.
• 5:10–5:40 — She thinks about corners. She gets two: mode switch mid-transaction, and a full queue. She misses the one that matters — a request arriving in the exact cycle the controller commits to a refresh, under the new priority order.
• Next morning — She writes coverage bins for what she tested. The bin for the refresh-straddle corner never gets defined, because she never imagined the corner. The coverage report is green. Everyone signs off.

Eleven weeks later that corner is an escape found in bring-up. Bring-up debug, root-cause, and the fix cost the program a chunk of a quarter. Nobody did anything wrong. The hole was invisible the entire time.

The new Thursday

Same PR, same 2:10pm. But now the merge event fans the change into a StudioX Mission that runs before Maya ever opens it. By the time she picks the PR up at 3:30, there's a draft waiting.

The Change Analysis Agent read the diff and identified the new arbitration mode and the exact priority path it touched. The Spec & History Agent queried the team's verification knowledge base — including two prior escapes in this same block, one of which was a refresh-timing straddle. The Coverage Agent checked the existing coverage model and found no bin covering priority behavior during a refresh commit. The Stimulus Agent drafted directed stimulus for the mode plus five corner cases — including the refresh-straddle, flagged and justified by the historical escape. The Report Agent assembled it into a portal Maya opens in one click.

Maya's afternoon now:

• 3:30–3:45 — She reviews the draft in the portal. She reads the observations rail to see why the refresh-straddle corner was proposed, and sees it's grounded in a real past escape. She trusts it.
• 3:45–4:30 — She corrects two of the directed cases, deletes one corner as not-applicable, and accepts the refresh-straddle and the new coverage target. Her expensive hour went entirely to judgment.
• 4:30 — Done. The coverage bin for the straddle now exists. The corner gets stimulated this week, not in bring-up.

That's the whole point. The blank page moved off Maya's desk. The invisible hole got a name in the PR.

The numbers, honestly

I don't like ROI claims that assume the machine is perfect, so here's what we actually measured across a four-week pilot on that team's forty-PRs-a-week cadence.

• Blank-page time. The ramp-up-and-enumerate cost dropped from roughly 45–60 minutes per change to a 10–15 minute review. Across ~160 PRs a month, that's the better part of two engineer-weeks returned — and returned as judgment time, not idle time.
• Corner coverage. The Mission proposed a median of five corner cases per change. Engineers accepted about 70% and deleted the rest as not-applicable — which is exactly the ratio I want. It's a strong first draft a human sharpens, not an oracle you rubber-stamp. The refusals are healthy.
• Holes named up front. Over the pilot, eleven coverage targets that would previously have been defined "someday, if we thought of it" got proposed on the PR and accepted. We can't prove each one prevented an escape. We can say each one turned an invisible gap into a decision made on purpose.

What made it deployable inside the perimeter

My deployment lens: this runs inside the company's own boundary. The agents reach the Git host and the coverage database through MCP servers the team registered — no code shipped to us, no repo leaving the perimeter. The verification KB with those two prior escapes stays on their infrastructure, grounded and isolated per agent. And because every proposal streams as an observation, the team's lead could audit why any corner was suggested before trusting the flow — which is what got it past their sign-off in the first place. When we later wired the optional "commit the coverage bins" action, it went through the decision queue with a human approval gate, so nothing touched the repo without a click.

None of this replaces Maya. It moves her to the top of the value curve — reviewing, correcting, deepening — on every change, from the first Thursday. If you want the leadership case for why the blank-page tax is worth attacking, read why it matters; for the architecture underneath, Mark's how it works. This is one instance of the broader AI Missions pattern we apply across workflow automation — encode the expert judgment, make it observable, keep the human in the loop where an action lands.