An AI Mission for Insurance: Fraud Signal Review

A fraud signal is an accusation in embryo. The question worth designing around is not how many a system can raise, but whether the one it hands to a human investigator could survive being read aloud to the person it names.

Three weeks after a kitchen fire, a policyholder is still living in a relative's spare room and still waiting. The claim has not been denied and nothing has been said to suggest it will be. It has simply gone quiet: a status that stopped updating, an adjuster who now answers with careful phrasing about the file being "under additional review," a settlement that was supposed to arrive in days and has now missed two rent cycles on the temporary accommodation. Somewhere upstream, a model scored the claim, a threshold was crossed, and a referral appeared in a special investigations queue with a code on it and very little else. Nobody in the building has yet formed a belief about whether anything improper happened. The person waiting has no way to know that, because from where they sit, the silence is indistinguishable from suspicion.

That gap between what the organisation actually believes and what the claimant experiences is the real subject of any serious conversation about AI in fraud detection, and it is almost never where the conversation starts. The industry talks about detection rates, model lift, the proportion of leakage recovered, and those numbers matter to the loss ratio. They say nothing about the fact that every flag raised is a small act performed on a real household, usually at the worst point in that household's year, and that the cost of getting it wrong lands on someone with no visibility into the process and no obvious way to contest something that has not formally been alleged.

The two errors are not the same size, so they should not be tuned as if they were

Any classifier can be pushed toward catching more or bothering fewer, and the whole engineering culture around detection treats that as a dial with a defensible optimum somewhere in the middle. The trouble is that the two failure modes on either side of the dial are not commensurable quantities that a single expected-value calculation can honestly reconcile. A missed irregularity costs money, and that money is diffuse: it dilutes across a book of business, it shows up as basis points in an annual result, and it is borne collectively by an institution built precisely to absorb aggregate loss. A wrongly raised signal costs a specific person a specific delay at a moment when they are already displaced, uninsured against the interim, and least able to absorb friction. One error is financial and spread; the other is personal and concentrated. Averaging them into a single cost function quietly declares that the second kind is a rounding error in the first, which is a moral claim dressed up as an optimisation, and it is the assumption most quietly baked into fraud-detection programmes.

Once you refuse that assumption, the design problem changes shape. The goal stops being to raise as many defensible flags as possible and becomes something narrower and much harder: to make sure that when a human being's claim is slowed down, there is something genuinely worth a human investigator's attention behind it, assembled well enough that the investigator can resolve it quickly in either direction. Speed of resolution matters more than volume of detection, because the harm to an honest claimant is almost entirely a function of how long the file sits in ambiguity. A signal that arrives with its supporting context already gathered and its contradictions already surfaced can be closed the same morning; one that arrives as a bare score costs the claimant every hour of queue time plus every hour of reconstruction. The same underlying model, wrapped in two different handoffs, produces two very different amounts of human damage.

There is also a discipline question hiding inside the volume metric. When a programme is judged by how many referrals it generates, it will generate them, and the queue fills with thin, ambiguous, unfalsifiable flags that nobody has time to work properly. Investigators triage by whatever is cheapest to dispose of, the genuinely complex cases age, and the honest claimants sitting behind the weak flags wait longest of all, because there is nothing in a weak flag that anyone can decisively clear.

The unit of output is a referral packet, not a score

The most useful reframing available here is to stop thinking of the AI's job as producing a judgment and start thinking of it as producing a document. What should leave the system is not a number attached to a claim but a packet: a plain-language statement of what pattern was observed, the specific records and timestamps that produced it, the ordinary explanations that were checked and either ruled out or left standing, the pieces of context an investigator would otherwise spend an afternoon gathering by hand, and an explicit account of what is missing or uncertain. That is genuinely hard work — pulling the policy history, the prior claims record, the repair estimates, the supplier documentation and the timeline of communications, then reconciling all of it into something a human can read in ten minutes. It is also exactly the kind of work that has always been done badly, because it is coordination labour across systems that do not share a memory, and no institution has ever had enough investigator hours to do it properly on every file.

This is where autonomous systems earn their place in claims, in a much less glamorous role than the one they are usually sold into. The valuable capability is not the pattern recognition, which insurers have had in some form for decades; it is the assembly. A reasoning system coordinating specialist agents across the policy administration system, the claims record, the document store and the communications log can build that packet for every candidate signal, in the hours after it appears rather than the weeks after it queues. Done well, this raises the evidentiary floor for what counts as a referral at all, because a signal that cannot be supported by a coherent packet simply never becomes one.

It matters enormously that the question stays a question. An automated system must never be the thing that concludes fraud occurred, and it must never be the thing that denies, reduces, or repudiates a claim. Investigation and determination belong to identified humans who are answerable for them — the investigator who works the file, the adjuster who handles the claim, the manager who signs off. That is not a nod to caution or a limitation waiting to be engineered away; it is the point of the architecture. Accusation is an exercise of institutional power over a person, and power of that kind requires someone whose name is on it and who can be asked to justify it. The right posture for the machine is the one platforms like StudioX describe as human-in-the-loop by design, where the agents do the gathering and the reasoning and stop hard at the boundary where determination begins. The broader argument for what an autonomous enterprise should and should not hand to software turns on precisely this distinction between executing work and owning a decision, and claims handling is one of the places where the line is least negotiable.

The test that should govern release: could you explain this to the person it concerns?

The most practical governance rule available is also the simplest, and it has the useful property of being hard to game. Before a signal leaves the building — before it slows a payment, triggers a request for further documentation, or sends an investigator to a doorstep — someone should be able to state, in ordinary language, what was observed and why it warranted a closer look, in terms that would be intelligible and non-humiliating if read back to the claimant. Not disclosed, necessarily; there are legitimate reasons investigative detail is not shared while a file is open. But explainable in principle, to that person, without embarrassment. A signal that fails this test is not a signal. It is a correlation that the organisation has decided to act on while hoping it never has to account for it.

Applied honestly, this test disqualifies a whole class of features that detection programmes drift toward under commercial pressure. It rules out anything that reduces to a claimant's postcode, their name, the language they file in, or any of the variables that function as proxies for demographic and protected characteristics even when those characteristics are nowhere in the model. Proxy discrimination does not require intent and is not cured by omitting the obvious fields; it emerges wherever a model learns from historical referral decisions that themselves encoded who past investigators found suspicious. A system trained on those labels will faithfully reproduce them and present the result as objectivity, and the explanation test catches it, because "your claim was reviewed because claims from your area are reviewed more often" is not a sentence anyone can say aloud with a straight face. It also rules out treating claimants as presumptively dishonest by default, since the overwhelming majority of people filing a claim are doing exactly what they paid for years to be able to do.

None of this is an argument against detection. It is an argument that the thing worth automating is the diligence, not the accusation, and that the difference between those two is the difference between a programme that earns trust and one that erodes it. It is worth noting that scepticism about autonomous systems in regulated work is well earned; Gartner has predicted that more than forty percent of agentic AI projects will be cancelled by the end of 2027, citing unclear value and inadequate risk controls, and fraud detection is a domain where inadequate risk controls have a face attached.

So the mental model worth carrying is not detection at all. Think of a fraud programme as a system whose real product is the quality of the questions it asks a human to answer, and whose real cost is measured in the waiting time it imposes on people who did nothing wrong. Judged that way, the best possible outcome of an AI investment is not more flags. It is fewer, better ones, resolved faster, each with a packet behind it thorough enough that the investigator can clear the honest claimant by lunchtime — and none that anyone in the building would be ashamed to explain to the person it named.