The useful version of AI in underwriting assembles the file, surfaces the anomalies and drafts the reasoning — and leaves the decision, and the accountability, with a named underwriter.
There are two products hiding under the label of the underwriting copilot, and the difference between them is the most important product decision in commercial-lines AI right now. One assembles the submission, checks it against appetite, surfaces what an experienced underwriter would want to see, and drafts the memo. The other quietly becomes the underwriter, with a human whose real job is clicking approve. The first is decision support. The second is decision transfer wearing decision support's badge, and it is how good tools produce bad books.
Our view is unfashionable in a market that rewards automation claims: for risks complex enough to have an underwriter at all, the copilot should make the underwriter faster and better informed, and the decision should demonstrably remain theirs. Not as a legal fig leaf — demonstrably, in the sense that the underwriter regularly disagrees with the tool and the workflow makes disagreement cheap.
What copilots are genuinely good at
Strip away the branding and the strong use cases are clerical, in the best sense. Underwriters spend a remarkable share of their week not underwriting: rekeying broker submissions, hunting for prior-year files, reconciling schedules against proposals, checking a risk against appetite documents that live in a PDF somewhere. A copilot that reads the submission, structures it, flags what is missing, pulls the comparable risks and pre-fills the referral form attacks the part of the job nobody trained for and nobody enjoys.
The second strong use case is anomaly surfacing. A model that has seen the whole portfolio can notice what any single underwriter cannot: this valuation is far from similar properties, this loss history pattern resembles accounts that later deteriorated, this broker's submissions systematically omit a field. Presented as questions rather than verdicts — this looks unusual, here is why — these prompts sharpen human judgement instead of replacing it.
Where the line actually is
The line is not a philosophical preference; regulators have started drawing it. The NAIC's model bulletin on insurers' use of AI, adopted in December 2023 and since taken up by a large group of US states, requires insurers to run a written governance programme for AI systems and holds them accountable for decisions those systems support — including compliance with the same unfair-trade-practice rules that govern human decisions. The direction of travel across markets is consistent: you can automate the work, but you cannot automate away the accountability.
That has a concrete design consequence. If the copilot's recommendation is effectively final — because the interface buries the evidence, because dissent requires a written justification while agreement takes one click, because the queue is sized so that reviewing properly is impossible — then the insurer is running an automated decision system without having validated it as one. The workflow, not the org chart, determines which product you actually built.
An approval rate near one hundred percent is not a sign the model is excellent. It is a sign the human layer has gone decorative.
Automation bias is the real adversary
The failure mode is not dramatic. It is drift. In the first month, underwriters check the copilot's work. By the sixth, the copilot has been right often enough that checking feels like waste — and the operation has silently crossed from support to transfer without anyone deciding to. Automation bias is well documented in every field that has instrumented decisions, and underwriting will not be the exception.
Countermeasures have to be structural, because good intentions decay faster than habits form.
- Measure and review the disagreement rate, per underwriter and per segment. Near-zero disagreement triggers a process review, not a celebration.
- Keep dissent cheap. Overriding the tool should cost one dropdown, not a paragraph of justification that agreement does not require.
- Sample agreed cases for senior review, not just overridden ones — the dangerous errors are the ones both the tool and a hurried human waved through.
- Rotate seeded test cases into the queue where the correct answer is known, so vigilance is observable rather than assumed.
The uncomfortable question of speed
None of this comes free. A copilot governed this way delivers less headline automation than one allowed to decide, and slower quotes than a fully automated flow. For genuinely simple, data-rich, high-volume risks, full automation with proper model governance is often the honest answer — motor and much of personal lines already run this way, and pretending a human reviews each policy would be theatre. The copilot pattern earns its cost precisely where risks are heterogeneous, data is partial and judgement is the product being sold.
That is the test we would put to any underwriting AI purchase: decide first which decisions you are willing to fully automate and govern as automated, and which you are not. Buy a decision system for the first group and a copilot for the second — and build the second so you can prove, months later, that the underwriter is still in the room.