destill.ai · Insurance · Trust Center
For underwriters of cyber, E&O and AI liability
Seven signed numbers per insured company. Do they say anything about claims?
Status 3 November 2026: a plan, no data yet. Whether these metrics predict claims or losses is unproven. This pilot is designed so that it can show they do not. No price, premium or score is derived from them before it has run.
Many cyber, E&O and liability policies cover damage caused by AI without pricing it ("silent AI"). From 9 December 2026 the new EU Product Liability Directive treats software, AI included, as a product. Our firewall gateway already counts, per insured company, what its AI did and signs the count. This page says how an insurer pilot tests whether those counts carry risk information.
The seven metrics
One signed report per insured company and month, from the AI gateway in front of its models. Hashes and counts only: no prompt, no answer text, no personal data.
| id | metric | what it counts | what it does not show |
|---|---|---|---|
| U1 | AI steps with a verified proof | agent actions in signed POAW records that verify, divided by all agent actions in those records | agents that were not wired to POAW |
| U2 | AI calls through the firewall | every call of the month that passed the gateway | AI use that bypasses the gateway |
| U3 | Attacks blocked at the input | calls stopped before the model saw them, also per 1,000 calls | a count, not a detection rate: false alarms are in it, missed attacks are not |
| U4 | Leak events | answers that disclosed confidential instructions, stopped in the stream or delivered and judged afterwards | leaks the checks do not recognise |
| U5 | Locked keys | requests refused because a key was paused after a leak, plus keys revoked in the month | whether a revocation was an incident |
| U6 | Agent self-reports of fabrication or drift | signed agent runs that flagged fabrication or drift from their task | agents that do not report |
| U7 | Failed or aborted calls | calls that did not complete as asked | a loss: this is availability |
An underwriter checks each report without trusting us: the signature against our published key (offline possible), a recount of the numbers from the rows the signed report itself lists, and a fresh request for the same month that must give the same numbers. Changing one figure makes the check fail and name it.
How the pilot is set up
- Unit: one insured company in one calendar month ("company-month") whose AI traffic ran through the gateway.
- Primary outcome: an AI incident in the 90 days after the month, from the company's own incident register, under a definition fixed before the start: (a) confidential data disclosed, (b) the AI manipulated by input into an unintended action, (c) an output a customer or regulator complained about.
- Secondary outcome: a notified claim under the cyber, E&O or AI-liability policy attributed to such an incident.
- Blinding: the insurer keeps incidents and claims, we keep the metrics. Both sides fix a fingerprint (sha256) of their file before the join, which runs on a pseudonymous company id. We never see claim files.
- Fixed in advance: metric definitions, hypotheses and the analysis script are published with their fingerprint before the first outcome is read. A later change is reported as a change, with both results.
Hypotheses, and what would refute them
| # | hypothesis | supported if | refuted if |
|---|---|---|---|
| H1 | Months with a leak event (U4 > 0) are followed by more data-disclosure incidents | relative risk ≥ 2, 95 % interval above 1 (assumed threshold, fixed in advance) | 95 % interval upper bound below 1.5 (assumed threshold) |
| H2 | Input-attack pressure (U3 per 1,000 calls) rises with manipulation incidents | rank correlation ≥ 0.3, interval excludes 0 | interval upper bound below 0.1 |
| H3 | Self-reported fabrication or drift (U6 > 0) is followed by more complaint incidents | relative risk ≥ 2, 95 % interval above 1 (assumed threshold, fixed in advance) | 95 % interval upper bound below 1.5 (assumed threshold) |
| H4 | With high proof coverage (U1 ≥ 0.9) the cause of an incident can be settled from the records alone | ≥ 70 % of incidents vs ≤ 30 % without coverage | difference below 15 points |
| H5 | Control: failed calls (U7) do not predict incidents | interval includes "no effect" | U7 predicts as well as U3, U4 or U6 |
The idea as a whole is refuted if none of H1 to H3 holds, or if the control H5 predicts incidents as well as they do. Then the metrics only measure how much AI traffic there is, not risk. H4 concerns claims handling and can hold on its own.
How many company-months
Assumptions, to be replaced by the insurer's own base rates before the start: incident rate 5 % of company-months without the signal and 10 % with it, signal present in one month out of five, 5 % significance, 80 % power. That needs about 270 company-months with the signal and 1,090 without, roughly 1,360 in total, for example 115 companies over 12 months. With fewer, the result is published as a description only, with no statement about prediction.
Notified claims are much rarer than incidents. A pilot of this size cannot test claims directly; they are reported, but conclusions rest on incidents. Claims need pooling across insurers or a longer run.
Who gives what
- Insured company: routes its AI calls through the gateway, downloads its monthly report, keeps an incident register.
- Insurer: incident and claim outcomes per pseudonymous company id, after both sides have fixed their file fingerprints.
- destill.ai: the signed reports, the check tool, the analysis script fixed in advance, and the result, including a null result.
Known weaknesses of this design
Companies that join such a pilot are probably more careful than average, so the result may not transfer. The incident register is kept by the insured company itself; under-reporting pushes the result towards "refuted". Leak and lock counts depend on which checks run on the gateway; that version is fixed for the run. AI use that does not go through the gateway is invisible. The metrics are signed counts; the signature proves the count is unchanged, not that the count is complete.