When AI incidents must be reported, thethe evidenceevidence trailtrail becomesbecomes criticalcritical

UK ministers are drawing up compulsory safety reporting for frontier AI, according to the Financial Times. The plans include mandatory reporting when AI agents behave maliciously, and possible legislation giving the AI Security Institute access to models before release. The AI minister, Kanishka Narayan, stated in his speech this week that the government can't rely on developers' assurances or wait for a major incident, and he is proactively pursuing "rigorous evaluations" of the latest generation of AI models to test safeguards and prevent risks.

To date, most of the debate has focused on the companies building frontier models, but there's a question banks, law firms, public bodies and insurers that run agents inside their own operations need to ask – if an agent misbehaves in your environment, can you figure out what happened?

The reporting problem starts with the record

Earlier this summer, an AI agent hacked an Australian health service website, although the incident only became public last month. Australia's prime minister said the developer took "way too long" to tell his government.

Delay of that kind usually has a mundane cause. Nobody could quickly reconstruct what the agent was allowed to do, what it touched and when a person stepped in. A reporting duty is only as good as the logs behind it.

For an organisation using AI agents on sensitive data, you should be able to answer these questions without hesitation:

• What permissions did the agent hold?
• Which data and systems did it reach?
• What consequential actions did it take?
• When did a human review or intervene?
• Who can see the log, and who controls it?

If the evidence sits inside a provider's platform, the organisation depends on that provider's account of events. Under a compulsory regime, that dependency becomes a regulatory risk.

Inspectability helps, and it needs the right framing

Open-weight models can be examined and tested by independent parties, which reduces some risks. Open-source models go further, because the code, and often the training details, can be reviewed too. The two terms are often used interchangeably, but they shouldn't be. Open weights let you test behaviour. They don't always show you how the model was built.

In the meantime, closed models leave some elements resting on provider assurances. That is a legitimate commercial choice, and it carries a cost when a supervisor asks for proof. The standard should be the same for every provider, whoever built the model and wherever it was built: show the evidence, and let someone independent check it.

What regulated organisations can do now

The rules are still being explored, so nobody can say what the final duties will look like. But some preparation is sensible, whatever they turn out to be. In the first instance, it is worth checking the following:

• Map your agents. Many organisations don't have a full list of where AI agents run, particularly where staff have adopted tools on their own. You can't report on what you haven't found.
• Log at the action level. Prompts and outputs are a start. Supervisors will want to see the actions an agent took against real systems, with timestamps.
• Keep control of the evidence. Hold audit trails in an environment you govern, with access rules you set. Where data must stay in the UK, as it must for many regulated clients, the logs should too.
• Test before you rely. The FT notes that a government exercise using commercial AI models to probe its own defences found about 1,000 cyber security flaws. Organisations can run the same kind of exercise on their own deployments.
• Rehearse the report. Draft an incident notice now, using a realistic scenario, and see how long it takes to fill in. The gaps will show up quickly.

Where sovereign infrastructure fits

The government's direction, as the FT describes it, includes keeping crucial technologies and data housed in Britain. For regulated organisations, the practical meaning is control over where AI runs, who can inspect it and who holds the records.

That is the thinking behind how Argyll Data Development designs sovereign AI environments: UK-resident inference, access controls the customer sets and audit trails the customer owns, so supervisors can verify events without relying on a third party's version. Oversight of frontier models remains a government and developer matter. Governance evidence inside a deployment is something an organisation can build today.

The next step

Compulsory reporting would reward organisations that already treat evidence as part of the system. Those that don't will be assembling it under pressure, after an incident, which is the worst time to find out what the logs don't show.