One record per change: what auditors actually want from AI-built software

Picture a change that went to production on a Thursday afternoon in March. Forty lines in a service that calculates fees. An agent wrote it, another agent reviewed it, the tests passed, someone clicked approve, and it shipped. Nothing went wrong.

Now picture an auditor in October asking about it. Who made this change? What was it asked to do? Which rules was it working under that day? Who approved it, and what did they see when they did?

In most teams using agents today, answering that means a small archaeology project. The pull request has some of it. The CI logs have some more, if they haven’t been rotated. The prompt lives in someone’s terminal history or a chat thread. The model version is in a provider dashboard, maybe. The approval is a green tick with a username next to it and no record of what that person actually looked at. Each source has a piece, and they rarely agree at the edges, which is exactly where auditors like to look.

I spent years on payments systems before this, and the audit question always arrived late, long after the people involved had moved on to other things. With human-written code you could at least find the engineer and ask. You can ask an agent too, but what comes back isn’t a memory. It’s a fresh answer from a model that wasn’t there. So the record has to exist on its own, written down at the time.

The questions the record has to answer

The questions themselves are predictable. An auditor isn’t interested in the agent’s chain of thought. What they want is closer to a well-kept ledger entry.

• Who made the change: which agent, acting on whose behalf, for which team.
• What changed, exactly, and what the agent read or relied on to make it.
• Under which rule: the version of the policies in force at that moment, and which checks ran against the change.
• Approved by whom: the named person, when, and what they were shown.
• Where it went: which environment it was deployed to, and whether it’s still running there.

Every one of those is easy to answer badly. “The agent” is not an answer to who. “The policy” is not an answer to which rule, because the policy has probably changed three times since March. And “approved by jsmith” tells you nothing about whether jsmith saw the diff or just the summary.

Why one record, and why at the time

The tempting fix is better logging. Turn up retention, ship everything to an observability tool, build a dashboard. That helps engineers debug. It doesn’t give an auditor what they need, because what they need isn’t more data. They need one account of each change that they can trust.

A few properties make that possible, and none of them is exotic.

It has to be one record. If the answer to “what happened” is assembled from five systems, then the answer is really an argument about which system to believe. A single record per change, holding all of it, removes the argument.

It has to be written as the work happens. Records reconstructed after the fact are reconstructions, with all the gaps that implies. If the record is a by-product of the work rather than a report about it, there’s nothing to reconcile.

It has to be impossible to change quietly. Nobody, including the team that runs the system, should be able to edit a record without that edit being obvious to whoever checks it next. That’s what lets an auditor rely on it without having to trust the people who produced it.

And it has to be readable by someone who isn’t an engineer. An auditor shouldn’t need a developer at their elbow to understand who did what. If they do, you haven’t built an audit trail. You’ve built a debugging tool with good intentions.

There’s one more property I’d add for anything regulated: being able to replay what happened. Given the record, you should be able to step back through the change and see what the agent saw at each point, which rules applied and where the human came in. That turns “we believe this is what happened” into “here it is, check it yourself”.

What regulators are reaching for

The rules don’t use this vocabulary, but they point the same way. Article 12 of the EU AI Act asks that high-risk AI systems technically allow events to be recorded automatically over the system’s lifetime, so that situations which may result in risk can be identified (Regulation 2024/1689, Article 12). Financial record-keeping rules such as MiFID II and DORA lean on a similar idea: that a supervisor should be able to reconstruct what happened, in order, from records that haven’t been tampered with.

None of those texts mentions agents writing software. But the moment an agent’s change touches a regulated system, its history becomes part of the evidence, and “we’d have to piece that together” isn’t a comfortable thing to say to a supervisor. I go through what Article 12 asks for in more detail in EU AI Act Article 12: what to log.

Where we’ve landed

This is one of the ideas Kizuna is built around. Every change an agent makes gets one record: who made it, what it touched, which rules it ran under and who approved it. That record is created as the work happens, it can’t be quietly altered, and someone outside the engineering team can read it and check it. I won’t go into how here. Whether it works is something you should be able to test for yourself.

The rules side of this is in set the rules once, let agents build within them, and the approval side in human gates that don’t bottleneck.

What to ask when you’re evaluating this

Whatever you’re looking at, ours or anyone else’s, a few questions get to the truth quickly.

Pick a real change from last month and ask to see its record. Then hand it to someone from internal audit and see whether they can answer the five questions above without help.

Ask what happens if an administrator edits a record. If the honest answer is “nobody would notice”, that’s your answer.

Ask whether the record and the work come from the same place. If the system does the work in one place and writes a log about it somewhere else, ask what happens when the two disagree, because one day they will.

Ask whether it all still works on your own infrastructure, including environments with no internet connection. A record that has to be shipped to someone else’s cloud to be signed or checked isn’t one you fully control.

And time it. When a reviewer asks a question about a change, how long until they have an answer they can verify? If it’s weeks, you have logs. If it’s minutes, you have a record.

If you’re working on this problem in a regulated setting, we’d like to hear from you at /contact.