The point of an audit trail is reconstruction. NIST’s definition describes a chronological record sufficient to examine the sequence of activities leading to a security-relevant operation or event (NIST, audit trail). A final model response alone is a poor record of a tool-using workflow because it leaves the earlier activities out. Decide what evidence the application must retain before an incident forces the question.
For example, a refund agent’s record could include the authenticated requester, policy version, proposed amount, approval event, tool arguments, tool result and final transaction identifier. Tie them together with a request identifier. Test whether someone outside the development team can reconstruct a failed refund from the stored record without rerunning the agent or relying on its own explanation.
Asking how an enterprise will account for an AI system’s actions, Manoj Agarwal says, “Is there an audit trail that you’re going and keeping on what AI is doing and not doing?” (episode 16, 27:03).
For builders, choose retention, access and redaction rules with the people responsible for the data. Avoid collecting secrets merely because they passed through a prompt. What is AI observability? covers investigating requests, while AI agent identity and permissions covers who may act. How to govern AI agents connects the record to ownership and review.
Sources
- NIST, audit trail — Primary source for the explanation and distinctions in this entry.
