When your AI agent needs an employee handbook
Are you building supervision frameworks that match the level of autonomy you're granting? Treating agents like assistants when they're acting like employees doesn't just create compliance risk—it creates the kind of accountability vacuum that ends badly.
In October I spent two days at IAPP Privacy. Security. Risk. 2025 in San Diego, watching 500+ practitioners try to solve problems that didn't exist two years ago. The conversations kept circling back to a tension I've written about before: we're building AI systems faster than we're building accountability structures around them. What struck me wasn't the regulatory uncertainty—that's nothing new. It was how the gaps between what developers build, what deployers control, and what regulators expect are creating real operational problems right now. Over the next few weeks, I'll pull patterns from sessions on AI agents, state innovation, cross-functional governance, and federal preemption—field notes on where handoffs are breaking down.
The "Minding Mindful Machines" session at IAPP PSR 2025 tackled a question that's getting harder to ignore: what happens when your AI agent screws up?
The analogy: treat AI agents like contract workers—smart but overconfident, prone to fabricating information, and vulnerable to social engineering. So how much autonomy do you want to give them? And when a hundred of these "workers" are collaborating on tasks, who's supervising?
A live demo showed the consent problem in action. An agent booking travel asked permission every few seconds—"Can I access your email? Can I search flights? Can I send this message?" The result? Consent fatigue. Users reflexively clicking "yes, yes, yes" just to complete the task. This is what consent frameworks look like when they meet autonomous systems.
When something goes wrong, explainability matters. An agent books the wrong venue or accesses the wrong data, and someone is standing before a judge, explaining the decision-making process. With multiple inputs, autonomous planning, and dynamic tool use, that explanation gets complicated fast.
The structural problem runs deeper: model developers sit "upstream"—they shape what agents can do during training and evaluation. Deployers sit "downstream"—they decide what agents should do in specific contexts. But users grant access without understanding what they're authorizing—one-time calendar access or persistent email monitoring? A governance gap where everyone assumes someone else is handling oversight.
The practical question for legal and product teams: Are you building supervision frameworks that match the level of autonomy you're granting? Treating agents like assistants when they're acting like employees doesn't just create compliance risk—it creates the kind of accountability vacuum that ends badly.
Governance frameworks need to shift from "what did the AI output" to "what did the AI decide to do, and why." That's not a technical problem—it's an organizational design problem. And it's not solved by adding another policy document. It's solved by embedding decision boundaries, escalation paths, and audit trails into the architecture itself.
