Microsoft's Copilot Studio fails legal workflow test
The final working version resembled "a glorified decision tree" more than an intelligent assistant.
Brandi Pack at UpLevel Ops put Microsoft's Copilot Studio through its paces, building a custom AI agent to review employee complaints for policy violations. The results were messy. Her initial FAQ bot couldn't reliably read its own source material and leaked personal planner data during testing. A more complex setup with multiple child agents, orchestration, and structured topics made things worse, not better.
Pack tested four different configurations. Each attempt required more rigid structuring to function, ultimately stripping away the generative capabilities that make AI useful. The final working version resembled "a glorified decision tree" more than an intelligent assistant. Even basic conversation flow broke down when the system tried to coordinate between components.
The gap between Microsoft's marketing and what Pack found in practice is hard to ignore. Microsoft positions Copilot Studio as production-ready for business automation, but Pack's testing shows it struggles with the nuanced, contextual work that legal teams actually do. For anyone evaluating AI tools in legal ops, this is a reminder: test the platform yourself before committing, no matter what the vendor says.
Brandi Pack
