"The harness is the floor, not the ceiling."
Trust comes from something above the harness: meta-cognitive priors.
That's the line from a recent piece on legal AI agents that I keep coming back to. The argument: the scaffolding we put around legal AI (tool schemas, retrieval, guardrails, evals) keeps it from being dangerous. It doesn't make it trustworthy. Trust comes from something above the harness: meta-cognitive priors. The agent's sense of what it doesn't know, when to stop, when to escalate.
That framing changes how you build.
Most legal AI programs I see right now are harness programs. They invest heavily in the floor: retrieval quality, citation checking, prompt templates, refusal rules. That work is necessary. It is also not sufficient. A harness-only agent will be confidently wrong inside its guardrails, and the guardrails won't catch it, because the failure isn't a rule violation, it's a reasoning gap.
The ceiling work is harder and less visible:
- Whether the agent knows when a question is outside its competence.
- Whether it recognizes a novel fact pattern versus a familiar one.
- Whether it escalates when escalation is the right answer, not just when a rule fires.
- Whether its confidence tracks its accuracy, rather than just the fluency of its output.
Those are meta-cognitive questions. They don't get solved by better prompts or more tools. They get solved by training, evaluation, and architectural choices that treat self-knowledge as a first-class capability.