"The harness is the floor, not the ceiling."

Trust comes from something above the harness: meta-cognitive priors.

1 min read
"The harness is the floor, not the ceiling."

That's the line from a recent piece on legal AI agents that I keep coming back to. The argument: the scaffolding we put around legal AI (tool schemas, retrieval, guardrails, evals) keeps it from being dangerous. It doesn't make it trustworthy. Trust comes from something above the harness: meta-cognitive priors. The agent's sense of what it doesn't know, when to stop, when to escalate.

That framing changes how you build.

Most legal AI programs I see right now are harness programs. They invest heavily in the floor: retrieval quality, citation checking, prompt templates, refusal rules. That work is necessary. It is also not sufficient. A harness-only agent will be confidently wrong inside its guardrails, and the guardrails won't catch it, because the failure isn't a rule violation, it's a reasoning gap.

The ceiling work is harder and less visible:

  • Whether the agent knows when a question is outside its competence.
  • Whether it recognizes a novel fact pattern versus a familiar one.
  • Whether it escalates when escalation is the right answer, not just when a rule fires.
  • Whether its confidence tracks its accuracy, rather than just the fluency of its output.

Those are meta-cognitive questions. They don't get solved by better prompts or more tools. They get solved by training, evaluation, and architectural choices that treat self-knowledge as a first-class capability.