Ken Priore The writing · kenpriore.com
Reflections · 2026-10-09 · 7 min read

Everything is standardizing at the description layer

maybe....

Everything is standardizing at the description layer

PLI put out an AI competency framework a little while back, and it's good work. Five pillars, twenty competencies, three proficiency levels, free to download. What I like most is what they chose not to do: no certification attached, no required sequence, so you can be strong in one pillar and thin in another without the whole thing collapsing into a scorecard.

The comparison people reach for, including the people who built it, is EDRM. That comparison is right, and it goes further than it usually gets taken, because the two standards it connects took opposite routes to the same-looking place. One had a deadline behind it. The other had nothing but the market. Only one of them ended up with a way to prove anything.

2000: The internet standardized without a regulator

There was a formal standardization attempt for web services, and on the open web it lost. Through the late nineties the industry built SOAP and the whole WS-* stack through W3C and OASIS with every major vendor in the room, and the result was fully specified, designed by committee, and heavy enough that using it meant buying a toolchain most teams never wanted.

REST won instead, and it was mostly a description of what people were already doing, with no conformance body, no certification, and no compliance suite behind it. It spread because the barrier to entry was close to zero. People copied the APIs that worked. JSON pushed out XML. HTTP verbs and status codes turned into shared grammar without anyone voting on it.

That mechanism explains most of what follows. You adopt the common pattern because you want other people's systems to work with yours, and every idiosyncratic choice costs you an integration somewhere down the line. Nobody sanctions you for holding out. The market just routes around you.

2005: What EDRM had that the story leaves out

Five years later, the legal profession did something that looks similar.

The Electronic Discovery Reference Model came out of work by George Socha and Tom Gelbmann in 2005, built because a market was forming faster than anyone could describe it. It's the left-to-right diagram everyone's seen, with volume dropping and relevance climbing as you move across, and twenty years later any training that touches the subject still opens on that picture. The story we tell credits the volunteers, and that's true as far as it goes.

The story leaves out the deadline.

The Zubulake opinions ran from 2003 into 2004 and laid down what courts would do about preservation, litigation holds and sanctions, including an affirmative duty on counsel to monitor compliance, and the amendments that brought electronically stored information into the Federal Rules were published for comment in 2004 and took effect in December 2006. EDRM launched in 2005. Right in the middle of all that, when the rules change was already public and already scheduled.

The usual telling is that you couldn't hold a meeting, because when one person said collection and another said review they meant different things. That's the symptom. The cause is that the amendments required parties to confer early about electronically stored information, and if you mandate a conference that can't function without shared terms you have created about the strongest demand for shared terms anyone could design.

So the volunteers built the vocabulary, and the rules made the vocabulary necessary. And because the pressure came from a court that could sanction you, e-discovery got something the web never did. It got defensibility. Documented custodians, documented search terms, sampling, validation, a record of decisions and why. Imperfect and uneven, but present, and present because somebody with subpoena power was going to ask.

2015: The description gets written down, and only the description

Back on the other path, the API world eventually produced the closest thing it has to a standard, and it arrived afterward, to write down what had already happened, the way a dictionary arrives after the language. Swagger appeared around 2011 and became OpenAPI.

It standardized how you describe an API and left how you build one alone, so an OpenAPI document tells you what an endpoint claims to do, nothing anywhere certifies that it does, and you can publish a specification your service violates without any authority showing up. Contract testing exists as a practice and never became an institution.

2024: The same shape, at much higher speed

MCP is the API pattern running fast. Anthropic released it in late 2024 and open-sourced it, and within months competitors had implemented it. REST took about a decade to get where MCP got in a year.

And it has exactly the same shape. A server declares its tools with a name, an input schema, and a plain-language description, and the model reads that description and decides whether to call it. Nothing in the protocol establishes that the tool does what its description says, or records what it did once invoked, or carries the authority the agent was acting under, or preserves what a human approved in any form that survives the session. Hosts do ask users to approve tool use, but that approval lives in the host and dies with the session.

Which means you can't reconstruct the decision chain. Logs aren't records. A log tells you something happened; a record tells you who was answerable for it, under what authority, before the fact.

Somebody will point out that this is a layering objection, that MCP was never meant to carry attestation, and that asking a connection protocol to carry legal identity is a bit like asking HTTP to. The protocol is well designed for what it does. The layer it would need doesn't exist anywhere, and MCP's success makes that absence more consequential, because an enormous volume of consequential tool calls now flows through a standard with nowhere to put a record.

In both cases the trust model comes down to a claim written by the party with the most interest in that claim being believed.

2026: The competency framework arrives, on the API path

There's nothing scheduled for AI competence the way there was for ESI. No rules amendment in the pipeline, no mandatory conference that falls apart without shared terms, no sanctions regime waiting for the firm that can't describe its training program. Measured against EDRM, a framework arriving without that pressure looks like it's missing something. Measured against REST, it's exactly on schedule.

And the API path is the one it's on, for a reason visible in the problem it was built to solve. Attorneys move between firms carrying very different understandings of what AI competency even means, and there's no exchange rate between one vendor's academy credential and another body's proficiency level. That's an interoperability problem in the strict technical sense, and the answer is portability: making a lawyer's capability legible when they move, the way a shared specification makes a system legible when it connects.

No regulator required. It spreads if adopting it is cheaper than maintaining a private vocabulary, and for most firms it plainly is.

The mirror

Which means the framework also inherits what that mechanism has never produced. A competency framework standardizes how we describe a human's capability to work with AI. MCP standardizes how we describe a tool's capability for an agent to use. Both spread through adoption, with no obligation behind either one. Both are declarations made by the party being described. And neither produces any evidence that the declaration is accurate.

The profession and the agent stack are making the same move at the same moment, and the two communities have no reason to compare notes, because one works out of a CLE catalog and the other out of a JSON schema.

The gap persisted on the technical side without costing much because API mismatches fail loudly and immediately. The integration breaks, the error surfaces within the hour, somebody fixes it. Competence and supervision don't fail that way, and when your feedback loop is measured in years and nothing announces the failure, the declaration has to be backed by something. An agent that exceeded its authority produces a dispute nobody can reconstruct afterward. A lawyer who's rated fluent and isn't produces a supervision failure that surfaces in litigation years later, if it ever surfaces.

What backing it would take

A declaration says what something can do, and a record says what it did.

The distinction underneath all of it is the one people keep collapsing: capability is what something can do, authority is what it was permitted to do, and accountability is who answers for what it did. A description standard covers the first. Nothing in either world covers the third.

A competency framework says a lawyer can verify AI output. A record would say this particular lawyer reviewed this particular output on this matter and made this call. An MCP server says a tool searches agreements. A record would say this tool was invoked under this authority at this time with this result, and a named person approved it. Generated at the moment of action, before the failure, rather than reconstructed after it.

Neither record exists today, both could be built, and the party positioned to build them is the one the record would catch.

The forcing function

Technological competence has been sitting inside the duty of competence in the model ethics rules for more than a decade, adopted in most states, so there's arguably already a duty to be competent with these tools. What's never existed is a demonstrable standard for it, and a duty with no measure attached is unenforceable in practice.

So the pressure won't arrive the way it did for e-discovery. There's no rules amendment coming. It'll come from ethics enforcement, or malpractice carriers, or the first firm that gets asked in a proceeding to show its people were competent to supervise the systems they were supervising.

And when that happens, somebody's going to point at a framework and ask why it wasn't being used, and the answer is going to be that the framework told everyone what to aim at while nothing told anyone what happened.

That's an argument for what comes next. The people building these vocabularies are the right people to build the next part, because they've already done the harder half, which is getting a profession to agree on what the words mean.

Drafting Note: This write up is based on a discussion of the PLI Framework in “From AI and the Future of Law” with Jen Leonard, Bridget McCormack, Farrah Pepper, and Kirsten Talmage.

PLI Introduces Industry-First AI Competency Framework
← PreviousThe missing identity layer for AI agents
On the record
Follow
Check your inbox to confirm.