ICML: The call is from inside the model

The first day of ICML was not what I expected. The scientists and academics here are starting to work on judgment. What it means to read a source and know when to stop looking. How much effort to spend for the outcome you need. This is the stuff I assumed would have to come from legal tech. Sitting here, I'm not sure it will.

There are document-based presentations, and there is talk of intent and judgment; that is much closer to legal tech work than I expected. And this isn't even the AI4law portion.

It becomes knowing how far you can push a capability, seeing where the gaps are, and putting a human in the loop to decide what to do about them. In the loop, on the loop, over the loop, however you want to say it. The work is knowing what to do with the answer.

The other half is cost. Everyone here understands token cost, and a huge share of these papers is about driving it down. We're probably near a high-water mark: we'll keep racing the biggest models and paying for them. But right behind that, a flood of capabilities is landing on cheaper models, and that's where much of the impact shows up.

I've been more and more of a believer that even if the models didn't progress any further than today, there is still a ton of untapped impact to come.