Reflecting

That was not quite the plan. We expected to build features; we ended up building a boundary - the place where a model's output stops being a generation and has to become something a person can actually act on. That turns out to be where most of the real work lives, and most of the interesting problems.

The one that took the most out of us was hallucination. A model will state something that reads as fact and is not, and by the time it reaches a person the true and the invented look identical. What runs now is a second model that reads back over an answer, sentence by sentence, against the material it was given, and marks the parts it inferred or could not support from any source - visibly, on the page, each mark opening onto its reasoning. It points at the uncertain parts rather than hiding them.

Provenance was the one we underestimated - saying, of each claim, whether it was measured, reported, or reasoned to, and not letting that collapse into a decorative badge. We hold it as a writing standard in everything we publish, and we have begun teaching the software to carry the same marks through to what a person finally reads. Getting it to hold all the way through, without decaying, is harder than it sounds.

And somewhere along the way we became slightly obsessed with repeatability: a run that records what was asked and why, reproduces the same way twice, and survives the model underneath it being swapped for a better one. A workflow that breaks when the model changes was, we decided, simply not good enough.

The one we did not expect to reach, we reached: wrapping a capability in a bounded, revokable licence, so that permission to act is something you grant and can take back rather than something you police after the output exists. We have now shown it on a real model capability, not a stand-in: the capability loads only under a live licence, and revoking the licence destroys the key that would decrypt it, so even a copy someone has already taken cannot run. It is the idea that has most changed how we think about safety.

The thing we keep coming back to is this. The model is the astonishing part; it is not the hard part for us. The hard part - the part worth doing - is everything that has to be true before a person can trust what it said. Working there has made us more opinionated than we expected about restraint: the machine proposes, the person decides, the system keeps the receipts. We designed the platform architecture with that at its core, rather than writing it onto a values page.

There were three threads we kept pulling on, and wrote up on their own. The first is simply how the thing got built - the timeline, the tools, the early versions that did almost nothing - because the path from a curiosity to a platform this broad is most of the interesting part. The second is what the harness around the model turned out to make possible: capabilities that are not in the model at all, and why keeping it one platform rather than a dozen has mattered more than we expected for trust and safety. The third is where this seems to be heading, now that the framework has a research arm and has begun proposing, testing, and shipping its own next capabilities - the nearest of which lands in a couple of weeks.

Next year the models will be better again. We expect to spend it in the same place.