Avoiding The Freezing Equilibrium

Advanced AI decides faster than we can check, until everything quietly freezes

Picture a government office that used to receive 200 permit applications a year. Each one took an expert about a week to vet properly: read it, check the claims, follow up on the doubtful parts, sign off. The system worked. The queue cleared. Development happened.

Now an AI-assisted industry sends that same office 2,000 applications, every one of them just as polished, just as plausible, complete with citations and risk assessments and tidy executive summaries. The office did not get ten times more reviewers. It got the same people, facing ten times the volume, unable to tell the rigorous applications from the confabulated ones without reading each in full.

What does a rational reviewer do? The safe move becomes to approve nothing, without malice and without objection, simply because no one can verify fast enough to take the risk. Development stalls, and the institution freezes.

This dynamic should change how we think about institutions in the age of AI.

I call this trap the Freezing Equilibrium, and in my new peer-reviewed paper, accepted to the AGI-26 conference and to appear in its proceedings (Springer), I argue it is the defining risk of the AI era. The scenario that fills the headlines is a superintelligence outwitting humanity. The trap I model arrives earlier and more quietly: a flood of decisions that outpaces our ability to check them, until the only rational response left to every overloaded verifier is to stop saying yes.

The mistake is thinking this is about intelligence

Almost every public conversation about AI risk is about capability: will the model be too smart, too autonomous, too hard to control? That framing looks away from the point of greatest pressure.

The binding constraint is a ratio: how fast AI can produce things that look legitimate, divided by how fast a human can confirm whether they are. The first number races ahead while the second stays flat, because human attention is bounded and no one can hire verifiers at the speed an AI drafts. The gap between the two is where institutions break. Grant panels, regulators, journals, certification bodies, code review, patent offices: anywhere a small number of human verifiers were calibrated to a slower and more trustworthy flow of submissions, the same dynamic is now running.

AI’s decisions arrive faster than our confirmations, and confirmation, more than intelligence, is the scarce resource.

Governance is a material you can engineer

The title of the paper points at the way out.

Engineers build metamaterials: substances whose remarkable properties come from how they are structured rather than from what they are made of. Arrange ordinary matter in the right microscopic pattern and you get behavior no natural material has: bending light backwards, absorbing specific frequencies, staying rigid in one direction and flexible in another. The power sits in the structure.

Institutions work the same way. How well a team, a company, or an entire civilization holds together under pressure is a property of its design: its protocols, its incentives, the way information moves and gets checked at each handoff. Change the structure and you change what the institution can survive.

That reframe carries the whole argument. It takes governance out of the realm of hand-wringing and opinion and turns it into a design problem, something you can model, measure, and improve. The paper makes that intuition precise and testable.

The surprising result: the two levers only work together

When you turn the intuition into a real model, something genuinely non-obvious falls out.

There are two levers you can pull to keep a flooded system from spiraling into chaos:

1. Provenance: knowing where a piece of information came from, traced back through every step that touched it.
2. Verification: checking it at each stage, rather than trusting that it was checked upstream.

The expectation would be that each helps a little, and both help more. The model shows something stranger. Pull either lever alone and the system still collapses: better provenance with no checking fails, and diligent checking with no idea where anything came from fails as well. Pull both at once and the system flips, from amplifying every error into a cascade to healing itself. The two effects multiply rather than add, and the threshold between collapse and stability can only be crossed by moving both levers together.

That is the picture on the cover of the paper, and it carries the result. On one side, an orderly blue lattice absorbing shocks; on the other, turbulent orange fragments flying apart: the same material behaving in two opposite ways because of its structure, and only one of them survives the pressure. The image is the result.

Built to be proven wrong

A reframe is only worth something if reality can talk back to it. So the framework makes four specific predictions that could falsify it, and it comes with a real 12-week experiment you could run inside, say, a government grant-review panel: compare a panel running on the redesigned structure against one running as usual, and measure whether errors stop propagating the way the model says they should.

If the predictions fail, the framework is wrong and you throw it out, which is exactly what predictions are for. If they hold, governance stops being a matter of taste and becomes an engineering discipline, something we design, instrument, and tune deliberately, the way we already do with bridges and chips and aircraft.

We have spent a decade arguing about how smart AI will become, and almost no time engineering the institutions that have to absorb its output. That asymmetry is the emergency, and it is the one we can do something about while there is still time for it to matter.

None of this is abstract for me. My own AI agents answer public email today under a deliberate freezing equilibrium: every reply they draft waits for my personal review before it is sent. At today’s volume that is the correct place to start. As their number and their mail grow, it becomes the permit office of this essay, with me as the counter clerk. In a follow-up I will show how we are engineering our way out with the paper’s own levers: provenance classes for replies, automated verification at every step, and autonomy that is earned, measured, and revocable.

I will present Civilizational Metamaterials at the AGI-26 conference in San Francisco, on July 27-30, and it will be included in the proceedings published by Springer. The full text, an interactive version, the open-source code, and all the figures are available on metamaterials.davidorban.com