Too Little, Too Late. Time for a Societal Review Board

Too Little, Too Late. Time for a Societal Review Board

A researcher walked out of Anthropic saying the labs are "gambling with our lives." His former colleague, who leads the company's alignment science team, backed him in public and put the odds of AI ending humanity within a decade above one in ten. Within a day the episode was being cast as fear marketing for an IPO.

It could both be true. I agree that AI has the potential to cause global damage, and that we need to step up our efforts to ensure this is done correctly.

In the same week OpenAI's president said we are in the AGI era, and Jensen Huang declared that AGI has arrived. I do not believe we have reached the AGI age. Instead, I think these are marketing comments to get more investment, and Nvidia has guaranteed up to $105 billion of financing for the company it congratulated.

Yann LeCun's answer was to put one of these models in a humanoid or a self-driving car and see how it does. I agree with him, but that does not mean we will never reach AGI. It means we have not yet, and we should be frank about that, because such claims scare the market, frighten consumers, and are just plain noise.

So the people building this technology spent five days telling us that what might kill us has arrived. Nobody checks either claim, and the argument about whether AGI exists is the least of it.

While the labs argued about the label, the handovers happened, quietly, in domains far from the AI debate.

The functions changed hands

While the labs argued about the label, something quieter and more consequential happened across five domains at once: functions that a society had assigned to accountable institutions moved to new holders, and the move was made by contract, filing or certificate rather than by anyone's vote.

The pattern is easiest to see in money. Blockchain was built to make banks unnecessary, and the banks took it, kept the efficiency, dropped the politics, and are building one digital dollar token together. The technology has not changed. Only who holds it has.

The same transfer is running through every other domain. In healthcare a machine reports the normal mammogram with no radiologist reading it and a pig kidney kept a man off dialysis for 271 days, so the reader and the donor, two of medicine's oldest scarcities, have started to be manufactured. In energy, two gas plants for Amazon and Alphabet were approved in under five months because nobody in the process had to ask whether they should exist, which means the question of what a utility is for has been answered by whoever could build fastest. In immersive technology the stack that Meta tried to own is being assembled by a handful of separate companies, and one of them went into liquidation with customer content still inside it.

Underneath all of it sits the transfer that makes the others hard to check: the builders pick their own examiners. OpenAI's model scored 62.7 percent on an independent rig and 99.9 percent on the company's own, and only one number traveled.

The ledger, the reader, the plant, the platform, the exam. Each was once held by an institution someone could question, and each has moved to a holder that answers to a contract. We are handing more decisions to machines and to a few companies, and the guardrails have not moved with them.

Regulation arrives too little, too late

A Fortune 200 leadership team asked me this week what governments should do about this. My answer was not more regulation. Regulation written for technology this fast arrives too little, too late.

Governments should instead require frontier labs, and every large organization building advanced technology, to have a board that approves their research before it starts. Call it a Societal Review Board. Ethicists, philosophers, scientists, consumers, regulators, with genuine power, as in the power to fire the CEO or the board if they go reckless.

Universities have run this way for decades. No study on human beings starts without an Institutional Review Board, a committee that can approve it, demand changes, or stop it. I have argued for years that companies whose technology reaches millions deserve the same gate.

It speeds up regulation without the kind that stifles innovation. We cannot expect regulators to step up in time, so accountability needs a new lever.

A board that can fire the board

Picture that board inside the utility affiliate that built two gas plants in under five months. Someone at the table asks whether the plants are net positive, not whether the paperwork was filed. Picture it at the bank consortium, or at the health system removing the radiologist from the normal case.

Now picture it inside a frontier lab whose own head of alignment science says there is no plan yet for controlling superintelligence, on the day the next model is scheduled to be trained. Someone at that table can stop the release until the plan exists, and the researcher who walked out has somewhere to go other than the door.

Members of these boards could even take a form of Hippocratic oath. A doctor's oath binds her to the patient in front of her, whatever the hospital's finances say that quarter. A Societal Review Board member's oath would bind her the same way, so that when she decides, humanity is her main interest and not the small group of people who happen to profit from the decision.

That matters because of what the board replaces. In every one of this week's handovers a test went missing when the function moved: the necessity hearing the gas plants never faced, the second radiologist the normal mammogram no longer gets, the independent examiner the labs no longer have to accept. A board with the power to stop a project is a new lever for accountability where the old test was lost, and it works inside the company at the speed the company moves, not at the speed regulators move.

Now more than ever, we cannot expect regulators to step up in time. Europe's AI Office sent its first requests for safety documentation to more than 30 providers while the models they concern were already in customers' hands, and the fines it can levy arrive after the harm, not before it. A board that must approve the research before it starts is the only mechanism that runs ahead of the release rather than behind it.

The case extends beyond the labs to any large company with a direct impact on society, wherever the technology moves too fast to be regulated. On this evidence that is a utility deciding what a grid is for, a bank deciding what money is, a hospital deciding who reads a scan, and a platform deciding what happens to a school's lessons when the platform dies. As we hand more decisions to machines and to a few companies, this is the guardrail that keeps those developments moving in the right direction, independent of the pace of regulators and lawmakers.

Why the Fortune 500 should want this

Fortune 500 companies have genuine power in this world, and they could even argue for these boards. Counterintuitively, it would slow down the frontier labs and give the incumbents more time to adapt.

They need it. Organizations largely think they still control the market in their domain. Large incumbents move far too slowly for the change the world is experiencing, and new tools bypass them.

With frontier labs attacking a wide variety of industries at once, many leaders are sleepwalking, thinking they still have time to adapt. They do not.

Institutions built to ration something scarce, the radiologist's read, the donor organ, the business day, cease to exist if they do not adapt once it moves to the digital realm. What is your organization still rationing that someone else can manufacture?

The question a Societal Review Board would ask

Every handover discussed this week in the daily Synthetic Minds newsletter looks like a gift from inside the company that makes it. Google pays for a Californian utility's grid program outright, with no ratepayer money, and every megawatt is capacity the public no longer funds. A bank consortium issues its own dollar token and calls it efficiency. A lab ships a model it cannot fully test and calls it progress.

What separates a gift from a transfer of power is whether the thing being built serves the best interest of humanity or of a small group of people. No company can answer that about itself, because the small group is the one asking. That is precisely why the question needs a Societal Review Board, seated inside the company but bound by oath to the people outside it, with the power to say no before the transfer happens rather than to check it afterward.

Nowhere does that question matter more than in AI itself. If the labs' own alignment lead is right that the odds of AI ending humanity within a decade are above one in ten, the board's first job is to make the lab prove it has a plan before the next model ships, and to stop the release if it does not.

If we let the frontier labs and their investors take over, utilities that were decentralized across a society get centralized in the hands of a few, and that is a dangerous precedent.

We need more philosophers in the room

Setting up such boards with the right people is the hard part. A philosopher has to hold her own against the engineer and the CEO, and we have spent years defunding the disciplines that produce her. That is why I have argued for a renaissance of the humanities, and why we need more humanities and arts scholars on every Societal Review Board.

The people building this technology say it could kill us all, and that it has already arrived. Both may be selling, and both may be right. Anything that can be built will be built.

The question that remains is who gets to say no, and whether their no would hold. Ethicists, philosophers, scientists, consumers and regulators, with the power to fire the board. Who would sit on yours?