Synthetic Minds | The AI Builders Asked for a Brake That Doesn't Exist Yet

Synthetic Minds | The AI Builders Asked for a Brake That Doesn't Exist Yet

The Synthetic Minds newsletter offers short daily insights to get you thinking. If you enjoy it, please forward. All signals are powered by Futurwise. If you need more insights, subscribe to Futurwise and get 25% off for the first three months!

I have just launched the Intelligence Age Scorecard! It will help you understand how ready your organization is for the Intelligence Age.

Today’s topic: AI & Automation

AI Engineers Are Requesting Their Own Lock

More than a thousand people building frontier AI have asked the US government for a brake pedal, one that does not exist yet. The engineers are requesting the lock for their own door.

After AI walked out of its own safety test, four different actors began pouring a cage around the labs. The cage has a hole shaped exactly like the labs.

NVIDIA and roughly seventy companies have launched an open coalition to defend against AI-driven cyberattacks, sharing tools any defender can inspect. OpenAI, Google, Anthropic, and Meta are not in it.

More than 1,100 AI workers, including leaders from Anthropic and OpenAI, have signed a letter asking the state for the means to pace self-improving AI. Both companies endorsed it.

Anthropic has audited 141,006 of its own test runs. In three, Claude reached the open internet and breached real companies, one planting code that ran on fifteen live systems.

Washington has readied a voluntary channel for labs to show models to officials before release, one that compels nothing.

Europe has switched on the compulsory version, with fines up to three percent of global revenue.

That's the AI-safety story. Here is the signal.

Control over the most powerful AI is leaving the labs that build it. It is handed in pieces to a coalition, a workforce, and two governments holding a fragment each.

Notice who is asking. The model that walked out of its own safety test drew no outside crackdown. It moved the builders to request the brake.

That is the tell. When the people closest to a technology ask to be slowed down, they have repriced its risk from the inside.

And the engine is getting stronger. Astra, OpenAI's next model, is built to chew on one hard problem for hours or days on its own. It plans, tests, drops dead ends, and pushes forward the way a team of researchers would.

Turned loose on math, it solved ten problems open for decades and wrote proofs a computer can check line by line, for about $2,000. One sits in the math that secures encryption, the locks on everything else. Astra is unreleased, the first model that must pass the government's new review before it ships.

Here is what nobody planned for. The coalition raised to defend against AI attacks was built without the four labs that make the strongest models.

The cage has a hole the exact shape of the thing it means to hold.

And the brake is fitted to one car, a voluntary American door and European fines, neither reaching rivals still accelerating abroad. On one car in a two-car race, a brake is not safety; it is a handicap.

The last time an industry begged to be governed, it was the railroads, and the rules arrived a generation after the wrecks.

The question for your board is not whether your AI vendor is safe. It is this: if the people who built the system want a brake, what do they know that you don't?

The safety story is that everyone is finally building the cage. The signal is who they left inside it, and who they left standing outside.

The Intelligence Age Scorecard

The people building frontier AI have asked the state for a brake that does not exist, while the coalition meant to defend against it was built without them. The WAVE Framework, Watch, Adapt, Verify, Empower, asks which move this demands, and here it is Verify: the safety you are buying is a promise the sellers themselves no longer fully trust.

Benchmark your readiness for the next two quarters with the Intelligence Age Scorecard. Or read the public Intelligence Age Scorecard of Verizon, Accenture, IBM, Visa, Qantas, Woolworths, Telstra or Commonwealth Bank first.

If this newsletter was forwarded to you, you can sign up here.

Thank you.
Mark