Staff Reports
Anthropic’s boss, CEO Dario Amodei, just rolled out a big idea: slow down the race for ever‑more powerful AIs and let outside watchdogs live inside the labs to check whether these systems are safe. The proposal sounds sensible on its face — who wouldn’t want safer AI? But the way Amodei wants to do it raises big questions about who gets to be the moral referee for the entire industry.
Amodei’s plan: “pace the frontier” and embed evaluators
In a recent essay, Amodei urged the industry to “pace the frontier” by slowing capability growth until safety tools catch up. The centerpiece of his pitch is a unilateral pledge: give third‑party evaluators permanent, employee‑level access to company systems so they can verify safety practices, report incidents, and assess alignment during training. He named Model Evaluation and Threat Research (METR) as an example of such an evaluator. That part of the plan is concrete and headline‑grabbing — a private watchdog inside private labs checking private systems.
What METR says — and why skeptics sniff around
METR already published a Frontier Risk Report from a pilot it ran with several labs, warning that internal agents plausibly had the means and opportunity to do small rogue deployments. METR also says it takes institutional grants, doesn’t accept direct money from frontier AI companies, and lists staff with serious credentials. Fine. But critics have raised persistent questions about METR’s independence, donor ties, in‑kind support from labs, and overlap with the effective‑altruism ecosystem. Tech leaders and investors have publicly asked for clearer, contract‑level disclosures and audit trails. In short: METR makes bold claims, and forces outside the group want to see the receipts.
Why a private club policing AI is a bad look
Hands‑off safety sounds good until you realize it hands control to a small, unelected circle of evaluators and donors. That opens the door to regulatory capture or cartel behavior: companies agreeing to slow only when their chosen referees say so, and those referees coming from networks with shared beliefs. Worse, Amodei’s essay doesn’t offer clear, objective metrics that would trigger a coordinated slowdown. Without bright lines, “pacing” is a vague instruction that can be bent to political or ideological ends. We should want safety checks. We shouldn’t want a secretive priesthood telling the country what counts as “moral” AI.
What responsible oversight actually looks like
If we’re serious about safe AI, demand transparency and public accountability. Before anyone embeds an evaluator, we should see clear contracts that spell out publication rights, redaction rules, conflict‑of‑interest policies, and donor disclosures. We need objective gating metrics for any slowdown and independent audits of evaluator funding and methods. And yes — government has a role: set enforceable standards, not hand the keys to a private guild. Let private groups help develop standards, but they can’t be the referees, judges and rule‑makers all at once.
Amodei’s proposal starts useful conversations about safety. But useful ideas can become dangerous when they’re controlled by a cozy circle behind closed doors. We should welcome third‑party evaluation — provided it’s truly independent, transparent, and accountable to the public, not to a clubhouse of like‑minded donors. Ask the hard questions now: which organizations will be embedded, what do their contracts say, who funds them, and what exact triggers will force a slowdown? Anything less is asking us to trust a handful of self‑appointed moralizers with the future of technology — and that would be a trust we shouldn’t give lightly.
