AI Slowdown

Plus: Commercial surrogacy regulation, Russian weapons, a new drug in San Francisco, and more…

“Do not underestimate the power of this technology,” he added. “These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.”

I’ve described, over the last few weeks, how swarms of rogue agents have acted against developer intentions and stolen data from other companies. Meanwhile, both OpenAI and Anthropic are preparing to go public sometime soon.

In an essay released this weekend (called “We Must Pace the Frontier“), Anthropic CEO Dario Amodei calls for a global slowdown of AI development, citing such risks as “losing control of AI systems” and “misuse of AI for cyberattacks and bioterrorism.”

“Along with my co-founders and employees, I have grappled with this duality of risk and benefit since the beginning of Anthropic,” writes Amodei. “Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless.” More prudence is necessary, he added; recursive self-improvement—when AI systems design or train their own successors—must be “pursued very carefully, if at all.” In other words, he wants a development slowdown for all large AI system builders.

Amodei’s statement was quickly cosigned by OpenAI head Sam Altman as well as Google DeepMind’s Demis Hassabis and xAI’s Elon Musk. Meanwhile, former President Barack Obama privately urged House Minority Leader Hakeem Jeffries (D–N.Y.) to develop a plan for AI regulation and urged Democrats to develop a clear plan for how AI ought to be regulated ahead of the 2028 presidential election.

“Stop pretending you need anyone else’s permission,” responded venture capitalist David Sacks. “Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR [the nonprofit Model Evaluation & Threat Research] is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.”

“Most of all,” he adds, “stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.”

Scenes from New York: “Over the last three years, Jewish institutions across New York City have added millions more to their budgets for security protections amid an increase in antisemitic attacks, rabbis and Jewish community leaders say,” reports The New York Times. “They have paid to hire more private security guards, increase the number of safety trainings at synagogues and schools, update their security cameras and fortify their buildings. They are also relying on more volunteer security guards at temples and encouraging those who attend services to be especially watchful. Their efforts have come into focus ahead of the 10-day period of somber introspection that takes place between Rosh Hashana and Yom Kippur, the holiest holiday in Judaism.”

QUICK HITS

• “Leaders of the John F. Kennedy Center for the Performing Arts, installed by President Donald Trump, say the center is on the brink of bankruptcy and argue that putting the president’s name on its facade—and thus ensuring his fundraising efforts—may be the only way to avoid imminent and ‘certain fiscal collapse,'” reports The Washington Post.
• “New Geran drones and Banderol cruise missiles are lethal reminders that Russia still has the resources and supply chains to field fresh threats, this time packed with advanced electronics that allow them to find targets accurately and evade interception,” reports The Wall Street Journal. This “was enabled in large part by components sourced from China, in violation of international sanctions, say Ukraine and others, based in part on wreckage from the few shot down so far.”
• A new drug, cychlorphine, is believed to be 10 times as potent as fentanyl. It can evade test strips, and one or two doses of Narcan reportedly aren’t enough to reverse its effects on the body.
• I would not be shocked if we see greater regulation of commercial surrogacy at some point soon:

Associate Editor, Reason

Want to support this newsletter and my work? You can:

Contribute to Reason. This newsletter (and everything Reason produces) relies on the support of readers like you. Contributions help us spread commentary like this to more people.

Forward this newsletter. Know of someone who needs to read it? They can sign up for free at this link.