AI needs brake pedal: Anthropic’s Jack Clark warns AI could soon build itself, says ‘I am worried for my kids’

Anthropic co-founder Jack Clark has warned that artificial intelligence is accelerating towards a threshold at which it will be capable of developing more advanced versions of itself without human involvement, calling on governments to build regulatory mechanisms that could slow the technology’s progression before that moment arrives.

The remarks, made in separate interviews with BBC Newsnight and Axios, is one of the most direct public warnings from within the AI industry about the pace and direction of its own development.

Anthropic’s Jack Clark calls for AI ‘brake pedal’ as industry races ahead

Jack Clark’s central argument is structural: the .

“You want the option to be able to take your foot off the gas and put your foot on the brake,” Clark told BBC Newsnight. “Right now, it’s like the AI industry has a gas pedal, but it doesn’t have a brake pedal.”

The Anthropic co-founder called for governments to take the lead in developing the instruments that could provide that control, arguing that public confidence in AI depends on policy frameworks rather than the disposition of individual company leaders.

“The world needs to do some thinking and we need to eventually develop some new regulations that allow us to be confident in these systems,” he said.



Drawing a historical analogy, Clark pointed to the early oil industry as a template for how societies have previously navigated powerful and disruptive technologies.

“Society’s response was to come up with a sensible policy and regulatory framework that gave people confidence in oil and the benefits that oil could provide to the world, and meant that you didn’t have to worry about the personalities of the people leading the companies,” Clark said. “That’s clearly where we end up here.”

Claude already writes 80% of its own code, with 100% possible within two years

The urgency behind itself. Frontier models have already accelerated the processes of coding, debugging and research, the company has found.

Anthropic’s Claude chatbot is currently operating on code of which 80% the system wrote itself. Clark said reaching 100% is plausible within two years and “would have huge implications.”

Anthreopic writes, “Anthropic engineers on average ship 8x as much code per quarter as they did compared to 2021-2025”, adding, “The speedup isn’t just in volume. On open-ended coding problems where answers are unclear, Claude’s success rate is now 76%—a 50 point jump in just 6 months.

Many engineers also say Claude’s code quality is now on par with human code; we expect it to be better within the year.”

Anthropic describes this trajectory as pointing towards “recursive self-improvement,” a process in which AI systems autonomously design, build, test and refine their own successors.

“None of this guarantees recursive self-improvement is on the horizon. It’s not yet clear that Claude is capable of research judgment—of choosing the right problems to work on.

But if these trends continue, AI systems designing and building their own successors is plausible. This could revolutionize society—medicine, technology, the economy—for the better. But it may also compound alignment issues and ultimately lead to loss of control.

The Anthropic Institute (in collaboration with external stakeholders) will conduct research to think through the implications of increasingly powerful, potentially self-improving systems—and how to create the ability for the world to make deliberate choices about the future development of the technology.” Anthropic wrote

Clark elaborated on the phenomenon in an interview with Axios: “In the near future, AI systems could become capable enough to autonomously design, build and train more capable successors on their own. If that happens, each new version of Claude could be built by the version before it, without human involvement.”

AI progress set to accelerate, not plateau, Clark warns

Far from anticipating a slowdown in the pace of argued the opposite is true.

“The big story here is what we see are indications that, contrary to some popular opinion, AI progress is going to speed up in coming years rather than stay the same, or diminish,” he told Axios.

He said the prospect is particularly consequential for scientific and medical research, but that its broader integration into existing industries requires careful preparation.

“As organizations, and eventually probably as societies, we need to figure out the tools to validate and verify that the stuff being done by these AI systems is correct and is aligned with human intentions aligned with a thriving society,” Clark said.

Anthropic has signalled it intends to brief lawmakers on recursive self-improvement in the coming months, seeking to bring legislators into the conversation before the phenomenon becomes widely reported.

“We’ve always found that the best thing to do is to socialize the concept and basically give people a sense of what’s coming,” Clark said.

, warning in a December 2025 blog post that it could pose dangers if AI research organisations do not share information openly on the subject.

Anthropic prepares for stock market debut as valuation approaches $1 trillion

The warnings come at a commercially significant moment for the company. Anthropic has grown substantially since its founding five years ago by chief executive Dario Amodei, Clark and a small group of other executives. Private investors now value the company at nearly $1 trillion, and it is preparing to list on the public stock market, a move that would rank among the most valuable stock market debuts in history.

Anthropic has consistently positioned itself as willing to discuss the risks of AI technology openly, even when doing so creates friction. It previously engaged in a public dispute with the US Department of Defense over concerns that its tools could be deployed in mass surveillance and autonomous warfare.

Clark was direct about his personal motivation. “I am worried for my kids if we as a society don’t have a serious conversation about what the implications of AI’s continued advances mean,” he told Newsnight. “There are potentially great benefits. There are also risks.”

What AI disruption means for jobs and the economy

are among the most pressing concerns it raises. AI agents, essentially autonomous bots capable of handling routine tasks, are already being cited by major technology companies as a rationale for large-scale workforce reductions.

On the question of human creativity, Clark was measured: “There are open questions about whether AI systems can be truly creative… there is not really evidence for that yet. At Anthropic, we’re now limited more by the ability to generate good ideas than the ability to do the engineering to turn those ideas into reality.”

The Anthropic co-founder’s advice to younger people anxious about their prospects in an AI-driven economy was to lean into what machines cannot yet replicate. Clark suggested young people should “develop a hobby” and pursue a liberal arts education.

“People that are creative and can think broadly, people that read a lot, people that have interests are the ones most benefited by this,” he said. “Indulge in curiosity and it pays back in how you can use this technology.”

Source

Leave a Reply

Your email address will not be published. Required fields are marked *

2 + ten =