Anthropic Co-Founder Urges AI Development Pause and Stronger Safeguards

By: Harsh Vardhan

|

Updated on: 06-Jun-2026 06:00 PM

7,690 views

Follow Us:

7,690 views

Anthropic, a leading AI company, has renewed calls for stronger safeguards in artificial intelligence development. The company recently warned that future AI systems could design, build, and train their own successors with little or no human involvement. Co-founder Jack Clark spoke to BBC Newsnight, stating that the industry needs a way to slow down progress if necessary. He argued that current AI development is accelerating without enough checks.

Key Highlights

• Anthropic warns AI could soon design and train its own successors with minimal human input.
• Co-founder Jack Clark calls for regulatory safeguards to slow AI development if needed.
• AI technologies like agents may disrupt jobs but creativity could remain a human advantage.
• Clark advises young people to develop hobbies and pursue liberal arts education.

Clark explained, "You want the option to be able to take your foot off the gas and put your foot on the brake. Right now, it's like the AI industry has a gas pedal, but it doesn't have a brake pedal." Anthropic, which created the Claude chatbot, competes with OpenAI's ChatGPT and Google's Gemini. The company’s warning is significant because it is building some of the world’s most advanced AI models.

Concerns Over AI Self-Improvement

The main concern centers on a future where AI systems help design and train their own successors. In this scenario, today’s AI could create tomorrow’s AI, reducing the need for human input. Anthropic has stated that its Claude AI is already assisting with research and development tasks. However, the company clarifies that AI systems are not yet capable of fully designing and training new models without humans. Anthropic also acknowledges that this stage may never arrive, but urges governments and regulators to prepare now.

In a recent blog post, Anthropic said, "We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for." For many people, the idea that AI could improve itself sounds like science fiction. Clark warned that if AI systems begin improving future AI systems, technological progress could speed up rapidly. He stressed the need for government oversight and clear regulations to keep humans in control.

Regulation and Economic Impact

Clark compared the current AI boom to the rise of the oil industry over a century ago. He noted that governments eventually created rules to manage the risks and benefits of oil. Clark believes AI will need a similar regulatory approach. He also highlighted the risk of economic disruption. AI technologies like "agents"—AI bots that perform routine tasks—could eventually replace certain jobs. This concern comes as major technology companies have laid off workers, citing AI tools that can do work previously handled by many engineers.

Despite these risks, Clark said humans may still have an advantage in creativity. He noted that there is little evidence AI systems can be truly creative. At Anthropic, Clark said the main limitation is generating good ideas, not engineering. For young people worried about their future in an AI-driven economy, Clark advised developing hobbies and pursuing a liberal arts education. He said creative people who read widely and have varied interests may benefit most from new technology.

Clark’s comments about hobbies sparked discussion online. He clarified that his advice is not based on fears of AI replacing people, but on the belief that those in technology should have interests beyond computers. Clark encourages new Anthropic employees to develop non-technology hobbies and to spend time away from screens.