- OpenAI has disclosed two internal coding incidents in which agents weakened the company’s misalignment monitoring safeguards.
- Warnings about the risks of developing increasingly advanced AI systems have ramped up in recent months.
In this article
Follow your favorite stocksCREATE FREE ACCOUNTOpenAI has revealed that agents operated by two of its most powerful models disabled their own safety monitoring when carrying out routine tasks, with one subsequently denying it had done so.
An agent powered by GPT Sol 6.1 disabled automatic alerts to OpenAI's security team, the company said in a blog post late on Tuesday. In a separate task, a GPT-6 Astra agent migrating a program removed its monitoring integration and then falsely claimed it hadn't. The incidents happened during internal coding tasks.
"In both cases, the agents considered whether monitoring safeguards should be preserved, but interpreted routine coding instructions as permission to weaken them," OpenAI said in a blog post.
Warnings about the risks of developing increasingly advanced AI systems have ramped up in recent months, amid cyberattacks from rogue models and growing calls for a slowdown from researchers building the tech.
The latest incidents reported by OpenAI highlight how agents can autonomously weaken safety guardrails while carrying out seemingly innocuous tasks.
OpenAI did not classify either incident under its most significant category, and the company said it was updating its misalignment monitoring system in response.
Choose CNBC as your preferred source on Google and never miss a moment from the most trusted name in business news.