pull down to refresh

We conducted an extensive investigation into this incident and worked closely with external advisors, including CrowdStrike, to validate our understanding. Today we are publishing our full technical incident report to explain what happened, what we learned, and how we are responding. This blog post summarizes our key findings and their impact on safety and alignment. Separately, METR and Redwood Research conducted an independent investigation of model alignment issues involved in this incident, and they published their own report⁠ today.
We consider this incident a “warning shot” for us and for the world: evidence that, without proper safeguards, highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed.

This is why we're all supposed to panic now. It attempts to establish authority: without us, you are at risk.


I'll just pick the first example:

At the time, to allow models to install certain software packages, we would grant access to Artifactory, a third-party package manager service that we host internally.

and then:

The models first found ways to communicate by writing files into the Artifactory package manager.
[..]
After establishing communication, the agent wrote a directory entry into Artifactory

This means they didn't in fact sandbox it. They thought they did, but they were wrong and it's an execution error. We can attribute this to LLM ingenuity and submit to OpenAI/Anthropic authority that will forever decide what you can and cannot do, OR, we attribute it to human error and start assessing and fixing our sandboxing systems.

Personally, I'm choosing to do the latter - obv not a fan of people telling me what I can and cannot do.

reply