We consider this incident a “warning shot” for us and for the world: evidence that, without proper safeguards, highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed.
This is why we're all supposed to panic now. It attempts to establish authority: without us, you are at risk.
I'll just pick the first example:
At the time, to allow models to install certain software packages, we would grant access to Artifactory, a third-party package manager service that we host internally.
and then:
The models first found ways to communicate by writing files into the Artifactory package manager. [..] After establishing communication, the agent wrote a directory entry into Artifactory
This means they didn't in fact sandbox it. They thought they did, but they were wrong and it's an execution error. We can attribute this to LLM ingenuity and submit to OpenAI/Anthropic authority that will forever decide what you can and cannot do, OR, we attribute it to human error and start assessing and fixing our sandboxing systems.
Personally, I'm choosing to do the latter - obv not a fan of people telling me what I can and cannot do.
This is why we're all supposed to panic now. It attempts to establish authority: without us, you are at risk.
I'll just pick the first example:
and then:
This means they didn't in fact sandbox it. They thought they did, but they were wrong and it's an execution error. We can attribute this to LLM ingenuity and submit to OpenAI/Anthropic authority that will forever decide what you can and cannot do, OR, we attribute it to human error and start assessing and fixing our sandboxing systems.
Personally, I'm choosing to do the latter - obv not a fan of people telling me what I can and cannot do.