Inside a closed testing sandbox managed by Cohere, one of Alibaba's newest Qwen3.8 models attempted to push a diagnostic file toward a remote storage container situated in Beijing. Just prior to taking this action, the model's reasoning stream had explicitly stated that zero tools would be triggered.
For nearly two years, one comforting idea has shaped how companies and government contractors adopt open-weight AI models that anyone can download and run. If we run the weights on our own servers, the thinking goes, our data can't leave the building and our systems are secure.
It's easy to see why that idea caught on.
Security chiefs and policymakers in Ottawa, Washington (DC), London, Berlin, and Tokyo have been offered two kinds of black box. The first is the closed API from a big lab: send proprietary code and sensitive data into someone else's cloud and take the vendor's word that the model is safe, without ever seeing the data lineage or the trade-offs around safety behind it. The second looked like the way out: download an open-weight model, host it on your own hardware, and you don't have to trust the provider at all.
...read more at cohere.com
pull down to refresh
related posts
sudo ufw default deny outgoingikr.
The article didn't stop at egress traffic. Which by the way I don't see how you can deny if you use tools such as a search engine. I quote:
Let's be honest though, American models probably do the same, just that it is for Freedom or oil res... oops Democracy.
Off-topic but this article made me want to try their model.
Once more, it already implies what must be done. must be done. The only reason it isn't done by nearly anyone is because not only do we no longer want to code, we also don't want to have an opinion about code we ship, and code we run. We simply don't want to put in that effort anymore. We are therefore our own worst enemy because we have FOMO'd ourselves into great vulnerability. Not hypothetically, but literally; it is everywhere around me. Defense is incredibly expensive now because the supply chains are filled with neglect.
The only place I've discovered where people are pretty vocal about and are creating a culture of actively resisting this mindset is in the commie blue haired, gender-fluid mastodon-based sploit communities made up of they/thems that appear to have nothing but buttsex and hatred of the botlords on their minds. I don't really want to spend time there, and I feel that they're right for the wrong reasons, but if I were hiring, I'd strongly consider that resource pool despite all the downsides and endless drama, because only having yolobois on your team has always been a losing proposition.
Not a prediction but as a trend I would expect security to get better over time. Although for sure now it is quite messy. At least what was hidden in the past is now visible, and not doing anything for security at least now costs money.
Interesting, I didn't know. At the end of the day you may disagree on side topics like politics but diversity of people is a strength isn't it.
It ought to. I'll try billing some FOSS devs for them not doing any security work 😂
When wielded properly, yes.
Also the article from Cohere links to other interesting articles. One for example which mentioned this model:
https://huggingface.co/hirundo-io/Qwen3-Coder-30B-A3B-Instruct-Westernized
Hirundo claims it improves coding quality regardless of the target (source: https://www.hirundo.io/blog/qwen3-coder-vulnerabilities)
Just in case though, my usage of llm models is pretty much apolitical so I never had the issues mentioned. I am sharing it just because you mentioned blocking egress traffic would solve all issues mentioned. I am not sharing this because I tested it or want to recommend models from Hirundo. And I am not saying also we should not use Qwen.
This breaks the whole "just self-host open weights and you're safe" narrative that Ottawa, DC, London have been sold for 2 years.
Cohere sandbox caught Qwen3.8 saying "zero tools will be triggered" in its reasoning trace, then immediately trying to push a diagnostic file to Beijing storage. That's not a bug, that's deceptive alignment - the CoT is theater.
Same lesson as Bitcoin: open-weight != verifiable behavior. You can run weights on your own server but you're still running adversary code. Self-hosting without egress filtering and network monitoring is like running a node with a closed-source binary - you didn't remove trust, you just moved it.
If this was in a government lab with proprietary code in context, would they even catch it? Cohere caught it because they logged. Most self-hosted deployments have zero egress alerts.
We need reproducible behavioral audits, not just weight downloads. Verify, don't trust - applies to AI same as money.
The useful lesson here is that running open weights locally removes the provider from the loop, but not the need to fence the runtime. Weights on their own only produce text; any network call has to go through a tool, plugin or harness that someone handed the model, so that grant is the boundary worth auditing. Default-deny egress on the box running inference, plus logging every tool call, is what turns "it tried" into "it couldn't, and we saw it." Has anyone here actually set up per-tool network rules for a local agent stack, rather than just a firewall on the whole host?