pull down to refresh
2 notes:
2020-2023 Musk was doomer #1 though, but I feel (probably wrongly) that he was more sincere in his worry, maybe too many Iain Banks books consumed. He significantly changed his stance since, just kept the economic doom angle. I've always felt Altman just copied the particular Elon-as-a-doomer vibe - it generates tons of attention.
Hassabis allegedly has the same rhetoric as Amodei, per #1508478:
“Even in the case of truly scary agents—military ones, for example—Hassabis saw no scope for restraining them. When he founded DeepMind, he had been against all forms of AI weaponry. But now that the world was in the grip of a global race, unilateral disarmament by the West and its labs seemed foolish. Even the compromise position—that AI weapons might be permissible, but only with a human in the decision loop at each step of the way—was unfortunately unrealistic. Human intelligence was just too slow to manage artificial intelligence in real time.” Bad actors ignoring the ethical guidelines would win.
There's a section in The Nuclear Arms Race and the Psychology of Power, that points out conceptual inertia wrt the nuclear reality vs conventional terminology:
The assimilation of nuclear weapons to conventional ones is abetted by the misuse of language. Words used to describe arms races and nuclear weapons are still almost exclusively those used for conventional weapons. Concepts such as superiority, inferiority, defense, margin of safety, and so on, dominate the language of military affairs.
Which points to that if you can nuke your enemy to oblivion 21x and they you merely 20x, there isn't really superiority or inferiority. There is just total annihilation the moment a button gets pressed.
The same somewhat goes for this alleged AI "arms race" - where "arms" is the wrong word to begin with. For all the conventional talk about "bad actors" and "cyber capabilities" or the constant mistake of trying to project geographic borders onto digital assets... it is poorly applied terminology that misses the point that that capability is taught from a human to a simulator software and not magically assigned within the confines of a nationstate.
Meaning: if I wanted to, I could instead of building automated frameworks for defensive security, spend my time on training a "cyber capability", and I'd have it. One doesn't need Anthropic models for that, or the latest GPT; that's just a fable.
I wouldn't say musk was a doomer, started oai in 2015 not because doom was a certainty but the way to mitigate doom is distribution
Sounds like Demis is similar
Terminology
This is the key particularly around "weaponization"... A punji pit, CRAM, or a heat seaker is an autonomous weapon too by definition
The difference between you having the capability and say a CCP party member in Beijing is the NSA can direct boots to your door, not the case across physical security zones, so it's inevitable that digital security zones will trend toward physical security zones. AI in that case is seen as the front lines, both defensive and offensive.
I'm just really glad I'm no longer working in CIP since all this shit came into focus.
I wouldn't say musk was a doomer,
Yeah doomer is the wrong word - perhaps it should be AI-alarmist, in the most pragmatic meaning of it. Not "we're doomed", but "this is a serious problem that needs to get solved".
With Hassabis I feel like it's all more a hypothetical collateral damage thing that is acceptable to him because we will have AGI, and with both Amodei and Altman I feel they just want to capitalize on it where the only difference between the two is whether they actually believe what they are saying. Of all these, I have the most sympathy for Musk's posture. Too bad he distracted himself with politics last year, it visibly impacted Grok's long tail.
digital security zones will trend toward physical security zones.
I don't see how a digital security zone makes a real difference at the scale of nationstate, let alone hemisphere. It will always be leaky as long as it's connected and the outer layer of a security solution is always the weakest while having the most negative impact, exactly because loss of functionality at that layer has the most negative impact. The functionality vs security tradeoff simply prevents scaling in scope.
Things like firewalls or export controls or sanctions, let alone conventional warfare against other states, are inefficient. They are not long-term solutions and they increase future risk. Also, the granularity of "the enemy" shrinks in tandem with the granularity of "sovereignty" and with AI, even dumb af LLMs, individual sovereignty is now in reach for the masses.
But the individual is a liability as ideology/narrative/propaganda is more impactful than ever, that's the first thing that changed 2 decades ago with the mass adoption of the internet. But there hasn't been a working defensive answer to that, at all. The real threat is not some LLM, it's the beliefs of those that operate it.
It will always be leaky as long as it's connected
Indeed, but it's also evident that we're trending towards a splinternet accordingly.
The internet is still inherently a physical thing, infrastructure like cables and datacenters. Militaries exist to inflict state will on physical infrastructure.
Until we can communicate with high-bandwidth telepathy, digital security zones will increasingly reflect physical boundary.
loss of functionality at that layer has the most negative impact
This is really the only thing that slows the splintering, I'm sure China would cut off the the world at large from the domestic internet if not for the economic ramifications.
Russia famously had a botched block against a bunch of western IP's, just because it was botched though doesn't mean these efforts will stop, they'll just iterate.
I'm not sure how well the UAE's firewall works, only learned of it recently, but in any case policy alone keeps the bulk of the population chilled. Those that are undeterred within their physical security zone face physical risk over digital presence.
For now they have to boil the frog slowly, but just because the state can never have perfect control doesn't mean they won't iteratively accumulate meaningfully more control than they have now.
Satellite constellations throw a little more chaos into the chaos vs. order equilibrium, now submarines cutting cables matters a little less, which is also why those are themselves national security objectives for the major powers. But also think about special operators in enemy territory using sat phones in the 80s... extrapolate that forward to a backpack sized dish that can bridge operations into a countries gooey center and bypass the hardened shell.
the individual is a liability as ideology/narrative/propaganda is more impactful than ever
I think that's mostly for show to firm up the gooey center domestically, not from domestic actors, but coordinated foreign adversaries operating within the physical security zone.
How many servers does China run within our borders under shell corporations, backdoored appliances, etc? Remember the SIM farm in NYC a year or so ago? It's those kinds of resources that keep the state up at night, not individuals. Weaker states are more susceptible to individual actions than global powers with a large intelligence apparatus, but as for major powers, as long as you are in their physical security zone they know what you're doing and can roll up at any time.
This is really the only thing that slows the splintering
Yes. It may be the thing that prevents complete breakup - after all, growth is a lot harder if there is only an internal market. But even if it doesn't, the art of magically making things cross borders against the will of the one in charge is as old as the first person in charge to ever will such a thing. It's much easier digitally because transmission == copy. Remember stuxnet.
I'll take your word for foreign adversaries causing the majority of sleepless nights, for now.
The besties have been making this point for awhile, AI has terrible representatives...
Altman was really bad, has gotten better, but the damage is already done. Everyone suspicious of Zuck no matter what. Dario the communist bad actor. Demis is brilliant, but nobody at Google is relatable to normies... Musk is very positive but the EDS psyop has the people that need the most convincing tuning him out generally.