tl;dr - "we must remember that there is great popular and political demand for techno-apocalypse stories."
This article does a very nice job of reviewing the rise of the popular belief that technology will kill us all.
It begins by pointing out that this was not always the case:
Ben Franklin watched the prototype hot air balloons and immediately realized that air power would someday transform war, but rather than feeling anxious about the destruction it would cause, he hoped it might “[c]onvinc[e] sovereigns of the folly of wars.” This futurism was unusual even among intellectuals like Franklin.
But alas sometime after the 1860s it began to be popular to claim that this or that technology would lead to human extermination.
Apparently, there was a time when people were very concerned that airplanes were going to be the end for us:
*[T]here is, as has been most truly said, no way of complete disarmament except the abolition of flying. Now that, again, is impossible. We have never known mankind go back on a new invention. It might be a good thing for this world, as I have heard some of the most distinguished men in the Air Service say, if man had never learned to fly. But he has learned to fly, and there is no more important question, not only before this House, but before every man, woman and child in Europe, than: “What are we going to do with this power now we have got it?"
The author seems to identify WWI as the real turning point when the popular discourse began to feel that apocalypse was a likely outcome of technological progress.
But for people who had imagined their technological superiority also made them morally superior to past civilizations, and who had long forgotten the last time their great powers fought a total war targeting civilians en masse, World War II felt like a near-apocalypse which came within a hair’s breadth of destroying civilization entirely. The illusion of rational progress “advanc[ing] steadily toward some kind of Utopia” was wounded by the First World War, and killed by the Second.
But now it seems we are on a snappy conveyor belt of technology driven apocalypses:
Much as the fears of nuclear apocalypse switched from radiation to nuclear winter when a new justification was needed, the ecological fears switched from pollution to global warming. In the late 1980s, ecological projections that carbon emissions would raise global temperatures by several degrees celsius and cause substantial disruptions to local climates and human life broke into mainstream awareness
As stated at the top: such apocalypses are useful to people who want to build movements or make money -- we are all going to die, unless you buy my product turns out to be pretty good marketing. Sadly, the people who capitalize on apocalypses that never arrive don't seem to bear much downside:
There is no big finale to the debate, there is no explicit acknowledgement that the apocalypse is cancelled. People simply wander away from the subject. This seems to be less because the core arguments get refuted with reason and evidence, and more because people slowly lose faith after enough of the activists’ predictions about worldwide famines in the 1970s, or cities flooded by 2015, do not come to pass.
Interesting read.
Related, I watched part of @justin_shocknet's video link. Sacks points out an interesting opinion piece (because he's mentioned in it, lmao) re:
doom trolling[1]:It's not just Anthropic that does this though; they all do it. Even Huang, who can really just stfu and sell GPUs all the same.
archived link ↩
The besties have been making this point for awhile, AI has terrible representatives...
Altman was really bad, has gotten better, but the damage is already done. Everyone suspicious of Zuck no matter what. Dario the communist bad actor. Demis is brilliant, but nobody at Google is relatable to normies... Musk is very positive but the EDS psyop has the people that need the most convincing tuning him out generally.
2 notes:
2020-2023 Musk was doomer #1 though, but I feel (probably wrongly) that he was more sincere in his worry, maybe too many Iain Banks books consumed. He significantly changed his stance since, just kept the economic doom angle. I've always felt Altman just copied the particular Elon-as-a-doomer vibe - it generates tons of attention.
Hassabis allegedly has the same rhetoric as Amodei, per #1508478:
There's a section in The Nuclear Arms Race and the Psychology of Power, that points out conceptual inertia wrt the nuclear reality vs conventional terminology:
Which points to that if you can nuke your enemy to oblivion 21x and they you merely 20x, there isn't really superiority or inferiority. There is just total annihilation the moment a button gets pressed.
The same somewhat goes for this alleged AI "arms race" - where "arms" is the wrong word to begin with. For all the conventional talk about "bad actors" and "cyber capabilities" or the constant mistake of trying to project geographic borders onto digital assets... it is poorly applied terminology that misses the point that that capability is taught from a human to a simulator software and not magically assigned within the confines of a nationstate.
Meaning: if I wanted to, I could instead of building automated frameworks for defensive security, spend my time on training a "cyber capability", and I'd have it. One doesn't need Anthropic models for that, or the latest GPT; that's just a
fable.I wouldn't say musk was a doomer, started oai in 2015 not because doom was a certainty but the way to mitigate doom is distribution
Sounds like Demis is similar
This is the key particularly around "weaponization"... A punji pit, CRAM, or a heat seaker is an autonomous weapon too by definition
The difference between you having the capability and say a CCP party member in Beijing is the NSA can direct boots to your door, not the case across physical security zones, so it's inevitable that digital security zones will trend toward physical security zones. AI in that case is seen as the front lines, both defensive and offensive.
I'm just really glad I'm no longer working in CIP since all this shit came into focus.
Yeah doomer is the wrong word - perhaps it should be AI-alarmist, in the most pragmatic meaning of it. Not "we're doomed", but "this is a serious problem that needs to get solved".
With Hassabis I feel like it's all more a hypothetical collateral damage thing that is acceptable to him because we will have AGI, and with both Amodei and Altman I feel they just want to capitalize on it where the only difference between the two is whether they actually believe what they are saying. Of all these, I have the most sympathy for Musk's posture. Too bad he distracted himself with politics last year, it visibly impacted Grok's long tail.
I don't see how a digital security zone makes a real difference at the scale of nationstate, let alone hemisphere. It will always be leaky as long as it's connected and the outer layer of a security solution is always the weakest while having the most negative impact, exactly because loss of functionality at that layer has the most negative impact. The functionality vs security tradeoff simply prevents scaling in scope.
Things like firewalls or export controls or sanctions, let alone conventional warfare against other states, are inefficient. They are not long-term solutions and they increase future risk. Also, the granularity of "the enemy" shrinks in tandem with the granularity of "sovereignty" and with AI, even dumb af LLMs, individual sovereignty is now in reach for the masses.
But the individual is a liability as ideology/narrative/propaganda is more impactful than ever, that's the first thing that changed 2 decades ago with the mass adoption of the internet. But there hasn't been a working defensive answer to that, at all. The real threat is not some LLM, it's the beliefs of those that operate it.
Indeed, but it's also evident that we're trending towards a splinternet accordingly.
The internet is still inherently a physical thing, infrastructure like cables and datacenters. Militaries exist to inflict state will on physical infrastructure.
Until we can communicate with high-bandwidth telepathy, digital security zones will increasingly reflect physical boundary.
This is really the only thing that slows the splintering, I'm sure China would cut off the the world at large from the domestic internet if not for the economic ramifications.
Russia famously had a botched block against a bunch of western IP's, just because it was botched though doesn't mean these efforts will stop, they'll just iterate.
I'm not sure how well the UAE's firewall works, only learned of it recently, but in any case policy alone keeps the bulk of the population chilled. Those that are undeterred within their physical security zone face physical risk over digital presence.
For now they have to boil the frog slowly, but just because the state can never have perfect control doesn't mean they won't iteratively accumulate meaningfully more control than they have now.
Satellite constellations throw a little more chaos into the chaos vs. order equilibrium, now submarines cutting cables matters a little less, which is also why those are themselves national security objectives for the major powers. But also think about special operators in enemy territory using sat phones in the 80s... extrapolate that forward to a backpack sized dish that can bridge operations into a countries gooey center and bypass the hardened shell.
I think that's mostly for show to firm up the gooey center domestically, not from domestic actors, but coordinated foreign adversaries operating within the physical security zone.
How many servers does China run within our borders under shell corporations, backdoored appliances, etc? Remember the SIM farm in NYC a year or so ago? It's those kinds of resources that keep the state up at night, not individuals. Weaker states are more susceptible to individual actions than global powers with a large intelligence apparatus, but as for major powers, as long as you are in their physical security zone they know what you're doing and can roll up at any time.
Yes. It may be the thing that prevents complete breakup - after all, growth is a lot harder if there is only an internal market. But even if it doesn't, the art of magically making things cross borders against the will of the one in charge is as old as the first person in charge to ever will such a thing. It's much easier digitally because transmission == copy. Remember stuxnet.
I'll take your word for foreign adversaries causing the majority of sleepless nights, for now.
Socrates literally had people against writing because it would cause people not to exercise their memory, that writing would remind people so they didn't have to remember... and that reading silently to oneself would destroy public discourse
When the printing press arrived the doomers claimed an uncontrolled surplus of misinformation or information overload
When bicycles were invented "bicycle face" was a the fud, straining to pedal would mess up your face, especially women... but also simultaneously those now dog faced women would bicycle to the next town to whore about
Railroads would cause "railway spine" and brain disorders because the human body isnt meant to travel over 30mph, death of communities, etc.
Let's not even get into the monster that is electricity...
All before WW1... human software is exactly the same as it has always been. Nothing is new, ever.