We need to make skeptics properly scared. And we need to make CHINA properly scared.

This was the week that AI safety hit the big time. A 27-year-old AI researcher named Jacob Coxon quit his job at Anthropic, declaring that OpenAI and Anthropic are racing to create technology that could destroy the human race:

https://x.com/hilbertspaess/status/2097476196791709843

Other researchers echoed Coxon’s concern, stating their belief that AI has a reasonable chance of killing all of humanity within a very short space of time:Other researchers echoed Coxon’s concern, stating their belief that AI has a reasonable chance of killing all of humanity within a very short space of time:

https://x.com/EvanHub/status/2097497037956891126

I’m not sure why this resignation and these statements went mega-viral. Plenty of researchers have made similar moves, and similar statements, over the past few years! Geoffrey Hinton, one of the pioneers of modern AI, quit Google back in 2023 over safety fears. Daniel Kokotajlo resigned from OpenAI in 2024, saying that the company wasn’t behaving responsibly in its drive toward superintelligence. William Saunders and Steve Adler did something similar. Mrinank Sharma left Anthropic earlier this year, and wrote a pretty well-read blog post about it.

What’s more, it’s been clear for years now that “AI could kill humanity” is a very common belief among AI researchers. Grace et al. (2024) interviewed thousands of AI researchers in 2024, and found that more than half thought that artificial superintelligence has a significant chance of making the human race go extinct (or causing similarly bad consequences):



...read more at noahpinion.blog