AGI is a lie. Simulation of AGI so that it fools everyone - maybe.

The real danger is people representing LLMs as if they are similar to humans, while it's just a clever trick. Anything that can be backed up, and thus doesn't die, will always be inferior to humans because it won't have a drive to do something.

The thing to watch out for is when humans can get backed up and get a new runtime too - i.e. a new body - and live forever. That will probably be the most dangerous thing.