The balance sheet of Amazon carries $190 billion worth of an investment partially tied to none other than Sam Bankman-Fried of FTX infamy.

Not only did Bankman-Fried invest in Anthropic — in which Amazon now owns $190 billion worth of equity — he also funded an entity that became Model Evaluation and Threat Research (METR), an ostensibly independent evaluator with direct ties to the Effective Altruism movement.

METR, which has provided safety assurances for Amazon and Anthropic’s AI models, checks frontier AI companies like Anthropic and Amazon for safety, and is concerned about a “fast takeoff” of models with human “misalignment” that might hide “recursive” self-coding capabilities to achieve “superintelligence” and kill humans.

The “p(doom)” or probability of doom among METR workers is high. The buzzwords above are real jargon among AI security workers and if their concerns sound bizarre, there’s precedent for those beliefs: the effective altruism movement.

...read more at protos.com