How do you think you’re going to die?
If you go by the current statistics for deaths in the US, it’s reasonable to expect that cancer or heart disease will eventually get you. But OpenAI Foundation board member and Alignment Research Center Executive Director Paul Cristiano thinks we should all be worried about another major mortality risk. Speaking at a lecture series this week, Cristiano bluntly told the audience in a Toronto Convention Center, “I think for most people in this room, the most likely reason they will die is because of an AI accident.”
In the opening lecture of a three-partseries, Cristiano outlined his worries about the risk of an impending “AI takeover.” He defined that as a situation where goal-pursuing AI systems “call the shots in the same sense that humans today call the shots and what happens on Earth” and where any human trying to stop them “would be as ineffective as if you or me was trying to stop the US military.” Such a situation would likely lead to mass death and perhaps uninhabitable environments for humans, Cristiano said.
Despite his grim assessment of our collective mortality risk, Cristiano admitted in his talk that there’s no “quantitative model” that can “put a number” on the likelihood of this kind of AI takeover. In almost the same breath, though, he said that if AI continues to develop at “the most rapid trajectory,” humanity is “looking at something like a coin flip of an AI takeover,” including a “meaningful risk” of one happening in the next six to 12 months. And even with realistic policy responses slowing things down, the risk remains “exceptionally high,” he said.
From here to there
Unlike some other recent AI doomsday prognosticators who have been a bit vague about the “how” of it all, Cristiano used his lecture to spin a step-by-step tale of escalating AI threats that sounds like it could come from a science-fiction novel. That line of reasoning leans heavily on this summer’s OpenAI/Hugging Face hacking incident as an example of “roughly the things we might expect to see shortly before something really bad were to happen.”