The Probability of AI Causing a Catastrophe: Experts Weigh In

Former OpenAI safety researcher Paul Christiano warns that there is a 10-20% chance of AI takeover, resulting in the death of many or most humans. He believes that the end by AI will come more gradually, with a year's transition from AI systems that are a pretty big deal, to kind of accelerating change, followed by further acceleration. Christiano's new nonprofit, the Alignment Research Center, is based on the concept of AI alignment, which broadly defined back in 2018 as having machines' motives align with those of humans.