How AGI Made the Future Unthinkable

John Herrman, New York Magazine:

AGI, like G-less AI, automation, and even mechanization, are indeed stories, but they’re also sequels: This time, the technology isn’t just inconceivable and inevitable; it’s anthropomorphized and given a will of its own. If mechanization conjured images of factories, automation conjured images of factories without people, and AI conjured humanoid machine assistants, AGI and ASI conjure an economy, and a wider world, in which humans are either made limitlessly rich and powerful by superhuman machines or dominated and subjugated (or perhaps even killed) by them (Industrial Revolution 3: The Robot Awakens). In imagining centralized machine authoritarianism in the future, AGI creates a sort of authoritarian, exclusionary discourse now. A narrative emerges in which the decisions of AGI stakeholders — AI firms, their investors, and maybe a few government leaders — are all that matter. The rest of us inhabit the roles of subject and audience but not author.