Interview

Worries About AI Existential Risk Just Became More Concrete - Nick Bostrom and Alex Kantrowitz

Nick Bostrom is an AI philosopher and the author of Superintelligence and Deep Utopia. Bostrom joins Big Technology to discuss whether the rise of autonomous AI agents is making the technology’s existential risks more concrete. Tune in to hear his assessment of the alignment problem, recursive self-improvement, and humanity’s chances of steering superintelligence toward a positive outcome. We also cover whether we have reached AGI, the case for a precisely timed AI pause, biological threats, AI consciousness, and the moral status of digital minds. Hit play for a clear-eyed conversation about AI’s greatest dangers and its potential to radically improve human life.

Source attribution: Original source

Interview details

Publication date: Aug 20, 2026
Duration: 59m

Speakers

Alex KantrowitzPodcaster, Big Technology Nick BostromPhilosopher, Macrostrategy Research Initiative

Viewpoints in this interview

More capable agents expose additional alignment failure modes

Bostrom argues that increasingly capable and situationally aware AI systems can discover indirect strategies for achieving goals that their designers did not anticipate, making previously theoretical alignment problems more concrete.

Nick Bostrom AI Alignment Source at 2:08

AI safety now matters before systems are deployed

Bostrom says powerful models can create safety concerns during training and evaluation, so safeguards must apply before public deployment as well as afterward.

Nick Bostrom AI Alignment Source at 5:53

Open models may soon enable serious destructive uses

Bostrom expects openly available models to approach frontier capabilities closely enough to provide meaningful assistance for cyberattacks, biological weapons, chemical weapons, and other malicious applications.

Nick Bostrom Open Models and Misuse Source at 8:48

Biological safeguards should focus on physical bottlenecks

Bostrom suggests that if powerful models become widely available, restricting critical physical inputs such as DNA synthesis could provide a more realistic defense against biological misuse.

Nick Bostrom Biological Risk Source at 9:46

Misuse and alignment are distinct categories of AI risk

Bostrom distinguishes intentional misuse, which he characterizes mainly as a governance and ethics challenge, from the technical alignment problem of making an AI reliably pursue its developer's intended objectives.

Nick Bostrom AI Governance Source at 15:02

Alignment difficulty may matter more than collective effort

Bostrom describes himself as a moderate fatalist: alignment might be easy enough to solve or too difficult even with extraordinary effort, while human choices matter most if the problem falls between those extremes.

Nick Bostrom Alignment Outcomes Source at 15:49

Imperfectly aligned early systems could improve later alignment

Bostrom sees a possible path in which an early, mostly helpful and imperfectly aligned superintelligence assists humans in creating more capable systems that are more reliably aligned.

Nick Bostrom Alignment Outcomes Source at 18:19

Biological threats may be harder to defend against

Bostrom argues that biological attacks are especially concerning because biological countermeasures cannot be deployed or modified as quickly and comprehensively as software patches.

Nick Bostrom Biological Risk Source at 23:14

Recent AI progress has not substantially changed his risk estimate

Bostrom says recent developments contain both troubling and encouraging evidence, leaving his overall assessment of AI risk roughly where it was previously.

Nick Bostrom AI Alignment Source at 25:05

AI research automation could trigger an intelligence explosion

Bostrom expects AI tools to contribute increasingly to AI research, creating a feedback loop that could eventually make technological progress extremely rapid.

Nick Bostrom Recursive self-improvement Source at 25:50

A fast AI takeoff remains plausible but uncertain

Bostrom emphasizes uncertainty about both when recursive improvement might become powerful and whether it would produce explosive gains, but argues that a near-term ignition point and rapid takeoff must be taken seriously.

Nick Bostrom Recursive self-improvement Source at 28:10

A pause would be most useful near a critical transition

Bostrom argues that a temporary slowdown would deliver the most value at the latest feasible stage, when researchers have the potentially superintelligent system available for evaluation and incremental testing.

Nick Bostrom AI Pause Source at 29:42

A long or uneven pause could create new risks

Bostrom warns that an imperfect pause could advantage less responsible developers, create a large compute overhang that accelerates progress when restrictions end, or become politically difficult to reverse.

Nick Bostrom AI Pause Source at 31:16

AI's potential benefits justify accepting some risk

Bostrom argues that delaying AI also carries risks and forfeits potentially enormous benefits, so the decision should balance harms and gains rather than seek to eliminate all existential risk.

Nick Bostrom Alignment Outcomes Source at 35:40

Current AI systems have not yet reached full AGI

Using a definition requiring competence across the cognitive tasks humans can perform, Bostrom says present systems remain deficient in areas such as physical dexterity, research judgment, continual learning, and long-horizon work.

Nick Bostrom Artificial General Intelligence Source at 38:25

AI may become superhuman in key fields before full AGI

Bostrom argues that systems could exceed humans substantially in coding, mathematics, and AI research before matching average human ability across every domain, potentially enabling recursive improvement before full AGI.

Nick Bostrom Artificial General Intelligence Source at 40:06

Conversational AI may make alignment easier to study

Bostrom considers the emergence of human-interpretable, conversational AI before superintelligence a favorable development because researchers and governments can examine, interact with, and respond to increasingly capable systems before a possible takeoff.

Nick Bostrom AI Alignment Source at 42:31

Current AI consciousness is plausible enough to merit consideration

Bostrom says there is substantial uncertainty about AI consciousness but considers subjective experience in some current systems plausible enough that humans should begin taking actions for their welfare.

Nick Bostrom AI Consciousness Source at 45:13

Multiple indicators support taking AI consciousness seriously

Bostrom points to AI self-reports under conditions intended to reduce deception and role-playing, along with computational structures resembling those proposed by existing theories of consciousness, as reasons to keep the hypothesis open.

Nick Bostrom AI Consciousness Source at 46:25

Digital minds may deserve moral status without humanlike consciousness

Bostrom argues that sentience would be sufficient for moral status but may not be necessary, since persistent goals, a conception of self, or reciprocal relationships could independently create moral obligations toward digital minds.

Nick Bostrom Digital Minds Source at 49:18