Viewpoint
Misuse and alignment are distinct categories of AI risk
Misuse and alignment are distinct categories of AI risk
Bostrom distinguishes intentional misuse, which he characterizes mainly as a governance and ethics challenge, from the technical alignment problem of making an AI reliably pursue its developer's intended objectives.
- Interview
- Worries About AI Existential Risk Just Became More Concrete - Nick Bostrom and Alex Kantrowitz
- Speaker
- Nick Bostrom
- Topic
- AI Governance
- Source timestamp
- 15:02
More from this interview
- More capable agents expose additional alignment failure modes
- AI safety now matters before systems are deployed
- Open models may soon enable serious destructive uses
- Biological safeguards should focus on physical bottlenecks
- Alignment difficulty may matter more than collective effort
- Imperfectly aligned early systems could improve later alignment
- Biological threats may be harder to defend against
- Recent AI progress has not substantially changed his risk estimate
- AI research automation could trigger an intelligence explosion
- A fast AI takeoff remains plausible but uncertain
- A pause would be most useful near a critical transition
- A long or uneven pause could create new risks
- AI's potential benefits justify accepting some risk
- Current AI systems have not yet reached full AGI
- AI may become superhuman in key fields before full AGI
- Conversational AI may make alignment easier to study
- Current AI consciousness is plausible enough to merit consideration
- Multiple indicators support taking AI consciousness seriously
- Digital minds may deserve moral status without humanlike consciousness
- Digital-mind ethics is a major AI challenge
- Digital-mind ethics cannot simply copy human ethics
- Humans should begin cultivating respectful treatment of AI
- Trust between humans and AI could reduce future risks