#62 – Häggström on AI Motivations and Risk Denialism

Algocracy and Transhumanism Podcast

In this episode I talk to Olle Häggström. Olle is a professor of mathematical statistics at Chalmers University of Technology and a member of the Royal Swedish Academy of Sciences (KVA) and of the Royal Swedish Academy of Engineering Sciences (IVA). Olle’s main research is in probability theory and statistical mechanics, but in recent years he has broadened his research interests to focus applied statistics, philosophy, climate science, artificial intelligence and social consequences of future technologies. He is the author of Here be Dragons: Science, Technology and the Future of Humanity (OUP 2016). We talk about AI motivations, specifically the Omohundro-Bostrom theory of AI motivation and its weaknesses. We also discuss AI risk denialism.

You can download the episode here or listen below. You can also subscribe to the podcast on Apple Podcasts, Stitcher and a variety of other podcasting services (the RSS feed is here).

Show Notes

• 0:00 – Introduction
• 2:02 – Do we need to define AI?
• 4:15 – The Omohundro-Bostrom theory of AI motivation
• 7:46 – Key concepts in the Omohundro-Bostrom Theory: Final Goals vs Instrumental Goals
• 10:50 – The Orthogonality Thesis
• 14:47 – The Instrumental Convergence Thesis
• 20:16 – Resource Acquisition as an Instrumental Goal
• 22:02 – The importance of goal-content integrity
• 25:42 – Deception as an Instrumental Goal
• 29:17 – How the doomsaying argument works
• 31:46 – Critiquing the theory: the problem of self-referential final goals
• 36:20 – The problem of incoherent goals
• 42:44 – Does the truth of moral realism undermine the orthogonality thesis?
• 50:50 – Problems with the distinction between instrumental goals and final goals
• 57:52 – Why do some people deny the problem of AI risk?
• 1:04:10 – Strong versus Weak AI Scepticism
• 1:09:00 – Is it difficult to be taken seriously on this topic?

Relevant Links

• Olle’s Blog
• Olle’s webpage at Chalmers University
• ‘Challenges to the Omohundro-Bostrom framework for AI Motivations‘ by Olle (highly recommended)
• ‘The Superintelligent Will‘ by Nick Bostrom
• ‘The Basic AI Drives’ by Stephen Omohundro
• Olle Häggström: Science, Technology, and the Future of Humanity (video)
• Olle Häggström and Thore Husveldt debate AI Risk (video)
• Summary of Bostrom’s theory (by me)
• ‘Why AI doomsayers are like sceptical theists and why it matters‘ by me