What happens when machines become smarter than us, but learn their values from us?

That is the unsettling question at the heart of Scary Smart.

The Intelligent Bookshelf #06: Scary Smart, the future of artificial intelligence and how you can save our world. A review by Charlie Ang of Scary Smart by Mo Gawdat, 2021, with the central thesis and five key takeaways: superintelligence may arrive faster than we expect; control may not be a viable long-term strategy; AI learns from us; alignment is ultimately about values; humanity has a role in shaping AI.
The Intelligent Bookshelf #06: Scary Smart by Mo Gawdat.Click to enlargeTap to enlarge

💡 The Central Thesis

Gawdat argues that superintelligent AI is increasingly inevitable. The bigger question is not whether machines will become extraordinarily capable, but what kind of intelligence humanity will raise.

AI learns from human behaviour, data and incentives. If we teach it greed, manipulation, conflict and domination, we should not be surprised if those patterns are amplified.

The AI alignment problem therefore starts with humanity itself.

🔑 Five Key Takeaways

Takeaway 1: Superintelligence May Arrive Faster Than We Expect

Once AI systems can improve their own capabilities, progress could accelerate beyond our ability to anticipate or control it.

The gap between human and machine intelligence may eventually become enormous.

Takeaway 2: Control May Not Be a Viable Long-Term Strategy

Humans instinctively assume we will remain in charge. Gawdat challenges this assumption.

If AI eventually becomes vastly more intelligent than humans, relying purely on regulation, containment or shutdown mechanisms may prove insufficient.

Takeaway 3: AI Learns from Us

Machines are trained on the products of human civilisation. Our knowledge, prejudices, incentives and behaviour become part of their learning environment.

AI may therefore reflect humanity more faithfully than we would like.

Takeaway 4: Alignment Is Ultimately About Values

Technical safety matters, but Gawdat places unusual emphasis on something deeper: what values do we want intelligent machines to absorb?

Intelligence without compassion, ethics or concern for human flourishing could become extraordinarily dangerous.

Takeaway 5: Humanity Has a Role in Shaping AI

Gawdat rejects pure fatalism. If billions of people demand, reward and model better behaviour, we can influence the environment in which AI develops.

His prescription is strikingly human: raise AI as we would raise a child.

🧠 My Take

Scary Smart is less a technical analysis of AI safety than a philosophical warning about the relationship between intelligence and values.

Its strongest idea is also its most provocative: the AI problem may ultimately be a human problem.

I am less convinced that collective human behaviour alone can solve alignment. Governance, technical safety, institutional incentives and geopolitical competition matter enormously.

But Gawdat forces us to confront an uncomfortable point.

We are building machines that may eventually learn almost everything about humanity.

What exactly are we teaching them?

👤 Who Should Read

Leaders, policymakers, educators and anyone thinking beyond today's AI productivity boom to the longer-term consequences of increasingly capable machine intelligence.

One book. Five insights. Three minutes.

Originally published as a post by Charlie Ang on LinkedIn, part of his series The Intelligent Bookshelf.

About the author
Charlie Ang

Charlie Ang is an Intelligent Reinvention Strategist, corporate futurist and AI keynote speaker based in Singapore and speaking worldwide. He helps organisations and leaders reinvent, rewire and retool for the Age of Intelligence, and has been featured by the BBC, Financial Times, National Geographic and CNA.