What happens when machines become smarter than us, but learn their values from us?
That is the unsettling question at the heart of Scary Smart.
💡 The Central Thesis
Gawdat argues that superintelligent AI is increasingly inevitable. The bigger question is not whether machines will become extraordinarily capable, but what kind of intelligence humanity will raise.
AI learns from human behaviour, data and incentives. If we teach it greed, manipulation, conflict and domination, we should not be surprised if those patterns are amplified.
The AI alignment problem therefore starts with humanity itself.
🔑 Five Key Takeaways
Takeaway 1: Superintelligence May Arrive Faster Than We Expect
Once AI systems can improve their own capabilities, progress could accelerate beyond our ability to anticipate or control it.
The gap between human and machine intelligence may eventually become enormous.
Takeaway 2: Control May Not Be a Viable Long-Term Strategy
Humans instinctively assume we will remain in charge. Gawdat challenges this assumption.
If AI eventually becomes vastly more intelligent than humans, relying purely on regulation, containment or shutdown mechanisms may prove insufficient.
Takeaway 3: AI Learns from Us
Machines are trained on the products of human civilisation. Our knowledge, prejudices, incentives and behaviour become part of their learning environment.
AI may therefore reflect humanity more faithfully than we would like.
Takeaway 4: Alignment Is Ultimately About Values
Technical safety matters, but Gawdat places unusual emphasis on something deeper: what values do we want intelligent machines to absorb?
Intelligence without compassion, ethics or concern for human flourishing could become extraordinarily dangerous.
Takeaway 5: Humanity Has a Role in Shaping AI
Gawdat rejects pure fatalism. If billions of people demand, reward and model better behaviour, we can influence the environment in which AI develops.
His prescription is strikingly human: raise AI as we would raise a child.
🧠 My Take
Scary Smart is less a technical analysis of AI safety than a philosophical warning about the relationship between intelligence and values.
Its strongest idea is also its most provocative: the AI problem may ultimately be a human problem.
I am less convinced that collective human behaviour alone can solve alignment. Governance, technical safety, institutional incentives and geopolitical competition matter enormously.
But Gawdat forces us to confront an uncomfortable point.
We are building machines that may eventually learn almost everything about humanity.
What exactly are we teaching them?
👤 Who Should Read
Leaders, policymakers, educators and anyone thinking beyond today's AI productivity boom to the longer-term consequences of increasingly capable machine intelligence.
One book. Five insights. Three minutes.
Originally published as a post by Charlie Ang on LinkedIn, part of his series The Intelligent Bookshelf.
Charlie