Laird Stewart | About | Blog | Newsletter | Projects | Publications | RSS |
A response to Cal Neport's recent Opinion piece in the NYT. I'm a big fan of Cal's; Deep Work, How to Be a Straight-A Student, and So Good They Can't Ignore You are some of my favorite books and among the most influential on my thinking. However, I've been discouraged by some of his commentary on AI.
Cal worries Silicon Valley execs' public fears about existential AI risk (regardless of their intention) has served to downplay risks/concerns about their current systems. This is probably true. This is bad if the risks/concerns around current system are more worthy of concern than the risks/concerns around more powerful systems.
Cal is also concerned that these EA/Rationalist ideas (i.e., fears about existential AI risk) are easily adopted by Silicon Valley elites because it strokes their ego: EA argues that working on AI risk is good and important. Therefore they are good and important. This protects their ego from outside criticism that big tech and billionaires are evil etc. This is also probably true. While this isn't an argument against the ideas, it should give us reason to think more critically about them.
So should we be more concerned about existential AI risk than risks from today's systems? How does Cal engage with this question? Unfortunately, judges ideas based on their origin (genetic fallacy) and reverts to ad hominem attacks and straw manning. Charitably, an opinion column is too short to dig into the arguments themselves, but then again, he titled it "How Scared Should We Be of A.I. Right Now?"
His overall argument is roughly:
He never engages with the risks being discussed. Calling its proponents weird is ad hominem. He also does a lot of straw-manning, two examples:
"Be very wary, for example, when A.I. leaders or online commentators begin talking in sweeping generalities about the future of the technology ... that their language models will soon become sentient beings intent on escaping their creators’ control"
Safety arguments don't hinge on sentience or malicious intent, they are concerned with capable systems pursuing misaligned goals.
"In these circles, it’s taken for granted that A.I. capabilities will rapidly accelerate and completely transform the world, and to talk about it in any other way would be considered uninformed"
I'd point to Toby Ord's (cofounder of EA) book, The Precipice. He is characteristically uncertain about AI timelines and final capabilities. But to be concerned about existential AI risk does not require certainty, only some non-zero degree of belief that it is possible.