This Is a Wrong Review.
That is, it's my review of Wrong by David H Freedman.
What it's about
The subtitle already gives you its main theme: "Why experts* keep failing us --- and how to know when not to trust them *Scientists, finance wizards, doctors, relationship gurus, celebrity CEOs, ... consultants, health officials and more"
Freedman tours the various ways in which expert advice fails. Chapter by chapter:
"Some Expert Observations"
This chapter is to convince his audience that experts are indeed often wrong. He realizes most people need little or no convincing on this point.
"The Trouble With Scientists, part 1"
How scientists can get it wrong at the level of individual studies.
"The Certainty Principle"
Some psychological factors tend us to accept certain advice too easily. For instance, we tend to trust experts who seem certain of what they're telling us.
"The Trouble With Scientists, part 2"
How scientific publication gets it wrong. The file-drawer problem, etc
"Experts and Organizations"
Business advice isn't that great either. For one thing, it tends to advise taking too many risks, because that's what big business successes did - but so did big business failures.
"Experts and the media"
The media often takes studies at face value and embellishes them to make them look relevant.
"The Internet And The Technology Of Expertise"
Will the Internet save us from mis-expertise? Across the internet, many schemes have been tried to winnow the good advice from the bad. None of them have fully accomplished this miracle yet.
Plos One
This almost belongs in "The Trouble With Scientists, part 2". There's an electronic journal called PLoS ONE that will publish any paper that isn't technically flawed, apparently to combat the file-drawer problem. Interesting, but probably won't catch on just by itself.
Side comment on prediction markets
Of particular interest to me was the mention of Anthony Kwasnica's research into prediction markets.
If you are at all interested in idea futures/prediction markets, you have probably that the market predicted Florida weather better than the meteorologists. Does that hold up in general? Looks like no; I emphasize that this is Kwasnica's current, ongoing research and not a final conclusion.
More specifically, his prediction market experiments are not outperforming conventional meteorological predictions, and their success seems all due to the participation of meteorology students. Ie, about what common sense would suggest.
"Eleven Simple Never-fail Rules for Not Being Misled By Experts"
That's the title of Chapter 9. Piques your skepticism, doesn't it?
That's because the title itself is "simplistic, universal, and definitive" - and many other things that chapter 3 says should raise your skepticism. That's not me ragging on Freedman; he's aware of that and says as much. Here he seems to be at least partly lampshading his own message.
What he actually gives you is more credible:
- Five typical characteristics of less trustworthy expert advice
- Seven characteristics of expert advice we should ignore
- Seven characteristics of more trustworthy expert advice
Appendix 4
Thruout the book, Freedman tells you that in appendix 4, he'll address the question of whether his book itself might be one of those pieces of bad advice.
It's worth waiting for. But resist the temptation to skip ahead to it; that will partly spoil it. I won't spoil it either, but I'll say this much: It's delightful reading. Freedman unflinchingly explores the ways in which he and his own book maybe shouldn't be trusted.
