Let’s get one thing straight before we start. I don’t like Ezra Klein. I love Ezra Klein. Not because people say the neoliberal heartthrob is as charming as he is handsome (although all those things are true!) and not because I often agree with positions of the center-left technocrat (I actively try to read people I disagree with, in order to avoid bubble-think).
No, my affinity for Ezra begins over a decade ago, when he was an obscure blogger and I was one of his very few readers. At the time, Ezra was one of a tiny number of writers doing serious policy analysis – evaluating legislation as empirically as a management consultant would evaluate a client’s budget. Considering the media environment at the time, this was pretty radical. News outlets would either explore the policy through fact-free hyper-normative opinion pieces, or do the sort of “straight-reporting” that would refuse to say 2+2 = 4 in order to appear unbiased. It’s hard to overstate how vacuous all of this could get – and how different what Ezra was doing was. While respectable outlets were all ‘Supporters say Medicare Part D will be very expensive. Opponents say it will be cheap. ¯\_(ツ)_/¯’, Klein would be all ‘the best analysis from the CBO and more independent economists indicate that Medicare part D will cost the federal government around $40bn this year and proceed with a CAGR between 3-5 percent for the next ten years.” It’s for this reason nerds like me fell in love with him. Indeed, my admiration was so deep that when I met him at a DC halloween party in 2006, I fan-girled so hard at the man dressed like the book “What’s the Matter with Kansas?” he asked me if I was OK with genuine concern.
Since then, Ezra has risen from obscurity to prominence. At his recent talk at UPenn, I expressed shock to a fellow MBA that, 15 minutes before the event began, all the seats had already been filled and it was standing-room only. She was less surprised, and noted that he is now “such a cultural figure.” This likely due to his role in founding Vox.com, an amazing news site so replete with charts and graphs, wonks all over the world now swoon for its founder.
It’s this high-regard in which I and others hold him that leaves something nagging at me. My man-crush, who has lived his life with a commitment to evidence and whose intellect is obvious to all he encounters, has been a less-than-stellar forecaster of elections. I actually do not think predicting political races is a particularly valuable media product; I find the policy analysis and context provided to news stories of Vox “explainers” to be much more valuable. Still, the fact that an obviously talented individual, who has spent his entire career working in politics, would make such a misguided forecast as to predict that Marco Rubio would win the GOP primary bugs me. Not because of what it tells me about Ezra, but what it reveals about the human mind, institutions, and culture.
This is where the book Superforecasting comes into play. Written by Penn’s own genius prognosticator Phil Tetlock, the work is chock full of lessons on prediction. One recurring theme is the failure of expertise. The most striking example is ‘The Good Judgement Project’. Tetlock gathered predictions from more than 20,000 educated and curious citizens with no background in foreign affairs, and instructed them to make predictions about a variety of current events. Placing their forecasts into an algorithm, he came up with a set of GJP predictions and entered them into a government forecasting tournament. GJP won – even though their competitors were intelligence professionals who were not only career experts but people with access to classified information the GJP was not privy to.
If novices can seemingly out-forecast experts, what does that say about expertise? If this occurs in the most serious circumstances like intelligence work, what does that say about a number of human endeavours where forecasting is vital? The stability of financial markets is an obvious example, and we’ve seen in recent history a novice ex-medical professional outperform the entire net expertise of big banks in his prediction of the 2008 housing crash (UnFun Fact: Michael Burry is now predicting a giant global water shortage).
Of Tetlock’s thousands of novices, 2% stuck out as highly and consistently accurate. He deems these people “superforecasters”, and attempts to glean lessons from them. What he learns then recommends seems intuitive – break down an issue into discrete parts, think probabilistically, minimize internal biases and utilize data and logic, keep a record or your predictions and review for lessons learned, etc.
I think all of Tetlock’s recommended behaviors are good practices, but I don’t think they would really be new to professionals. Did the CIA experts who said there was a “slam-dunk” case for Iraq having WMD really not understand the importance of minimizing internal biases? Did the bankers who invested in subprime loans really not understand the importance of breaking an issue down into discrete parts? Did Nate Silver, the statistician who literally became famous for mocking political talking-heads for ignoring polling data prior to the 2008-election, really not understand the importance of utilizing data when he predicted a Trump collapse?
So here’s my theory as to why novices often get it right and experts often fail. Exerts by definition have expertise, which helps them get picked up by establishments. Establishments often have perverse incentives, for a variety of reasons, to offer untrue projections. Moreover, these tendencies are exacerbated by groupthink, which even the most aware expert is susceptible to because they are part of establishments. And establishments inherently include socialization, which causes the human mind to defer to a consensus that they might not agree with were they not surrounded by it.
I asked Ezra at his recent talk whether he thought the reason his forecasts were inaccurate and mine (I’ve predicted the rise of both Obama and Trump when both were laughed at by pundits and in single-digits on the prediction markets) have been amazing might be because I’m on the outside (I don’t work in politics, policy, or media) and he is part of the establishment, thereby sucked-in by groupthink. His answer?
“I’m going to turn over all of my investments to you.”