“How could I have been so mistaken as to have trusted the experts?”
-John F. Kennedy after Bay of Pigs
Turning to experts for solutions to problems can be appeasing for a number of reasons the most common of which is the comfort in outsourcing the decision making process. In my own experience I’ve found organizations love to delegate decisions, especially technical decisions, to an ordained group of “Experts”. These shaman will fly in on Monday afternoons and confer in conference rooms until Thursday at which point they will collectively provide their OPINION on the matter at hand. This OPINION will taken as fact via a combination of confirmation bias and office politics. Decisions will be made based on the new found “facts” and opportunities will be lost.
This post; however, is not about the use of Guru’s in the corporate world, but how we turn to these anointed professionals for help making decisions in our everyday life. In this post we will explore the effectiveness of investment professionals, political pundits, doctors and realtors with the goal of providing you with a little extra motivation to do your own research.
Turn on Fox Business, MSNBC, or pop open the Wall Street journal and you can witness the worship of investment “greats” providing you a myriad of explanations of where the market is going or what the hot stock is. Though Wall Street analysts make average salaries measured in the million$, are these analysts anymore effective than dart throwing chimps? The answer is no!
Need further evidence?! A 1995 study of the highest paid Wall Street analysts invited by Barron’s to its annual roundtable to make recommendations revealed investments made based on the projections of these oracles merely matched the average market return (Source: Mlodinow). Furthermore, studies from 1987 and 1997 found that the recommendations from the television show Wall $treet Week significantly lagged the overall market performance, while a Harvard study of 153 investment newsletters indicated “no evidence of stock pickling ability” (Source: Mlodinow).
You may now be asking yourself how these stock market gurus who apparently have at best the insight of a dart throwing chimp be touted as “beating the market” and be paid outrageous amounts of money? The answer is the illusion of patterns. Humans have evolved to be excellent at pattern recognition, allowing us to perform feats such as circumnavigating the globe with the stars. Excellent abilities to recognize patterns is a double edged sword, as we can also identify patterns where none exist. The example below provides a distribution of the top 300 fund managers and the number of consecutive years they beat the market (S&P 500) and the success of students correctly guessing a series of coin flips. The two distributions are almost identical providing insight that the ability of fund managers to beat the market equivalent to the probability of correctly calling “heads” or “tails”. As the population of coin flipping students or fund managers increases the probability of a small number of “extraordinary performers” increases to the point of certainty.
Top 300 Fund Managers vs. 300 Coin Flipping Students (Source: Mlodinow)
After recognizing the failure of political scientists to accurately predict the fall of the Soviet Union, Phil Tetlock (then a professor at UC Berkeley) undertook a 15 year study to evaluate the accuracy of political predictions. Tetlock’s findings were published in his book, Expert Political Judgement. Tetlock’s findings concluded that experts were barely better than random chance (think dart throwing chimps) at predicting events. In fact, events Tetlock’s “experts” predicted had a zero percent chance of occurring, actually occurred about 15% of the time, while absolute certain events (per the “experts”) did not occur about 25% of the time. Tetlock has noted that the more interviews an expert participated in the worse his or her prediction accuracy (Source: N. Silver)!
In his own review of political pundits, Nate Silver evaluated predictions made by panelist on the television program The McLaughlin Group. Silver analyzed almost 1000 predictions and found that the pundits were about as accurate as a coin flip. Furthermore, Silver noted that “experts” on The McLaughlin Group predictions were influenced by their political affiliations (Source: N. Silver).
Persuasion Tip #1: When you identify as part of a group, your opinions tend to be biased toward the group consensus.
Win Bigly by Scott Adams
As it may be easy to “write off” investment analysts and political pundits as modern day snake oil salesmen, lets take a brief look at Medical Doctors. Studies have shown that radiologists fail to identify lung disease in about 30 percent of the X-ray results they read, despite the clear presence of the disease on the film (Source: Malkiel). Other experiments have shown that professional psychiatrists were unable to distinguish between the sane & insane (Source: Malkiel).
A classic example is a study by the American Child Health Association performed in the 1920’s were 1,000 children from the New York City public schools were examined by physicians to determine the need for a tonsillectomy (Source: Malkiel). Of the original 1,000 students, 611 (61.1%) were identified as needing their tonsils removed. The remaining students were then evaluated by another group of physicians who then selected 174 (44.7%) as requiring a tonsillectomy. The remaining group of 215 students were evaluated by a third set of physicians who concluded that 99 (46%) of those students were in need of the operation. The final 116 children were examined a final time and the next group of physicians recommended that 51 (43.9%) students needed a tonsillectomy. The results of this study indicated that parents taking their children to a New York City physician in the 1920’s for tonsil issues were effectively paying doctors to flip a coin!
Realtors, or as I refer to them as the most morally bankrupt reptiles on the plane (I’ve recently had some bad experience with realtors), are another example of anointed professionals who often fail to deliver the insight and results their clients expect. Realtors, as the video from the Freakonomics Movie, demonstrates how real estate agents are incentivized to work against their clients to quickly ascertain a sale.
If you have found yourself complaining about the time on the market of your home or lack of open houses and your realtors only advise is “lower the price” the video below is essential. The previous examples of investment professionals and medical doctors hinted at the influence of random luck, the example of real estate professionals provides insight into how incentive systems lead anointed professionals to provide you with misinformation. Leverage websites such as Realtor.com and data from your local realtor association to generate your own insight.
Wrapping it Up:
Gurus and professionals are not infallible as the examples provided indicate. In some cases as we explored with investment professionals and tonsillectomy diagnosis, professionals may about as effective as dart throwing chimps or coin flipping. Professionals will be influenced by the consensus of the groups they are in, as was the case with political pundits. Lastly, they may be incentivized to act against your best interests (think real estate agents). Thus, it is essential to do your own homework! Filter the signal from the noise in the data you collect and mindful of the bias of Guru’s as a result of incentives and affiliations.
“All professions are conspiracies against the laity.”
-George Bernard Shaw, Major Barbara