Large Groups Can Be an Excellent Source of Hidden Value
Early Test Examples
Sociologist Kate Gordon asked nearly 200 students to rank various items by weight and found the group’s averages to be 94% accurate; and better than all but 5 individual guesses.
Theoretical Physicalist Norman Johnson used computer simulations of individuals navigating a maze to demonstrate the collective potential for accuracy that large groups of people or entities possess. In this experiment, Johnson modeled a maze that could be navigated to completion via many different paths through various junctions or what he called nodes. He then released the computer simulated individual agents into the modeled maze one at a time with no instruction. As you’d expect, chaos ensued with the agents collectively taking a wide range of paths before each completed the maze.
Johnson then ran the simulation a second time but slightly altered the experiment for the second run by allowing each agent to use the information they’d ‘learned’ from their first trip. The result? The collective solution of the group’s second trip through the maze was just nine steps long, which just so happened to be single shortest solution possible.
The Real World
Perhaps one of the most vivid real-world examples of how harnessing the collective intelligence of large groups can be applied relates to the assignment of responsibility for the tragic loss of the U.S. space shuttle Challenger and her seven crew on the morning of January 28, 1986.
As author James Surowiecki succinctly summarizes in his 2005 work The Wisdom of the Crowd: ‘At 11:38 AM EST on January 28, 1986, the space shuttle Challenger lifted off from its launch pad at Cape Canaveral. Seventy-four seconds later, it was ten miles high and rising. Then it blew up. The launch was televised, so news of the accident spread quickly. Eight minutes after the explosion, the first story hit the Dow Jones News Wire.’ January 28, 1986 was a Tuesday. The financial markets were open and actively trading per normal market schedule.
To truly appreciate the role that large group dynamics played in determining responsibility for the Challenger incident, it is important to first adequately characterize the scale of the complexity and costs involved with the Space Transport System (STS), more commonly referred to as the shuttle.
- The STS was comprised of four primary structures: the shuttle itself, the shuttle’s 3 main engines, the external fuel tank, and dual solid rocket boosters.
- There were four primary contractors responsible for delivering the individual structures and managing ground operations: Rockwell International (shuttle itself; main engines), Lockheed (ground operations), Martin Marietta (external fuel tank), and Morton Thiokol (solid rocket boosters).
- The STS included more than 2.5 million moving parts
- Total program cost for STS development and launches exceeded $113.7 billion.
- The youngest shuttle, Endeavor (built to replace Challenger), cost $1.7 billion to construct.
- Cost per launch at end of the STS program (2011) was estimated by NASA to be $775 million.
All four of the main contractors were publicly traded on the day of the incident. Twenty-one minutes after launch, and only thirteen minutes after the Dow Jones News Wire first reported the incident? Lockheed down 5%, Martin Marietta down 3%, Rockwell down 6%. Morton Thiokol? Trading of Morton Thiokol had to be halted almost immediately after news of the incident broke. When trading of Morton Thiokol shares resumed more than an hour later, they were down 6% and would finish the day down 12%. The stock prices for each of the other three contractors had rebounded and closed down under 3% for the day.
What this means, Surowiecki rightly concludes, is that the stock market had, within just a few minutes, been able to accurately determine that Morton Thiokol was responsible for the Challenger disaster. By contrast, it would take the Rogers Commission another 132 days to arrive at the same conclusion when it released its final report on June 9, 1986[1]
How
Stock markets are essentially machines for calculating the present value of all of the free cash flow that a company is expected to earn in the future. The steep decline in Morton Thiokol’s stock price, particularly in comparison to the only slight declines of its competitors, was a clear sign that investors believed Morton Thiokol was primarily responsible for the disaster and that the company’s future free cash flows would be significantly damaged as a result.
There were no public comments made by officials or news articles published during that day singling out Morton Thiokol as the company responsible. There was no time for media speculation, momentum trading, or other ‘hysterical’ phenomena to play a role. There were no sell orders from any Morton Thiokol executives or others who may have had insider information. There was no evidence of any large-scale dumping of Morton Thiokol stock while buying any of STS’ other three contractors. Morton Thiokol’s stock price on January 28, 1986, was being driven by large volumes of, mostly uninformed, investors who simply refused to buy the stock.
So well within 30 minutes of the shuttle exploding, the stock market had correctly determined fault and, by the end of the trading day, assigned scale to the financial consequence.
Each investor, through either their action or affirmative inaction, was answering the question ‘how much less are each of the STS contractors worth now that Challenger has exploded?’ This question had an objectively correct answer. It is a scenario where a large group’s average estimate – which is essentially what a stock price is – is likely to be accurate. Once you aggregate all of the minute and miniscule bits of information about the incident that all buyers and sellers had in their heads that day, it summed to the truth with a certainty.
Underlying Dynamic
There are four conditions that characterize large groups: (1) diversity of opinion (each member has some private knowledge; regardless of amount), (2) independence of thought, (3) decentralization (each member may specialize and leverage local knowledge), and (4) aggregation (some mechanism exists for turning private judgement into collective decision.
If a large group satisfies these four criteria, its judgement is mathematically likely to yield accurate results. By engaging a large enough group of diverse, independent entities to make a prediction or to estimate a probability, and then average these estimates, the errors each member makes in arriving at their answer will cancel out the error of another. As Surowiecki put it: ‘each person’s guess, you might say, has two components: information and error. Subtract the error, and you’re left with the information.’
[1] Finding that the o-ring seals on the booster rockets made by Morton Thiokol became less resilient in cold weather (the low air temperature on-site the night prior was 26F and 36F at time of launch), creating gaps that allowed hot exhaust gases to escape and penetrate the main external fuel tank resulting in the explosion.