I immediately saw that there was a significant correlation between the independently collected error rates and the measures of team effectiveness from my survey. But then I looked closely and noticed something wrong. The direction of the correlation was exactly the opposite of what I had predicted. Better teams were apparently making more – not fewer – mistakes than less strong teams. Worse, the correlation was statistically significant. I briefly wondered how I could tell my dissertation chair the bad news. This was a problem.

No, it was a puzzle.

And because competencies are unmeasurable, it is impossible to prove or disprove the assertion that everyone who excels in a particular job possesses a particular set of competencies. It is equally impossible to show that people who acquired the competencies they lacked outperformed those who did not—that, in other words, well-rounded people are better. These two statements together are the foundation for most of what companies do to develop the talents of their people, yet each of them is unfalsifiable—you will find no academic papers in any peer-reviewed journal proving the necessity of possessing certain competencies, and no proof that acquiring the ones you lack nets you any increase in performance. Both of these assertions, despite the good intentions that created them, are conjured from thin air—and we can never know if they are correct.

This is known as the “alignment” problem. It’s discussed at length in other excellent books, like Superintelligence, Stuart Russell’s Human Compatible, and Brian Christian’s The Alignment Problem, so I won’t go into it in depth here.

What this means is that when you have to think about connections between parts, rather than just counting the individual parts of a system, the number of possibilities grows very rapidly; the potential combinations multiply, rather than just adding up. Very quickly, they multiply up to astronomically huge numbers, spelling absolute death to any hope of knowing the entire state of the system.

In fact, knowing only a few of the feedback circuits can be actively misleading, if you rely too greatly on your partial information. It is a sobering thought, for example, that despite employing some of the best and brightest* analysts in the world, the advice given by the US State Department over the last fifty years could comfortably have been outperformed by a parrot that had been trained to repeat the phrase, ‘Don’t start a war.’ The repeated failures of the State Department are not the consequence of ignorance; they are the consequence of having very good and deep — but not total — knowledge of an extremely complicated situation, in which facts outside of that information set turned out to be crucial. Knowing a great deal of detail about a subset of a system has a habit of increasing your confidence in your opinions disproportionately from their reliability.

*The phrase ‘the best and brightest’ is often used by people who don’t know that its original context was ironic. It entered the language as the title of David Halberstam’s book about the policy mistakes of the Vietnam War.

Of course, Glenn and Cooper trained and practiced and honed their capabilities for thousands upon thousands of hours. But it is not the thousands of hours per se that made them such exquisitely designed creatures for flying. It was thousands of hours expended directly in line with a preexisting set of encodings. John Glenn would have trained and practiced just as intently for thousands of hours in chemistry, in pursuit of medical school, or in football, in hopes of an athletic career, but all that energy would have been spent out of frame. John Glenn would maybe have worked just as hard, trained just as much, but he wouldn’t have become John Glenn.