I immediately saw that there was a significant correlation between the independently collected error rates and the measures of team effectiveness from my survey. But then I looked closely and noticed something wrong. The direction of the correlation was exactly the opposite of what I had predicted. Better teams were apparently making more â not fewer â mistakes than less strong teams. Worse, the correlation was statistically significant. I briefly wondered how I could tell my dissertation chair the bad news. This was a problem.
No, it was a puzzle.
Related Quotes
And because competencies are unmeasurable, it is impossible to prove or disprove the assertion that everyone who excels in a particular job possesses a particular set of competencies. It is equally impossible to show that people who acquired the competencies they lacked outperformed those who did notâthat, in other words, well-rounded people are better. These two statements together are the foundation for most of what companies do to develop the talents of their people, yet each of them is unfalsifiableâyou will find no academic papers in any peer-reviewed journal proving the necessity of possessing certain competencies, and no proof that acquiring the ones you lack nets you any increase in performance. Both of these assertions, despite the good intentions that created them, are conjured from thin airâand we can never know if they are correct.
So far, weâve seen that 1) human beings can never be trained to reliably rate other human beings, that 2) ratings data derived in this way is contaminated because it reveals far more of the rater than it does of the person being rated, and that 3) the contamination cannot be removed by adding more contaminated data. And this means, in turn, that ratings-based tools, be they annual engagement surveys, performance-rating tools, 360-degree surveys, or any of the many other varieties at large, do not measure what they purport to measure. And this means, in turn, that discussions based on the data generated by these tools do not accurately reflect the truth of you.
My eureka moment was this: better teams probably don't make more mistakes, but they are able to discuss mistakes
Eskreis-Winkler and Fishbach conducted five studies to test the hypothesis that failure, rather than promoting learning, actually undermines it. In one study they asked participants a series of questions starting with identifying which of two symbols from a fictional ancient script represented an animal. Afterward, one group of study participants was told, âYou are correctâ (success feedback). The other group was told, âYou are incorrectâ (failure feedback). To see how well they learned from each type of feedback, participants were given a follow-up test. This time they were asked to look at the exact same symbols and asked to identify which one represented a nonliving entity. Sounds pretty straightforward, right? Yet those who had been told they were correct in the first round scored higher in their second test than those told their answers were incorrect. Over and over, people learned less from being given information about what they got wrong than about what they got right.
What Wiseman noticed that day can be seen as a vital element of TPS: a deeply ingrained belief that problem-solving is a team sport. Failures are opportunities for improvement. Competent professionals are expected to successfully execute most of their tasks, so successes are not seen as worthy of colleaguesâ valuable time. Hence the âpuzzledâ look on Mr. Choâs face. Puzzlement occurred because an expected behavior (share your problems so we can work on them together) didnât happen, while an unexpected one (bragging) did. What I love most about this story is that Wisemanâs boasting would not have raised an eyebrow in 99 percent of work environments Iâve studied. We are socialized to share accomplishments and good news in front of the boss. Nothing puzzling about it! The most impressive result of TPS in my view is that the system normalizes failureâbad news, requests for help, and problems alike. It creates a community of scientists. Not incidentally, the essence of failing well is thinking like a scientist.