(Again, to review from the previous chapter, a “luck event” meets three tests: First, you didn’t cause it; second, it has a significant potential consequence, good or bad; and third, it has an element of surprise, some aspect of the event is unpredictable before it happens.) Any framework that didn’t account for unpredictable and unforeseen events would be incomplete, and I couldn’t be intellectually satisfied until we wrestled with the question of luck. The concept of return on luck accounts for the undeniable fact that luck happens (a lot) yet captures the essential truth that luck itself cannot cause greatness. Catastrophic bad luck can kill a potentially great company, but good luck cannot make a company great. Luck doesn’t build great companies that last; people do.

I was also told that a brand-new CEO shouldn’t be trying to make huge acquisitions. I was “crazy,” as one of our investment bankers put it, because the numbers would never work out and this was an impossible “sale” to the street.

The banker had a point. It’s true that on paper the deal didn’t make obvious sense. But I felt certain that this level of ingenuity was worth more than any of us understood or could calculate at the time. It’s perhaps not the most responsible advice in a book like this to say that leaders should just go out there and trust their gut, because it might be interpreted as endorsing impulsivity over thoughtfulness, gambling rather than careful study. As with everything, the key is awareness, taking it all in and weighing every factor—your own motivations, what the people you trust are saying, what careful study and analysis tell you, and then what analysis can’t tell you. You carefully consider all of these factors, understanding that no two circumstances are alike, and then, if you’re in charge, it still ultimately comes down to instinct. Is this right or isn’t it? Nothing is a sure thing, but you need at the very least to be willing to take big risks. You can’t have big wins without them.

For these reasons changing values has particularly great significance from a longterm perspective. Looking to the past, we see that such changes have had an enormous impact on the lives of billions of people. Looking to the future, if we can improve the values that guide the behaviour of generations to come, we can be pretty confident that they will take better actions, even if they’re living in a world very different from our own, the nature of which we cannot predict.

For this reason, even if you think that the absence of future generations is not a moral loss or that the end of civilisation would be a good thing (issues that I discuss in Part IV), it’s still very important to avoid AI takeover or the lock-in of bad values. There will be future generations of intelligent beings either way, and by preventing the takeover of the world by an AI with bad values, you are changing how good or bad the future is over the course of civilisation’s life span. That’s the main effect, rather than any impacts on civilisation’s life expectancy.

Schein et al. 2020; Green and McClellan 2020.” (notes to footnote 24, Chapter 10, p.234)