Much the same will be true for the issues that I cover in this book. I’m not saying that we should be confident that value lock-in or major catastrophe will occur this century. What I am saying is that their chance of occurring is very real—certainly more than 1 percent, and certainly greater than many everyday risks, like dying in a car crash. When combined with how much is at stake, the expected value of trying to ensure a good future is enormous. When we’re applying the significance, persistence, and contingency framework, we should therefore be thinking about expected significance, expected persistence, and expected contingency.

In this chapter I discuss value lock-in: an event that causes a single value system, or set of value systems, to persist for an extremely long time. Value lock-in would end or severely curtail the moral diversity and upheaval that we are used to. If value lock-in occurred globally, then how well or poorly the future goes would be determined in significant part by the nature of those locked-in values. Some changes in values might still occur, but the broad moral contours of society would have been set, and the world would enact one of only a small number of futures compared to all those that were possible.

When we look at history, we see that the predominant culture in a society tends to entrench itself, eliminate the competition, and take steps to replicate itself over time. Indeed, many moral views regard their own lock-in as desirable.

But if we are guilty of gross moral errors ourselves, then locking in our present values would also be a disaster.

Instead, we should try to ensure that we have made as much moral progress as possible before any point of lock-in. Political philosophers often argue over what an ideal state would look like. I think we should accept that we don’t know what the ideal state would be; the primary question is how we can build a society such that, over time, our moral views improve, people act more often in accordance with them, and the world evolves to become a better, more just place.

This last point—that we need to structure global society so that cultural evolution guides the world towards better values and better societal structures—highlights an issue facing the design of a morally exploratory world that I’ll call the lock-in paradox. We need to lock in some institutions and ideas in order to prevent a more thoroughgoing lock-in of values. One challenge is that these institutions and ideas will be morally controversial; for example, from many fundamentalist religious perspectives, the idea that we would encourage or even allow a diversity of worldviews might be regarded as abominable. Similarly, the idea that the path to the correct moral view is via reflection and good-faith debate, rather than studying the scripture of a holy book, is not one that everyone would accept.

The lock-in paradox thus resembles the familiar paradox of tolerance— the necessity for liberal societies to defend themselves against intolerant views that would undermine their freedom, even if doing so requires curtailing the very tolerance they want to preserve.