This suggests that, as longtermists, when trying to improve society’s values, we should focus on promoting more abstract or general moral principles or, when promoting particular moral actions, tie them into a more general worldview. This helps ensure that these moral changes stay relevant and robustly positive into the future.

In this chapter I discuss value lock-in: an event that causes a single value system, or set of value systems, to persist for an extremely long time. Value lock-in would end or severely curtail the moral diversity and upheaval that we are used to. If value lock-in occurred globally, then how well or poorly the future goes would be determined in significant part by the nature of those locked-in values. Some changes in values might still occur, but the broad moral contours of society would have been set, and the world would enact one of only a small number of futures compared to all those that were possible.

Would we wield such unprecedented power responsibly? Worryingly, the pursuit of value lock-in has been common throughout history. We saw that when the Qin took control of China, they undertook a programme to systematically eradicate competing schools of thought; similarly, the Han systematized Confucian teachings to the detriment of competing schools. The Mohists, too, desired to lock in their own values indefinitely, if only they had the power. They saw moral disagreement as the biggest problem in the world and thought that the solution was to ensure that everyone had the same values.

But if we are guilty of gross moral errors ourselves, then locking in our present values would also be a disaster.

Instead, we should try to ensure that we have made as much moral progress as possible before any point of lock-in. Political philosophers often argue over what an ideal state would look like. I think we should accept that we don’t know what the ideal state would be; the primary question is how we can build a society such that, over time, our moral views improve, people act more often in accordance with them, and the world evolves to become a better, more just place.

This last point—that we need to structure global society so that cultural evolution guides the world towards better values and better societal structures—highlights an issue facing the design of a morally exploratory world that I’ll call the lock-in paradox. We need to lock in some institutions and ideas in order to prevent a more thoroughgoing lock-in of values. One challenge is that these institutions and ideas will be morally controversial; for example, from many fundamentalist religious perspectives, the idea that we would encourage or even allow a diversity of worldviews might be regarded as abominable. Similarly, the idea that the path to the correct moral view is via reflection and good-faith debate, rather than studying the scripture of a holy book, is not one that everyone would accept.

The lock-in paradox thus resembles the familiar paradox of tolerance— the necessity for liberal societies to defend themselves against intolerant views that would undermine their freedom, even if doing so requires curtailing the very tolerance they want to preserve.