For this reason, even if you think that the absence of future generations is not a moral loss or that the end of civilisation would be a good thing (issues that I discuss in Part IV), itās still very important to avoid AI takeover or the lock-in of bad values. There will be future generations of intelligent beings either way, and by preventing the takeover of the world by an AI with bad values, you are changing how good or bad the future is over the course of civilisationās life span. Thatās the main effect, rather than any impacts on civilisationās life expectancy.
Related Quotes
I now believe the worldās long-run fate depends in part on the choices we make in our lifetimes. The future could be wonderful: we could create a flourishing and long-lasting society, where everyoneās lives are better than the very best lives today. Or the future could be terrible, falling to authoritarians who use surveillance and AI to lock in their ideology for all time, or even to AI systems that seek to gain power rather than promote a thriving society. Or there could be no future at all: we could kill ourselves off with biological weapons or wage an all-out nuclear war that causes civilisation to collapse and never recover.
The fact that our time is so unusual gives us an outsized opportunity to make a difference. Few people who ever live will have as much power to positively influence the future as we do. Such rapid technological, social, and environmental change means that we have more opportunity to affect when and how the most important of these changes occur, including by managing technologies that could lock in bad values or imperil our survival. Civilisationās current unification means that small groups have the power to influence the whole of it. New ideas are not confined to a single continent, and they can spread around the world in minutes rather than centuries.
Values changes are significant because they have major impacts on the lives of people and other beings. But from a longtermist perspective, they are particularly significant compared to other sorts of changes we might make because their effects are unusually predictable.
If you promote a particular means of achieving your goals, like a particular policy, you run the risk that the policy might not be very good at achieving your goal in the future, especially if the world in the future is very different from today, with a very different political, cultural, and technological environment. You might also lose out on the knowledge that we will gain in the future, which might change whether we even think that this policy is a good idea. In contrast, if you can ensure that people in the future adopt a particular goal, then you can trust them to pursue whatever strategies make the most sense, in whatever environment they are in and with whatever additional information they have. You can therefore be fairly confident that you have made the achievement of that goal more likely, even if you have no idea at all what the world will be like when those future people act.
For these reasons changing values has particularly great significance from a longterm perspective. Looking to the past, we see that such changes have had an enormous impact on the lives of billions of people. Looking to the future, if we can improve the values that guide the behaviour of generations to come, we can be pretty confident that they will take better actions, even if theyāre living in a world very different from our own, the nature of which we cannot predict.
Positive moral change is not inevitable. Itās the result of long, hard work by generations of thinkers and activists. And if thereās any change thatās not inevitable, itās concern for future peopleāpeople who, by virtue of their location in time, are utterly disenfranchised in the world today.
If we are careful and far-sighted, we have the power to help build a better future for our great-grandchildren, and their great-grandchildrenā down through hundreds of generations. But we cannot take such a future for granted. Thereās no inevitable arc of progress. No deus ex machina will prevent civilisation from stumbling into dystopia or oblivion. Itās on us. And we are not destined to succeed.