Your company is a maximization machine—it wants to make the best use of its finite resources—so it is greatly interested in identifying precisely who to invest in, and how.

The problem with this stems from the way your company executes on these good intentions. Why, for example, does it assume that it will net a good return only from certain people? Surely, the cliché that “Our people are our greatest asset” applies to all of the people in the company. As we’ve seen, every human brain retains its ability to learn and grow throughout adulthood. For sure, each brain grows at a different speed and in a different way, but this implies only that each person learns differently, not that—categorically—some people do and some don’t. Therefore, the best course of action for any maximization machine worth its salt would be to figure out where and how each brain can grow the most, rather than zeroing in on only a select few brains and casting aside the others.

Dahl worked on this problem for many years. The obstacle was that he had almost no idea how his poker bot actually worked. The structure of its neural net was no easier to interpret than the nervous system of an invertebrate, and trying to tease out game-playing strategies by examining the individual weights of the grid was like trying to unravel consciousness

by looking at brain cells through a microscope. This was the criticism of neural nets and the thing that so biased the academic community against them. Once a neural net hit a training plateau—and they almost always did—there was rarely an obvious way to make it better. Classical programming was orderly and logical, but tinkering with neural nets required a different cast of mind. Dahl compared it to running a biology experiment: outcomes were unpredictable, and altering seemingly minor variables could have all manner of unanticipated results. Dahl tried everything he could think of to improve his no-limit poker bot. He fiddled with its evaluation function, he futzed with his computer’s memory, he replaced the activation trigger for the neurons, he even synthesized a simpler data universe for the bot to explore—but he never got it to play at an expert level.

Disruptive technologies, Christensen had observed, often grew out of hobbyist communities. They were developed using “bootlegged resources” in which “off-the-shelf components” were redeployed for something other than their intended purpose. They started out wonky but rapidly improved along attributes of performance that established players ignored.

But even once you had absorbed this lesson, it wasn’t easy to implement. Pursuing niche markets cost profits, making investors question your sanity. This, too, Christensen had foretold: “One of the reasons managers at established firms find it difficult to serve emerging markets is that their investors and customers tell them not to.”

That was the real secret of The Innovator’s Dilemma, which readers often missed. It was not a book about how to succeed; it was a book about how not to fail. Christensen’s book wasn’t a how-to for start-ups but a counterinsurgency manual for senior managers at stagnating firms. Thirteen years in, Huang felt that Nvidia was at risk of becoming such a firm, and it was as much paranoia as optimism that led him to pursue the mad-science market.

Firms that attempted to replicate GPT with in-house data often produced shambolic “knowledge engines” that were little better than toys. These AIs supplemented the standard large-language-model training set with emails, mission statements, patent applications, legal memoranda, and other exciting selections from the internal corporate syllabus. As the buzz percolated through middle management, executives at marketing, media, and health-care firms launched ambitious initiatives, sometimes openly telling staff that many employees would be laid off once the neural nets were working. But much of what was produced was vaporware: late, expensive, and barely functional. Many users felt that AI technology simply

wasn’t ready.

One of its signals has been so amplified that it drowns out the others. The ‘profit motive’ isn’t something that can be ascribed to corporations — they don’t have motives. What they have is an imbalance between the two key higher-functions — here-and-now versus there-and-then. They aren’t capable of responding to signals from the long-term planning and intelligence function, because the short-term planning function has to operate under the constraints of the financial market disciplinary system. Either a corporation has a survival condition based on needing to make a monthly interest bill, or there’s an implicit threat from the financial environment that if it fails to behave in a particular way, it will be taken over by an outside entity.

If you take away that pressure, it’s quite likely that the natural equilibrium of corporate decision-making systems will be less hostile to human life. Viable systems fundamentally seek stability, not maximisation.