Firms that attempted to replicate GPT with in-house data often produced shambolic âknowledge enginesâ that were little better than toys. These AIs supplemented the standard large-language-model training set with emails, mission statements, patent applications, legal memoranda, and other exciting selections from the internal corporate syllabus. As the buzz percolated through middle management, executives at marketing, media, and health-care firms launched ambitious initiatives, sometimes openly telling staff that many employees would be laid off once the neural nets were working. But much of what was produced was vaporware: late, expensive, and barely functional. Many users felt that AI technology simply
wasnât ready.
Related Quotes
The Thinking Machine - Stephen Witt
Introduction:
âThis is the story of how a niche vendor of video game hardware became the most valuable company in the world. It is the story of a stubborn entrepreneur who pushed his radical vision for computing for thirty years, in the process becoming one of the wealthiest men alive. It is the story of a revolution in silicon and the small group of renegade engineers who defied Wall Street to make it happen. And it is the story of the birth of an awesome and terrifying new category of artificial intelligence, whose long-term implications for the human species cannot be known. At the center of this story is a propulsive, mercurial, brilliant, and extraordinarily dedicated man. His name is Jensen Huang, and his thirty-two-year tenure is the longest of any technology CEO in the S&P 500.
Huang is a visionary inventor whose familiarity with the inner workings of electronic circuitry approaches a kind of intimacy. He reasons from first principles about what microchips can do today, then gambles with great conviction on what they will do tomorrow.
Dahl worked on this problem for many years. The obstacle was that he had almost no idea how his poker bot actually worked. The structure of its neural net was no easier to interpret than the nervous system of an invertebrate, and trying to tease out game-playing strategies by examining the individual weights of the grid was like trying to unravel consciousness
by looking at brain cells through a microscope. This was the criticism of neural nets and the thing that so biased the academic community against them. Once a neural net hit a training plateauâand they almost always didâthere was rarely an obvious way to make it better. Classical programming was orderly and logical, but tinkering with neural nets required a different cast of mind. Dahl compared it to running a biology experiment: outcomes were unpredictable, and altering seemingly minor variables could have all manner of unanticipated results. Dahl tried everything he could think of to improve his no-limit poker bot. He fiddled with its evaluation function, he futzed with his computerâs memory, he replaced the activation trigger for the neurons, he even synthesized a simpler data universe for the bot to exploreâbut he never got it to play at an expert level.
Disruptive technologies, Christensen had observed, often grew out of hobbyist communities. They were developed using âbootlegged resourcesâ in which âoff-the-shelf componentsâ were redeployed for something other than their intended purpose. They started out wonky but rapidly improved along attributes of performance that established players ignored.
But even once you had absorbed this lesson, it wasnât easy to implement. Pursuing niche markets cost profits, making investors question your sanity. This, too, Christensen had foretold: âOne of the reasons managers at established firms find it difficult to serve emerging markets is that their investors and customers tell them not to.â
That was the real secret of The Innovatorâs Dilemma, which readers often missed. It was not a book about how to succeed; it was a book about how not to fail. Christensenâs book wasnât a how-to for start-ups but a counterinsurgency manual for senior managers at stagnating firms. Thirteen years in, Huang felt that Nvidia was at risk of becoming such a firm, and it was as much paranoia as optimism that led him to pursue the mad-science market.
Uszkoreit, seeing the analogy with the brain, wanted to do the opposite, piping massive amounts of text, words, and computing firepower through a simple yet elegant funnel. Uszkoreit outlined his thinking in 2023: âIf you are given a piece of hardware that has the very key strength of doing lots and lots of simple computations in parallel, as opposed to complicated, structured computations sequentially, then really thatâs the statistical property you want to exploit.
Still, if Bengio, Hinton, and Sutskever had been sidelined by capital, the points they made remained valid. They had seen better than anyone the potential of what AI technology could be, and they had the academic credentials to prove it. If they were worried now, I felt it was worth listening to. âRight now there are ninety-nine very smart people trying to make AI better and one very smart person trying to figure out how to stop it taking over,â Hinton said.