The Thinking Machine - Stephen Witt

Introduction:

“This is the story of how a niche vendor of video game hardware became the most valuable company in the world. It is the story of a stubborn entrepreneur who pushed his radical vision for computing for thirty years, in the process becoming one of the wealthiest men alive. It is the story of a revolution in silicon and the small group of renegade engineers who defied Wall Street to make it happen. And it is the story of the birth of an awesome and terrifying new category of artificial intelligence, whose long-term implications for the human species cannot be known. At the center of this story is a propulsive, mercurial, brilliant, and extraordinarily dedicated man. His name is Jensen Huang, and his thirty-two-year tenure is the longest of any technology CEO in the S&P 500.

Huang is a visionary inventor whose familiarity with the inner workings of electronic circuitry approaches a kind of intimacy. He reasons from first principles about what microchips can do today, then gambles with great conviction on what they will do tomorrow.

EIGHT: The Compulsion Loop

“The accounting scandal occurred during one of the worst bear markets in history. Suffering under the simultaneous bursting of the dot-com bubble, the 9/11 attacks, and the Enron bankruptcy, the S&P 500 lost nearly half its value. Coincident with these misfortunes, Nvidia started squabbling with Microsoft. The dispute was attributed to pricing and intellectual-property issues, but Nvidia’s growing sense of entitlement played a role. Nvidia employees were unabashedly elitist. They considered themselves the best—and they were—but their pride could sometimes sound like narcissism.

NINE: Cuda

“To distinguish himself, Huang had to pursue a strategy that so defied conventional business logic that ATI wouldn’t follow. He had to build an exploratory product, like a $300 entry-level scientific supercomputer that not only didn’t have competitors but also didn’t even have obvious customers. The zero-billion-dollar market, by definition, was one that only he would participate in—one that only he would even see. Huang was going to build a baseball diamond in a cornfield and wait for the players to arrive.

“Catanzaro was convinced the solution was to redesign the microchip anew. He cofounded the UC-Berkeley Parallel Computing Lab in the mid-2000s, along with several colleagues. There, Catanzaro made a list of existing parallel applications. The business problem, he could see, was that even for the supposedly hungriest customers, the demand for computing power was capped: once you sold an oil prospector a supercomputer, you saturated demand for years. What you needed, Catanzaro figured, was an application that was so hungry for computation that it could never be satisfied. You needed another application like 3D graphics that demanded more computer power once its initial needs were fulfilled. Eventually, Catanzaro deduced what had to be parallel computing’s killer app. “The answer to that was AI,” Catanzaro said. “I came to AI from the bottom up. I came from a circuits perspective. I felt it was just inevitable that AI was the most important computational workload.

NINETEEN: Power

“These GPUs used a lot of juice. A standard Google search required about a third of a watt-hour’s worth of electricity. With generative AI enabled, the same Google search required ten times that, which was enough to power a light bulb for about twenty minutes. Ask GPT to write you a five-thousand-word term paper, and you used enough energy to run a microwave for an hour. Industrial demand was greater; executives were excited about the prospect of replacing human labor entirely.