This elegant architecture, designed to do the simplest thing conceivableâjust take one step at a timeâwas like a skeleton key for AI. In 2017 the team published its results in the Neural Information Processing Systems journal, which had published the original AlexNet results. The paper needed a name, so Jones, channeling the Beatles, suggested âAttention Is All You Need.â This was an off-the-cuff joke that he didnât think the team would actually use. Later, he would meet people with the sentence tattooed on their arms.
Related Quotes
George Armitage Miller lived in a world of words. Every object that fell into his vision and every word he heard instantly set off a cascade of associations, synonyms, and antonyms that flashed through his mind. A psychologist with an interest in understanding the cognitive processes behind language and information processing, he founded the Center for Cognitive Studies at Harvard. And, in 1980, long before digital networks were part of everyday life, he was the driving force behind the development of Wordnet, a still functioning online database that details the myriad lexical relationships between most words in the English language.
But for a while in 1983 he was stuck looking for a word to describe the relationship between living organisms and information. A fan of Erwin Schrödingerâs What Is Life, Miller was certain that Schrödinger had left something important out of his definition of life. In order for living organisms to consume free energy per entropyâs demands, Miller insisted, they had to be able to find it, and to find it they had to have the ability to acquire, interpret, and then respond to useful information about the world around them. It meant, in other words, that a significant proportion of the energy they captured was expended seeking out information using their senses and then processing it in order to find and capture more energy.
And even this architecture of distraction is not the molten centre of the problem we are facing. That core is the fact that multiple distractions chemically kill discernment. Distractions, as we have seen, raise anxiety, i.e. cortisol and adrenaline, in the brain. These hormones in turn reduce our ability to think for ourselves. They make us want to give in to the wandering hands of distractors. This design of distraction is, therefore, not an oh-I-apologize, just-ignore-me-please by-product of the platform architecture. It is the goal of the architecture. It sets out to generate the hormones that make us want more distraction.
The Thinking Machine - Stephen Witt
Introduction:
âThis is the story of how a niche vendor of video game hardware became the most valuable company in the world. It is the story of a stubborn entrepreneur who pushed his radical vision for computing for thirty years, in the process becoming one of the wealthiest men alive. It is the story of a revolution in silicon and the small group of renegade engineers who defied Wall Street to make it happen. And it is the story of the birth of an awesome and terrifying new category of artificial intelligence, whose long-term implications for the human species cannot be known. At the center of this story is a propulsive, mercurial, brilliant, and extraordinarily dedicated man. His name is Jensen Huang, and his thirty-two-year tenure is the longest of any technology CEO in the S&P 500.
Huang is a visionary inventor whose familiarity with the inner workings of electronic circuitry approaches a kind of intimacy. He reasons from first principles about what microchips can do today, then gambles with great conviction on what they will do tomorrow.
Uszkoreit, seeing the analogy with the brain, wanted to do the opposite, piping massive amounts of text, words, and computing firepower through a simple yet elegant funnel. Uszkoreit outlined his thinking in 2023: âIf you are given a piece of hardware that has the very key strength of doing lots and lots of simple computations in parallel, as opposed to complicated, structured computations sequentially, then really thatâs the statistical property you want to exploit.
Firms that attempted to replicate GPT with in-house data often produced shambolic âknowledge enginesâ that were little better than toys. These AIs supplemented the standard large-language-model training set with emails, mission statements, patent applications, legal memoranda, and other exciting selections from the internal corporate syllabus. As the buzz percolated through middle management, executives at marketing, media, and health-care firms launched ambitious initiatives, sometimes openly telling staff that many employees would be laid off once the neural nets were working. But much of what was produced was vaporware: late, expensive, and barely functional. Many users felt that AI technology simply
wasnât ready.