13vii26
Latent Space as a New Medium Kevin Kelly
A Large Language Model (LLM) is like a small zip file that contains all human knowledge. It takes massive arrays of 100,000 GPU chips working in the cloud, and costing billions of dollars, to compress all of human writing into a small working model that could run on one single GPU chip. Even the biggest frontier models compress down to several hundred gigs, which is small enough it can fit on a card in your palm. In a strange but real way the resulting tiny file contains all the information that is on the internet and in our libraries. This tiny card holds a significant proportion of what humans collectively know. Of all the remarkable aspects of AI, this astounding feat of compression may be the least appreciated. This dense, high order compression of human knowledge — called "latent space" — may also be a new medium itself.This extreme compression of knowledge within latent spaces was not the original intention of the researchers who invented LLMs. The book smartness they contain came as sort of a surprise to the people training them, and we are still trying to figure out how they actually work. What we can say for sure is that the LLM does not contain copies of everything it knows. For instance it knows all Shakespeare plays, and it could create a new play that sounded exactly like Shakespeare, and can even quote famous lines in his plays, but nowhere in the model are the actual texts of Shakespeare. Instead there is simply the abstract information about all the plays, the plots, the characters, the words, the style, the references. Likewise, the LLM could recognize the face of almost any person, and it could generate any possible human face, but nowhere in its code are copies of human faces. Rather, the model is storing all the information about human faces, without storing any faces.
...LLMs were originally invented to do automatic language translation, that is all. But instead of teaching it the rules of language, which is what earlier AI researchers did, this time no language expertise was required. Instead, a neural net absorbed a very large database of human written language (the internet), with the goal of having the neural net (AI) extract out all the hidden patterns of language below our awareness contained within those billions of documents. The goal of the program was to replicate, imitate and synthesize the patterns of language as it is used everyday by humans.
...Latent space is an abstraction, a map built not in two dimensions, but in billions of dimensions. Imagine a brain made up of billions of straight long arrows going in all directions. Each arrow is dedicated to one idea or one thing
...Every thing, every concept has a specific location in the map of this huge space, but instead of having just two coordinates (x,y) each thing has a billion-long coordinate. So an old rusty gasoline lawnmower buried in weeds is a very specific intersection with a very long address. Each of its thousands of attributes (rust, gas, lawn, cut, weeds, push, red, dirt, clippings, roar, etc.) has its own direction intersected. Nearby in latent space is a lawn mower that is more in the rust direction, or less red, but also more catlike, or more doglike, or less spaceship-like, or more like whipped cream. That point may represent a real thing or only a virtual or theoretical thing. This mapping works for not just nouns, but any idea, any sound, any image. The whoosh of a splash of water is a direction in latent space. The aha moment in invention. The fright seeing a snake on a path. The notion of a prime number. All these are contained within a single map. This is one of the most astounding, yet underappreciated aspects of an LLM latent space: Everything — everything! — appears on just one map. We’ve never had a system to integrate everything we know and everything we can imagine. One map for all! This has long been a holy grail.
...While training, the LLM is fed millions of books, billions of web pages, and billions of pages of text from social media. It reads every word on each of them, and once this entire library of material is loaded into its mind, it massively calculates all the interconnecting vectors, all the relative directions pointing to each other. The scale of this vast synchronized parallel calculation is staggering. It then throws away the books, the text, the images, and only keeps this tangled web of directions and vectors. These billions of directions are called its parameters. As we build larger and larger models, mapping more and more material, the parameters increase. The latest models on the frontier of AI contain trillions of parameters, meaning there are trillions of directions, or trillions of attributes that it uses to map every idea or thing it has seen.
...Something as complicated as a book winds up as both a point in latent space and a journey through latent space. All the notions encountered in a story (window, mid-day stroll, street, vendor, chat, anger, fight, forgiveness) are directions, and as sentences pile up, the directions shift around, going one way and then intersecting in another. The story is really a journey through latent space, which very much mirrors the journey-like experience we have when we read.
So a book contains a sequence of vectors in latent space. But the sum meaning of a book is also just a single point or direction in itself. For instance if I reference the book The Iliad, I’m referring to the whole book, and its vector is closely related, and therefore "nearby" to the other epic war narratives like Beowulf, The Mahabharata, or even Apocalypse Now, even though many parts of them only tangentially intersect. The more related a thing or idea is, the more directions (vectors) it shares with similar things. This is in part how LLMs know stuff. They search for patterns nearby.
When you ask an LLM a question, it will find the answer in latent space. Your question itself begins as a direction, which points to the answer. The LLM addresses each word in your prompt one by one, with each new word shifting the direction of where it goes. The model travels through latent space with each word of the prompt, searching for its answer, step by step. In this way the answer is grown, rather than found.
Latent space Wikipedia
The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook Xinlei Yu et al. at arxive.org (2026)
What Is a Latent Space? towardsdatascience.com
The billion-dimensional map hidden inside every AI model Ellsworth Toohey at boingboing
other Spaces to explore:
metaspace (whatever that is...)...much more satisfying than being confined to Cartesian xyz 3-spacehyperspaces
Mindspace
L-space (of Terry Pratchett) via lspace.org and Other Resources of L-space
L-space is a mystical dimension connecting all libraries that allowed time-travel, and was only accessible by the most senior Librarians. It supposedly contains all books that have been, are, will be, or even could be...Thoughts on L-Space Christopher Lockett, The Magical Humanist
...books bend space and time...
...All libraries everywhere are connected in L-space...