As I work on Natural Information Architecture, I’m sharing notes and quotes from my sources of inspiration and provocation. As always, your questions and suggestions are welcome.
On Intelligence (2005) by Jeff Hawkins was a source of inspiration for my book on the design of paths and goals, Planning for Everything (2018).
All our knowledge of the world is a model based on patterns. (63)
Our brains use stored memories to constantly make predictions about everything we see, feel, and hear. (86)
What we perceive is a mix of what we sense and memory-derived predictions. (87)
Prediction is not just one of the things your brain does. It is the primary function of the neocortex, and the foundation of intelligence. (89)
In A Thousand Brains (2022), Jeff Hawkins builds on his memory-prediction framework to advance a new theory of distributed intelligence that’s grounded in neuroscience.
A map of a town is a model of the town, and the grid lines, such as lines of latitude and longitude, are a type of reference frame. (4)
The brain’s model of the world is built using maplike reference frames. Not one reference frame, but hundreds of thousands of them. (4)
[Vernon Mountcastle] said that the fundamental unit of the neocortex, the unit of intelligence, was a “cortical column.” (24)
There are roughly 150,000 cortical columns stacked side by side in a human neocortex. (25)
The brain relies on a general-purpose method of learning. (26)
The idea that diverse abilities such as vision, touch, language, and philosophy are fundamentally the same is hard for many people to accept. (70)
We can think of reference frames as a way to organize any kind of knowledge. (70)
The neocortex uses the same basic method to model your body as it does to model objects such as coffee cups. (76)
Say I want to create a reference frame in which I can organize knowledge about all the people I know. One dimension I might use is age. I can arrange my acquaintances along this dimension by how old they are. Another metric might be where they live relative to me. This would require two more dimensions. Another dimension could be how often I see them, or how tall they are. I am up to five dimensions. This is just an analogy. (77)
When a column learns a model of something, part of the learning is discovering what is a good reference frame, including the number of dimensions. (77)
When the subjects thought about birds, they were mentally “moving” through the map of birds in the same way you can mentally move through the map of your house. (79)
Thinking is actually moving through a space, through a reference frame. (80)
What we think next depends on which direction we mentally move through a reference frame. (80)
Discovering a useful reference frame is the most difficult part of learning. (81)
By turning the words “Gettysburg Address” into a link to the full speech, I can include all the details of the speech as part of my essay without having to retype it. (87)
Cortical columns create reference frames for every object they know. Reference frames are then populated with links to other reference frames; it is reference frames all the way down. (87)
To be an expert in any domain requires having a good reference frame, a good map. (87)
Albert Einstein started with the same facts as his contemporaries. However, he found a better way to arrange them, a better reference frame, that permitted him to see analogies and make predictions that were surprising. (88)
The neocortex has many models of any particular object. The models are in different columns. They are not identical, but complementary. For example, a column getting tactile input from a fingertip could learn a model of a cell phone that includes its shape, the textures of its surfaces, and how its buttons move when pressed. A column getting visual input from the retina could learn a model of the phone that also includes its shape, but, unlike the fingertip column, its model can include the color of different parts of the phone and how visual icons on the screen change as you use it. (96)
Knowledge in the brain is distributed. Nothing we know is stored in one place, such as one cell or one column. Nor is anything stored everywhere, like in a hologram. (97)
There isn’t a single model of coffee cups. What you know about coffee cups exists in thousands of models, in thousands of columns. (97)
The neocortex is not dependent on a single cortical column. The brain continues to function even if a stroke or trauma wipes out thousands of columns. (98)
Scientists have long assumed that the varied inputs to the neocortex must converge onto a single place in the brain where something like a coffee cup is perceived. This assumption is part of the hierarchy of features theory. However, the connections in the neocortex don’t look like this. Instead of converging onto one location, the connections go in every direction. This is one of the reasons why the binding problem is considered a mystery, but we have proposed an answer: columns vote. Your perception is the consensus the columns reach by voting. (99)
Is the neocortex organized as a hierarchy or as thousands of models voting to reach a consensus? The anatomy of the neocortex suggests that both types of connections exist. (106)
It doesn’t matter that the image you are observing is two-dimensional; the models in your neocortex are three-dimensional, and that is what you perceive. (109)
You can use words to describe it to me, but the knowledge is not stored in words or rules. The knowledge is the model. (125)
Book Notes
As an author of information architecture and ambient findability, the thousand brains theory of intelligence resonates. Our boxes and arrows and strange connections are reflections of the neocortex. Wayfinding in digital and physical environments is the same. The work of an information architect is to create useful reference frames.
While I love the work of Dave Gray, Dan Roam, and Joe Elmendorf, I am disappointed by attempts to sketch my mental models. In the past, I blamed my lack of skill, but now I think it’s also because any sketch is impoverished relative to its mental model. The richness of our multisensory mental models can never be captured by words-pictures.
This book rewired my brain. It changed how I think about insight and intelligence. It refined my mental model of mental models. But it also raised my hackles.
While Part 1 is evidence-based, Parts 2 and 3 are speculative and dangerous. I believe Hawkins is right that we won’t get AGI without embodiment (robots able to learn via sensory experience with reality). But I am horrified by his false binaries of old-new brain and emotion-intelligence. In his view, the old emotional brain is bad, and we should modify human genes, so our progeny has more “desirable” characteristics.
Ironically, Jeff hasn’t yet had the epiphany of the era of artificial intelligence.
I feel, therefore I am.
He needs a new reference frame to help him see intelligence isn’t why we matter.