Coming out of stealth · October 28Out of stealth · October 28
Cognitive infrastructure
← ArticlesResearchResearch

Cognitive Infrastructure

Why AI Needs a New Systems Architecture


A dense network of white lines and nodes on black, brightest at the center.

1. Intelligence Is Not a Single Operation

The recent success of Large Language Models has fundamentally changed how we think about artificial intelligence. Today, a single language model is expected to answer questions, plan, summarize, write code, analyze documents, simulate conversations, perform scientific reasoning, make strategic decisions, critique its own outputs, and orchestrate increasingly complex agentic workflows.

It is tempting to conclude that intelligence itself is a single computation. However, we believe this is the first architectural mistake.

Human intelligence cannot be reduced to one single operation. It is a remarkably rich collection of distinct cognitive processes that cooperate to solve problems. We analyze. We compare. We generalize. We plan. We simulate future scenarios. We imagine alternatives. We perform counterfactual reasoning. We place ourselves in someone else's shoes. We revise our beliefs when presented with new evidence. We detect contradictions. We prioritize. We abstract. We discover patterns. Each of these represents a different computational operation with different objectives, different inputs, and different notions of success.

Software engineers would never expect a CPU to expose a single instruction called COMPUTE(). Modern processors expose hundreds of instructions because computation itself is diverse. Arithmetic, memory access, branching, synchronization, and vector operations exist as separate primitives because they solve fundamentally different problems.

We believe intelligence is no different. Rather than viewing reasoning as a monolithic capability, we believe it is better understood as a rich instruction set composed of specialized cognitive operations that together give rise to intelligent behavior.

All of this, of course, raises a natural question: if intelligence is composed of many distinct operations, what exactly do those operations operate on?

2. Every Computation Requires a Representation

If intelligence is composed of many distinct cognitive operations, those operations must share something in common. Every computation operates on a representation. Arithmetic operates on numbers. Databases operate on tables. Graphics engines operate on meshes and scene graphs. Operating systems manage processes. Compilers transform intermediate representations. Every mature computational discipline eventually converges on an internal object that captures the state of the computation while remaining independent of the interface through which users interact with it.

This separation is so fundamental that software engineers rarely think about it. When writing SQL, we do not imagine the database executing SQL directly. SQL is a language for expressing intent. Underneath, the database transforms that query into relational algebra, optimizes it, selects execution plans, and manipulates internal data structures before producing a result. Likewise, compilers do not optimize C++ or Python directly. They transform source code into intermediate representations upon which dozens of optimization passes operate before producing machine code.

The representation is not an implementation detail; it is what makes computation possible. Representations allow multiple operations to cooperate over the same underlying state. They make optimization possible. They make composability possible. They make specialization possible. Most importantly, they allow each computational operation to evolve independently without changing the representation itself.

This observation raises an interesting question for artificial intelligence. If planning, simulation, Theory of Mind, belief revision, counterfactual reasoning, and causal reasoning are genuinely different cognitive operations, what shared representation allows them to cooperate?

Today's AI systems rarely ask that question. Instead, they largely assume that the interface through which intelligence communicates is also the representation upon which intelligence computes. We believe those are two fundamentally different things.

3. Language Is the Interface, Not the Computational Substrate

Language is one of humanity's greatest inventions. It allows one mind to communicate with another despite each possessing different experiences, different beliefs, and different internal representations of the world. Every sentence is an act of compression: an attempt to project an enormously rich internal understanding into a finite sequence of symbols that another person can reconstruct. Language succeeds precisely because it does not attempt to communicate everything. It abstracts. It compresses. It leaves details implicit. It assumes shared context. It relies on background knowledge. It allows ambiguity when precision is unnecessary and precision when ambiguity becomes dangerous. Its purpose is not to faithfully reproduce an internal mental state, but to construct sufficient shared understanding for communication to occur.

That distinction is important. Throughout the history of computing, interfaces have almost never served as the computational substrate itself. SQL is not how a database computes. It is how users express intent. Graphical interfaces are not how operating systems execute programs. Source code is not how compilers optimize software. Every mature computational discipline separates the interface through which humans communicate from the representation upon which computation takes place.

We believe intelligence should be no different. Language is an extraordinary interface for interacting with intelligence. It is arguably the best interface humans have ever developed. But communicating thought and performing thought are fundamentally different computational problems.

Humans constantly perform cognitive operations that never become language. We mentally simulate future scenarios while driving. We predict another person's reaction before speaking. We recognize tension in a room without consciously verbalizing it. We revise our understanding as new evidence appears. We notice contradictions before we can explain why they feel wrong. Much of intelligence unfolds before language, and often without language altogether.

This is why we believe language should remain the primary interface for intelligence, but not necessarily the substrate upon which intelligence computes.

Once we separate communication from computation, an obvious question emerges. If language is the interface, what is the computational state upon which intelligence actually operates?

4. LLMs Are the Monoliths of AI

Software engineering has undergone the same architectural transition repeatedly over the past several decades. Although compilers, databases, operating systems, and distributed systems evolved independently and solved fundamentally different problems, they all converged toward remarkably similar architectures. Interfaces gradually became separated from computation, computation converged toward explicit intermediate representations, and increasingly rich functionality decomposed into independently evolvable components. This convergence is difficult to dismiss as coincidence. We believe it reflects a more fundamental principle of computation: as systems mature, the computational substrate naturally separates from the interface through which humans interact with it, allowing specialized operations to cooperate over a shared representation rather than compete inside a single monolithic execution engine.

Compilers illustrate this progression particularly well. Programmers write C++, Rust, or Python, yet compilers do not optimize source code directly. Instead, the front-end translates source code into an intermediate representation specifically designed for computation rather than communication. Every subsequent optimization pass (constant propagation, dead-code elimination, register allocation, vectorization, loop transformations, and many others) operates over that shared representation before the back-end ultimately generates machine code. The breakthrough was therefore not the optimization passes themselves. It was the introduction of an explicit computational representation that allowed independently specialized transformations to cooperate while remaining completely decoupled from the language used by the programmer.

Databases followed precisely the same trajectory. Although developers interact with databases through SQL, databases do not execute SQL internally. SQL exists to communicate intent. The database parser transforms that intent into relational algebra, query planners construct execution plans, optimizers rewrite those plans, indexes accelerate retrieval, and execution engines ultimately produce the requested result. Once again, the interface through which humans communicate became distinct from the representation upon which computation actually occurred.

Distributed systems reached the same conclusion from yet another direction. Early software systems accumulated authentication, storage, scheduling, messaging, business logic, monitoring, and countless other responsibilities inside a single executable. These systems often achieved extraordinary success precisely because everything lived in one place. Over time, however, the growing diversity of computational responsibilities made independent evolution increasingly difficult. The transition toward service-oriented architectures and microservices was not motivated by a preference for distributed systems, but by the realization that fundamentally different computational concerns deserved different implementations while remaining interoperable through stable interfaces and shared contracts.

Although these examples originate from different branches of computer science, they all reveal the same underlying architectural principle. Mature computational systems separate communication from computation, computation from representation, and complex behavior into independently evolvable operations acting upon a common substrate. Every time software engineering encountered increasing complexity, it eventually converged on this decomposition.

Artificial intelligence has not yet completed that transition.

Today's AI systems increasingly ask a single computational primitive (the language model) to implement planning, simulation, long-term memory, Theory of Mind, counterfactual reasoning, tool orchestration, natural language generation, and an ever-expanding collection of additional cognitive capabilities. As new responsibilities emerge, they are incorporated into the same execution engine rather than decomposed into independently evolvable computational operations. The remarkable success of Large Language Models has therefore delayed, rather than eliminated, the architectural transition that every mature computational discipline has eventually undergone.

This is why we describe Large Language Models as the monoliths of artificial intelligence. The observation is not intended as a criticism of language models themselves, any more than successful monolithic software systems should be viewed as engineering failures. On the contrary, monoliths are often remarkably successful because they allow new capabilities to emerge rapidly behind a single abstraction. Their very success, however, eventually concentrates so many computational responsibilities within the same execution engine that further progress increasingly depends upon architectural decomposition rather than incremental improvement.

If software engineering has repeatedly converged toward this pattern, the natural question is whether intelligence requires the same transition. More specifically, if cognition is to decompose into independently specialized computational operations, what representation will allow those operations to cooperate while preserving a coherent understanding of the world?

5. Belief State: The Computational State of Intelligence

Suppose we accept the conclusion of the previous section. Intelligence, like every mature computational discipline, should eventually decompose into independently specialized computational operations. Planning should not be implemented in exactly the same way as simulation. Counterfactual reasoning should not necessarily share the same implementation as Theory of Mind. Belief revision should not require the same optimization strategy as language generation. If these operations are to evolve independently while remaining interoperable, they must operate upon a shared computational representation.

What properties should such a representation possess?

  • First, it must be independent of language. A computational representation should preserve meaning regardless of how that meaning happens to be expressed. Two people can describe the same situation using entirely different words, just as the same idea can often be translated across languages without fundamentally changing its meaning. A representation designed for computation should therefore capture understanding itself rather than the particular symbols used to communicate it.
  • Second, it must be persistent. Intelligent behavior unfolds over time. Every new observation changes our interpretation of the world. Every conclusion influences future reasoning. Every hypothesis introduces new possibilities while eliminating others. Planning depends upon previous observations. Counterfactual reasoning explores alternative interpretations of the current situation. Theory of Mind constructs representations of another person's understanding. Belief revision updates previous conclusions as new evidence becomes available. None of these operations begin from scratch. They all transform an evolving computational state.
  • Finally, the representation must be sufficiently expressive to support an arbitrarily rich catalog of cognitive operations without being tied to any particular one. A compiler's intermediate representation does not belong to constant propagation, register allocation, or vectorization; it belongs to the compiler itself. Likewise, the computational representation of intelligence should not belong to planning, simulation, or language generation. It should provide a common substrate upon which every cognitive operation can cooperate while remaining independently evolvable.

We call this computational representation the Belief State. The name is intentionally literal. Just as a running program maintains a program state describing its current execution, an intelligent system maintains a Belief State describing its current interpretation of the world. It is not a conversation history, a prompt, a vector database, or simply a memory. Those are mechanisms for storing or retrieving information. A Belief State is something fundamentally different: it is the computational state upon which intelligence operates.

This distinction profoundly changes the role of language. Language no longer carries the burden of computation. It becomes the interface through which observations enter the system and through which conclusions are communicated back to the outside world. Between those two interfaces lies an explicit computational substrate over which intelligence itself unfolds. Planning transforms the Belief State into possible future states. Simulation explores alternative trajectories. Counterfactual reasoning branches it into hypothetical worlds. Theory of Mind constructs additional Belief States representing other agents. Belief revision updates it as new evidence arrives. Dissonance detection identifies inconsistencies within it. Every cognitive operation becomes an independent transformation over the same evolving representation.

Notice that this is precisely the pattern we observed in every mature computational discipline. Compilers introduced intermediate representations before optimization passes became independently engineerable. Databases separated SQL from relational algebra before query optimization became possible. Distributed systems established stable interfaces before services could evolve independently. In each case, the intermediate representation preceded the decomposition. We believe intelligence follows the same architectural principle. Once an explicit computational state exists, pipeline decomposition naturally enables instruction-set decomposition, allowing intelligence to evolve from a single monolithic computation into a rich ecosystem of independently specialized cognitive operations.

6. Engineering Intelligence

Once intelligence possesses an explicit computational state, the architectural consequences are surprisingly far-reaching. The primary benefit is not simply that the system becomes easier to understand. It is that intelligence itself becomes an engineering discipline.

Today, improving an AI system often means improving the language model responsible for every cognitive operation. Better planning, better memory, better simulation, better tool use, and better reasoning all ultimately compete for improvements to the same computational component. Because these capabilities are tightly coupled, progress in one area frequently requires changes to the entire system.

An architecture centered around an explicit computational state fundamentally changes this relationship. Once cognitive operations cooperate over a shared representation rather than through prompts alone, they become independently engineerable. Planning can evolve without redesigning memory. Better approaches to Theory of Mind can be introduced without modifying language understanding. New simulation algorithms can be incorporated without rebuilding the entire reasoning stack. Scientific progress becomes modular because every cognitive capability becomes a replaceable computational component rather than another responsibility assigned to the same monolithic execution engine.

This architectural transition mirrors every major evolution in software engineering. Compilers accelerated once optimization passes became independent transformations over an intermediate representation. Databases accelerated once query planning, optimization, indexing, and execution became independently evolvable components. Distributed systems accelerated once storage, computation, authentication, messaging, and scheduling became independent services communicating through stable interfaces. In every case, decomposition did not merely improve maintainability. It dramatically accelerated the rate at which the entire discipline could evolve.

We believe intelligence is approaching exactly the same transition.

Once cognition becomes composable, an entirely different software ecosystem becomes possible. Instead of exposing intelligence through a single opaque endpoint, Cognitive Infrastructure exposes a growing catalog of reusable cognitive capabilities operating over a common computational substrate. Planning becomes a service. Simulation becomes a service. Belief revision becomes a service. Theory of Mind becomes a service. Constraint propagation becomes a service. Dissonance detection becomes a service. Language generation itself becomes one cognitive service among many rather than the architectural center of the system.

Notice how closely this resembles the evolution of cloud computing. Modern software engineers rarely ask a cloud provider to "run an application." Instead, they compose storage services, messaging systems, authentication providers, databases, search engines, caches, schedulers, and countless other infrastructure components into increasingly sophisticated systems. Each service solves a specific computational problem while remaining interoperable through stable abstractions. We believe intelligence will ultimately evolve in exactly the same direction. Rather than asking a language model to perform every conceivable cognitive task, developers will compose specialized cognitive services into systems whose capabilities far exceed those of any single component.

This is what we call Cognitive Infrastructure. It is not another model, another orchestration framework, or another memory layer added around existing language models. It is a systems architecture that treats intelligence as a collection of independently evolvable computational capabilities operating over a shared computational state. Once that abstraction exists, the history of software engineering suggests that the resulting ecosystem will evolve more rapidly, become easier to optimize, and ultimately support applications that would have been prohibitively difficult to engineer within a monolithic architecture.

If the history of computing teaches us anything, it is that architectural abstractions determine what becomes possible. We believe the next generation of intelligent systems will not be defined solely by better language models, but by better computational substrates and the engineering ecosystems that emerge around them.

And if this thesis is correct, the AI stack of ten years from now will look remarkably different from today's. Language models will remain indispensable, but they will no longer occupy the center of the architecture. Instead, they will become one component within a broader computational ecosystem built around explicit state, specialized cognitive operations, and reusable cognitive infrastructure.

Follow the research