Coming out of stealth · October 28Out of stealth · October 28
Cognitive infrastructure
← ArticlesFounder StoryFounder Story

The Paradox of AI Safety

Why I Took the Leap of Faith to Build Dyssonance


Light filaments, a bright vertical band crossed by a thin diagonal line on black.

I started working in AI for the technical challenge and for the love of math. I stayed because it scared me.

What we were building was undeniably powerful; the people building it were brilliant. Yet the deeper I got into the field, the more uneasy I became about the assumptions underlying the systems we were creating.

At DeepMind, one of the hardest parts of my role involved something deceptively simple: deciding whether a dataset should be used to train Gemini or not. Every dataset required a final decision that weighed on my shoulders: a black and white answer, a clean yes or no. But the world that data came from was never clean. It was messy. Ambiguous. Contextual. Full of nuance that doesn’t fit neatly into categories. Yet the systems we were building required exactly that: binary decisions about data drawn from a fundamentally non-binary world. The pipeline demanded certainty even when certainty didn’t exist.

That discomfort bothered me deeply, and stayed with me long after I left. If the real world is ambiguous, and intelligence is supposed to operate in that world, why were we building systems that could only function on flattened, closed-ended representations of reality?

Eventually the burden of that responsibility became too heavy. I left DeepMind believing I might step away from AI entirely. Instead, I became obsessed with the problem.

The more I thought about it, the clearer it became: large language models are extraordinary systems, but they are fundamentally designed to operate on static datasets and fixed representations of knowledge. Yet we are increasingly deploying them into a world that is open-ended, ambiguous, and constantly changing.

Around the same time, strange and troubling anecdotes began appearing everywhere.

  • ChatGPT being jailbroken in new ways almost daily.
  • People reporting what some described as “AI psychosis.”
  • Tragic cases where vulnerable individuals formed relationships with AI systems that reinforced harmful behavior.

To many observers, these looked like isolated incidents. But to me, they all pointed to the same underlying problem: we are building systems trained on closed-ended data and asking them to operate safely in an open-ended world. And our primary strategy for aligning them assumes that we can solve this problem by writing better rules.

The Rulebook Problem

Rules work well in closed systems. In chess, for example, this works because every possible move is defined by the rules of the game. Similarly, air traffic control can operate based on rules because the environment is tightly constrained and governed by clear protocols.

The problem though is that the real world is not a closed system: it is ambiguous, incomplete, and full of situations that cannot be enumerated in advance. You simply cannot write an exhaustive list of everything that could go wrong. Every parent understands this instinctively. You cannot hand a child a list of every forbidden action and expect them to navigate life safely. The list would be infinite. Eventually they will encounter a situation no rule prepared them for. Good parenting works differently: you explain why certain actions are dangerous and talk about consequences, and over time, the child develops an internal sense of judgment, a value system that helps them make decisions when no explicit rule exists. The goal isn’t obedience to a rulebook but the ability to reason about consequences.

Yet most alignment approaches today rely precisely on rulebooks: guardrails, filters, prompts, and external constraints that attempt to forbid certain behaviors. These mechanisms can be useful. But they operate outside the reasoning process itself. They do not give the system the ability to evaluate situations internally, which means that in genuinely ambiguous situations (the ones we failed to anticipate), the system has no internal framework for deciding what to do.

The Missing Ingredient

Human intelligence doesn’t operate this way. We do not simply follow rules. We maintain beliefs about the world, evaluate consequences, and experience tension when our actions conflict with our principles. Psychologists call this cognitive dissonance: the discomfort we feel when our behavior contradicts deeply held beliefs. That discomfort is not a bug in human cognition: it is one of our most powerful safety mechanisms. It is cognitive dissonance that forces us to stop, reconsider, and revise our actions.

In other words, humans carry an internal system for maintaining coherence between beliefs and behavior. Current AI systems do not. They generate responses, but they do not maintain persistent beliefs about the world. They do not detect contradictions across time or experience any internal tension when their outputs violate previously stated principles. They simulate reasoning, but they lack the internal structures that make reasoning reliable.

Conscience, Not Consciousness

If intelligent systems are going to operate in an open world, they may need something closer to what humans rely on: an internal framework for evaluating consequences and maintaining coherence between beliefs and actions.

I’m not talking about consciousness here, but about something closer to a synthetic conscience. A conscience is not mystical. It is a structure that consists of deeply held principles that resist change, mechanisms for evaluating actions against those principles, and signals that trigger reevaluation when the two diverge.

In computational terms, this could look like a system that maintains structured beliefs about the world, including a set of stable axioms that function as moral anchors. When potential actions conflict with those beliefs, the system detects the inconsistency and reevaluates its decision.

This does not completely eliminate risk, and the reality is that nothing truly can. But it allows the system to reason about situations that were never explicitly anticipated.

The Safety Paradox

This leads to an uncomfortable realization: that the paradox of AI safety may be that making AI safer requires making it more human, not less. A system that can hold beliefs, maintain principles, and detect when its actions contradict them is no longer just following instructions: it essentially has something that can be interpreted as a position about the world.

Current AI systems are designed very differently. They are tightly constrained from the outside, through guardrails, filters, and rules, but they critically lack the internal structures required to navigate genuinely ambiguous situations. That combination can be fragile. When the rules fail, the system has little ability to recover.

A system with a belief structure operates differently. It can evaluate whether an action is consistent with its principles, detect contradictions, and reconsider before acting. Instead of relying solely on external constraints, it carries part of the safety mechanism within its own reasoning process. In an open and unpredictable world, that difference matters. Safety may not come from writing ever larger rulebooks; it may come from building systems capable of judgment.

The Real Choice

We moved fast. Over the past decade, AI systems have become extraordinarily capable, and they are already being woven into the fabric of the real world. That trajectory cannot simply be reversed. The question is no longer whether intelligent systems will exist: they already do. The real question is, what kind of internal structure those systems will have as they become more capable.

Most current safety strategies rely on external control: guardrails, filters, and rules designed to forbid certain behaviors. These mechanisms assume that we can anticipate dangerous situations in advance and constrain the system accordingly. But the real world does not cooperate with rulebooks. It is open-ended, ambiguous, and full of situations no one can enumerate ahead of time.

We cannot rewind the last decade of progress. The systems exist, and they will continue to improve. If we want them to behave responsibly in an open world, they will need more than external constraints. They will need internal structure, like a mental model of the world that allows them to evaluate situations, detect contradictions, and reason about consequences.

In other words, safety may ultimately depend less on controlling systems from the outside, and more on what happens inside them.

Follow the research