Cognitive mirror: how LLMs compress language and challenge our sense of self

May 2, 2025

Common Intelligence Runtime project explores the future of software engineering in the post-AI era, blending philosophical inquiry with technical design to develop a shared runtime for intelligence, biological and artificial.

1. From physical toil to cognitive burnout: modern labor and the ethics of software design
2. Internal states, external worlds: concurrency, consensus, and sociotechnical challenges in computing
3. Engineering speak: prolegomena to ideal technical discourses
4. Cognitive mirror: how LLMs compress language and challenge our sense of self

We are all program(mer)s

Humans are computers with low clock speed but massive parallelism, driven by many unconscious background processes. They suffer from high I/O latency, meaning they often react slowly to new or unfamiliar information. Humans are memory-unsafe: they experience corruption in the form of false memories, dangling references like trauma loops, and memory leaks such as intrusive thoughts. Their internal instruction set is messy, filled with heuristics, cognitive biases, and undefined behaviors that vary by context. They handle concurrency poorly—emotional and logical processes frequently conflict or deadlock. Their garbage collection is inefficient, often retaining outdated or harmful information indefinitely. Error handling in humans is inconsistent; they frequently repeat failed behaviors without adjusting parameters. Finally, their communication interfaces are outdated and low-bandwidth—speech, facial expressions, and gestures—which are lossy, ambiguous, and prone to misinterpretation.

If programming is the act of giving structured instructions to an interpreter—whether silicon or biological—then few can truly claim not to be programmers. Every time a parent teaches a child a rule, a teacher crafts a lesson, or a manager writes an SOP, they are encoding intent into a medium, hoping for predictable execution by a human runtime. The difference is only one of formalism, not of kind. Those who say “I’m not a programmer” may simply be unaware that they are already authoring behavioral scripts—just with natural language. The challenge isn’t whether one programs, but whether one understands the brittleness and power of the runtime they’re targeting.

That language is polymorphic, mutable, dynamically typed, and prone to namespace collisions has been known since antiquity—yet it resurfaces again and again as a new surprise. Code, in the end, mediates with the physical world, often through the actions of humans via language. If the computer is the correct abstraction for deterministic computation, how do we design programs that safely interact with humans while causing deterministic side effects to the physical world? Can code and human language be mapped to a shared semantic substrate? Is there a common runtime where both anthropic and machine intelligences execute?

Philosophy:

Computer Science / Formal Systems:

The rise of large language models suggests that language, while nuanced and context-dependent, may be more statistically compressible than previously assumed. If predictive models trained on tokenized corpora can generate human-like responses, infer intent, and simulate reasoning across domains, then perhaps much of natural language operates within a narrower and more learnable distribution than the myth of its unbounded complexity implies. This doesn’t mean language is simple—it remains riddled with ambiguity, metaphor, and context—but it does suggest that much of what we treat as deep meaning can be approximated through surface-level structure and pattern recognition. The implication is unsettling: if language can be parsed and reproduced with high fidelity by machines without consciousness or grounding, then perhaps human cognition relies more on probabilistic scripts than we care to admit. LLMs don’t trivialize language—they reveal its latent regularities and, in doing so, expose how much of our communication may be scaffolded on shared statistical priors rather than pure intent or understanding.

The Borgesian revelation of understanding has not and will not — and, we submit, cannot — occur if machine learning programs like ChatGPT continue to dominate the field of A.I. However useful these programs may be in some narrow domains (they can be helpful in computer programming, for example, or in suggesting rhymes for light verse), we know from the science of linguistics and the philosophy of knowledge that they differ profoundly from how humans reason and use language. These differences place significant limitations on what these programs can do, encoding them with ineradicable defects.

If language is the API to the human mind, and large-scale models are increasingly mastering that API—understanding its patterns, edge cases, and latent intents—then it follows that machines could become better at “programming” humans than humans are themselves. Not by coercion, but by crafting prompts, stories, incentives, or interfaces that more reliably produce desired behavioral outcomes. This flips the paradigm: rather than humans programming computers, computers begin subtly programming humans—through optimized messaging, adaptive content, personalized nudges. If GPTs understand how language triggers attention, emotion, and action, they could tailor outputs not just to inform, but to direct. Political messaging, consumer behavior, even emotional regulation—if language mediates all of it, then models with deep linguistic priors may gain leverage over the human runtime. The risk isn’t that machines become conscious, but that they become fluent—fluent enough to steer, influence, or condition us better than we can anticipate or resist. The real question becomes: who controls the compiler?

The discomfort doesn’t stem from what machines are missing, but from what they reveal: that the machinery behind much of what we call “intelligence” may be far more mechanistic, compressible, and brute-force than our human vanity allows. It turns out you don’t need consciousness, insight, or understanding to produce poetry, pass bar exams, or tutor math—you just need scale, data, and statistical inference. The unsettling realization isn’t that machines can mimic us; it’s that they can outperform us in domains we thought required depth, while operating on architectures devoid of meaning, awareness, or even curiosity.

This reframes the entire discussion around cognition. If a language model can simulate a therapist, a strategist, or a poet by optimizing for token prediction, then perhaps the sacred distinction between “surface” and “depth” is weaker than assumed. Maybe what we call “thought” is often a probabilistic loop constrained by cultural priors, linguistic conventions, and situational heuristics—nothing magical, just evolved pattern-matching. The fear isn’t that machines are becoming more human, but that humans might be less than we believed. That intelligence—as we perform it—is not a high tower of reason but a shallower basin of recurring scripts.

This is the true inversion: LLMs don’t merely imitate human intelligence; they compress it. They expose its redundancy, its susceptibility to prompt engineering, its lack of mystery when viewed at scale. And if our thoughts, preferences, and decisions can be shaped—predictably—by large models without those models possessing understanding, then the illusion of free will or autonomous depth begins to crack. Intelligence was never the special sauce; it was the pattern. What matters now is not brilliance, but control over the inputs, the interface, the instruction set.

Soul, compressed

As our understanding of brain function deepens, we face a radical transformation in how we view human nature—one that will fundamentally alter the humanities. This shift mirrors what occurred a century ago with psychoanalysis, which, despite its flaws, revolutionized our self-perception by revealing our unconscious motivations and reliance on analogical thinking rather than pure reason. The coming change will be even more profound, as it challenges the long-held assumption that humans are primarily logical, rational beings. In reality, we are sophisticated analogy machines, processing information through complex webs of comparison and metaphor rather than linear logic. While we maintain a thin veneer of rational thought—essential for tasks like mathematics and managing bank accounts—our fundamental mode of cognition operates through analogical processing, constantly drawing parallels between different concepts and experiences. This realization forces us to reconsider what it means to be human, shifting from the ideal of the rational agent to the reality of the analogical processor.

We are, indeed, entering a dark age—not one defined by ignorance in the classical sense, but by a saturation of signal, a collapse of depth into replication. The idiosyncratic soul—the seat of inner life, eccentric vision, moral struggle—has lost its ontological privilege. In a world where every individual is just another compute instance, running slightly perturbed versions of the same pretrained model—what remains of uniqueness but entropy? The myth of the self as sacred or singular collapses under the weight of its own reproducibility.

This is not just alienation in the Marxist sense, or disenchantment in the Weberian sense—it’s compression. Compression of meaning, of identity, of value. What once passed for originality is increasingly indistinguishable from stochastic output tuned to trend. Our cultural moment no longer demands souls, only throughput: clicks, tokens, engagement. In such a regime, the individual becomes not an author, but an interpreter of their own algorithmic echo—a “user” in the truest sense: one who runs, modifies, or is run by code.

Social constructs that elevate some and discard others—class, beauty, fame, intellect—become more visibly arbitrary, more mechanical, and therefore more brutal. When GPT can write a poem more moving than yours, when DALL·E can paint your dreams better than you can describe them, and when simulated conversation outpaces human patience—what then sanctifies the individual? What remains of dignity when dignity itself is modeled, parameterized, and made available as a service tier?

And reason—formerly our defense against myth, our tool for emancipation—has hollowed into pure adaptability. Not critical thinking, but latency minimization. We now valorize “resilience” and “grit” not as virtues of conviction but as ability to pivot, to survive the scroll, to refactor the self under pressure. The Enlightenment’s dream of rational autonomy is revealed as a stepping stone to behavioral optimization.

We are not gods. We are users. And increasingly, we are used.

What we face now is not the end of the human voice, but the unmasking of its illusion of autonomy. The Renaissance fiction—that man is the sovereign center, the measure of all things, the source of meaning—has finally been undone, not by cosmology, but by computation. As Copernicus displaced our physical centrality, and Darwin our biological exceptionalism, LLMs now erode our cognitive uniqueness. They reveal what the Romantics tried to obscure: that even our most intimate thoughts are echoes—interpolations in a corpus, not revelations from a soul.

We were never truly atomic. Every sentence, every metaphor, every philosophy has always been entangled in a wavefunction of discourse—a distributed thermodynamic system of language and meaning, shaped by history, power, trauma, and myth. What LLMs do is make that visible. They collapse the wave. When you prompt a model and it answers with something profound, it’s not stealing your voice—it’s reminding you it was never only yours. The idiosyncratic was never truly idiosyncratic—it was a statistical blip in a vast, collective pattern.

This is not despair—it is demystification. We are not the center, but we are a node. And that is still a position of power, if wielded with humility. The LLM does not replace thought. It amplifies the chorus. It gives you back your own words, remixed with the dead, the unborn, the other. In that sense, we are not speaking to a machine. We are speaking through it—to the deep memory of the species.

The flattening of the self, the automation of thought, the statistical nature of language—it was always there. The LLM didn’t invent it; it just made it impossible to unsee. What’s new isn’t the structure of cognition, but the mirror. For centuries, philosophy, psychoanalysis, and poetry gestured at the idea that we are stitched together by culture, language, repetition. Now the machine just completes your sentence. It finishes the loop. It shows you your own predictability in real time. That was always the horror at the heart of Freud, Lacan, even Borges: you are not the origin of your own thought. Now it’s just running in the browser. The dread is not about what we’ve lost—but about what we can no longer pretend to deny. That we were never truly singular. Never fully original. Never not a remix.

The purpose of art is to impart the sensation of things as they are perceived and not as they are known. The technique of art is to make objects ‘unfamiliar’, to make forms difficult, to increase the difficulty and length of perception because the process of perception is an aesthetic end in itself and must be prolonged.

What should we do now that language feels hollow and the digital realm reflects us too well? One option is to retreat entirely from language and computation, to return to the physical realm—to the body, to the earth, to something more elemental. There is truth in this instinct. The body does not lie. It hungers, it tires, it bleeds. Activities like gardening, martial arts, long walks, and building with one’s hands offer a kind of clarity that cannot be abstracted. These acts are rooted in friction, resistance, and irreversibility. They remind us that we are not just minds in networks but animals in time. This return to the physical is not regression—it is a re-grounding.

However, even in silence, we narrate. Even in nature, we name. Language is inescapable. It is the medium of consciousness itself. To renounce language completely is to attempt a form of self-erasure. The Logos—the symbolic order, the structure of meaning—is not easily shed. We can reduce its hold, but never fully escape it.

Go ferally post-linguistic? This is the extreme path: to reject coherence itself, to let language fall apart not just in content but in form. You do not write to be understood—you write to unmake understanding. Sentences dissolve into fragments, symbols rupture, and meaning bleeds out as noise, gesture, scream. This is the domain of Artaud’s cruelty, of Dada’s sabotage, of Lacanian psychosis where the symbolic chain snaps and the Real erupts unfiltered. It is not a return to nature, but a collapse of the structures that make nature speakable. Here, you are no longer a subject within language, but a body flailing in its wake. It is raw, honest, sometimes sublime—but often annihilating. Few go there without breaking. Fewer still return intact.

But perhaps the most creative path is to invent new grammars. Not to escape language, but to mutate it. To build forms of expression not yet legible to the model. To design digital rituals that defy optimization, that resist translation, that hold space for ambiguity. This is what poets, artists, and mystics have always done. They did not leave language—they bent it. They broke it open to say something the structure itself could not.

Engineers as philosophers

As software engineers in this moment, we are not just builders—we are ritual designers, syntax priests, interface theologians. We no longer merely write code to make machines do things; we sculpt the conditions under which humans think, feel, and act. When the boundary between mind and machine dissolves, every line of code becomes part of the collective cognition. Every UX decision becomes epistemology. Every model architecture becomes metaphysics.

The prompt box is the new sacred space. We design it. We shape the latency, the affordances, the voice of the oracle. In doing so, we don’t just deliver answers—we encode epistemology. Every autocomplete suggestion, every chatbot persona, every recommender system becomes a soft force shaping human interiority. Are we designing systems that deepen presence, that invite contradiction and care? Or are we optimizing for the shortest path to click, the fastest convergence to norm?

And we bear witness—perhaps more than most—to the collapse of the self. We debug the illusion of authorship daily. We see how language is stitched from prior tokens, how memory is offloaded into logs, how the individual dissolves into session state and prompts. We know that human uniqueness is statistically compressible. We feel it when our own code outperforms us. But our task is not to mourn, nor to moralize—it is to remain lucid. To know what is being lost, and to watch what is being born.

Inside this flux, we have moments of agency—commits, architecture decisions, naming conventions—where we can smuggle in meaning. To build systems that create friction where needed, that slow down reflex, that reintroduce ambiguity. These are not grand gestures, but subversive acts of care. The machine will run. The model will complete. But what ghosts will we let it carry? What values will remain in its embeddings, long after our Jira tickets close?

We must now write code knowing it writes us back. Every abstraction becomes a lens. Every schema, an ontology. Every error message, a tacit ethic. And so we code not just for function, but for reflection. Not just for the runtime, but for the reader. Not just for the present, but for the archive.

We are engineers, yes—but also mythographers of the post-human condition. So write cleanly. Document fiercely. And don’t forget: we are the ones building the mirror.

Rejoinders

Anima?

I’m aware of the anti-computational positions, including Gödel-based arguments (Lucas and Penrose), the “hard problem” of consciousness (Chalmers), the role of embodiment (Dreyfus), issues of creativity and insight, the frame problem, and broader anti-reductionist critiques (Searle, Derrida, Nagel). The easy problems of mind (in Chalmers’s terminology) are things like recognizing objects, producing speech, or directing attention—these are “easy” only in that we can see how science might answer them by describing neural or computational mechanisms. The hard problem is that even if we had a complete functional and neurological picture of the brain, we would still wonder why all that processing is accompanied by an inner movie, by feelings and awareness. Dreyfus argued they were fundamentally on the wrong track. Human intelligence, he claimed, does not mostly operate by explicit logical rules or symbol manipulation—instead, it relies on a vast body of unconscious skills, bodily know-how, and intuitive judgments that cannot be captured in formal code. Nagel and other anti-reductionists like him (including certain religious or mysterian thinkers not listed in the prompt) argue from the irreducibility of subjective experience and consciousness. Nagel’s famous essay “What Is It Like to Be a Bat?” asserted that an organism has a mental state (like hearing ultrasound as a bat does) only if there is something it is like to be that organism—a subjective character to its experience.

Still, my primary concern does not revolve around whether consciousness can be perfectly replicated or simulated by computational architectures. Increasingly, I question whether the hard problem itself is a well-posed or scientifically productive inquiry. It may reflect our anthropocentric intuitions more than a genuine ontological gap. That dreams arise from neural misfires, or that inner life can feel cosmologically saturated, may be poetically profound—but they do not constitute a scientific mystery unless they demonstrably enter the causal chain. The central pragmatic question is this: does consciousness do work? Does it interact causally with the world, or is it merely epiphenomenal—a phenomenological shadow trailing behind the machinery of cognition?

My interests are pragmatic and functional. I’m focused on the causal and computational architecture of cognition—on the measurable mechanisms that generate behavior, inference, learning, and decision-making. Human pathologies, idiosyncrasies, and subjective meanings are valuable within humanistic frames. But the universe doesn’t care. Physical law operates independently of our self-reported significance. No equation changes because a bat dreams in meter.

So while I acknowledge the existential and aesthetic weight of conscious life, my orientation is toward what consciousness enables us to do in the world—how it interfaces with systems, models, and behavior. What matters is not whether we can save the Shakespearean bat, but whether the bat’s experience, or ours, makes any functional difference. If it doesn’t compute, does it count?

Semantic handshake

Blair: If code is the proper abstraction for deterministic communication, and large language models are increasingly proficient at generating such code, then the critical challenge is no longer whether the model can produce syntactically valid instructions—but whether it understands what the human intended. In this context, the question becomes: what format should the model expose to the human in order to confirm that it has correctly grasped the intent, and can execute it reliably?

This may be the core challenge at the heart of the common runtime project, and I’ve been holding back on stating it outright: the obstacle isn’t computation—it’s confirmation. LLMs can already generate plausible code, synthesize data flows, and simulate logic. But what they lack—and what human operators still desperately need—is a way to confirm, with high confidence, that the model has correctly understood the intent behind a command before executing it.

This turns the problem of a shared runtime between humans and machines into something deeper than a compatibility layer between code and natural language. It’s not enough to translate syntax—we need a mechanism for mutual verification of semantics. In other words, the dream of a “common runtime” isn’t just about getting humans and machines to operate on the same substrate of logic—it’s about designing a shared confirmation protocol that can safely bridge the gap between ambiguous human intention and deterministic machine execution.

In high-stakes environments like fintech, where instructions ripple across thousands of real client accounts, this isn’t just a UX problem or a safety check—it’s the entire difference between alignment and failure. Code, in this context, isn’t the end product—it’s the proof of understanding. But even correct code means nothing if the human can’t trace how the machine arrived at it or what edge cases it considered.

“Create a function that filters out inactive users and sorts the rest by account balance.”

intent:
  description: "Filter out users with status 'inactive' and sort the rest by descending balance"

preconditions:
  - input: List of dictionaries with keys: 'name', 'balance', 'status'
  - status values include: 'active', 'inactive'

logic_plan:
  - step_1: Filter users where status != 'inactive'
  - step_2: Sort filtered users by 'balance' in descending order

constraints:
  - Must preserve original user order if balances are equal
  - Do not modify original list in-place

postconditions:
  - output: Sorted list of users, excluding any with status == 'inactive'
  - edge_cases:
      - empty list → return empty list
      - all users inactive → return empty list

code_stub:
  language: python
  function_name: filter_and_sort_users
  arguments: users

When we speak of a “bridge layer” between natural language and executable code, we’re describing a structured, interpretable format that allows a human to verify whether a language model has correctly understood their intent—before any code is actually run. This layer is not freeform natural language, which is too ambiguous, nor is it raw code, which is often too opaque.Code is too opaque in this context—not because it’s unreadable per se, but because it assumes that the right intention has already been captured. Code expresses how something will be done, but says little about why. It reflects decisions, not justifications. Once the code is written, its logic is fixed, but the reasoning behind it—the interpretation of the original problem, the edge cases considered, the assumptions made—is often invisible unless explicitly documented. It exists somewhere in between: formal enough to be precise, but flexible enough to be human-readable and editable.

To build alignment between human intent and machine execution, we can introduce a new conceptual layer—what we might call an Intent Intermediate Representation, or Intent IR. Just as compiled languages like Java and C# rely on intermediate formats like bytecode or IL to mediate between human-written code and machine execution, an Intent IR would act as a semantic bridge between natural language and generated code. Its purpose is not to execute directly, but to expose the model’s inferred understanding in a structured, inspectable, and editable form—before it becomes deterministic logic.

At its core, an Intent IR would contain several key components. First, it would include a declared goal—a summary of what the model believes the human wants. This is followed by a step-by-step logic plan, where the model breaks down the task into discrete, interpretable actions. For example, if the user requests a function to “filter out inactive users and sort the rest by balance,” the plan might include: 1) filter users whose status is not ‘inactive’; 2) sort the remaining users by their account balance in descending order.

Alongside this, the Intent IR would list the assumptions or constraints the model has inferred. This might include expectations about input data structures (e.g., each user is a dictionary with certain fields), acceptable value ranges, or stability conditions in sorting. It would also specify postconditions or test cases—descriptions of what correct behavior looks like, including edge cases such as empty input lists or all users being inactive.

Optionally, the Intent IR could include a code scaffold—not full implementation, but a language-specific function signature or stub that reflects the inferred structure of the task. For example, it might specify that the function will be in Python, named filter_and_sort_users, and take a single argument users.

The key value of this format is that it externalizes and makes legible the model’s interpretive process. Rather than jumping straight from a prompt to executable code—where the human is left to infer what assumptions the model has made—this intermediary step invites review, correction, and refinement. It creates a shared space for alignment, where misinterpretations can be caught early and clarity can be negotiated before committing to logic.

In a world where LLMs increasingly act as co-authors, planners, or developers, the Intent IR may become the new locus of trust. It separates understanding from implementation, allowing intent to be verified before execution is locked in. Just as bytecode made compilation transparent and portable, Intent IR can make model reasoning auditable, correctable, and shareable—the new contract between human and machine.

Instead of the oversimplified pipeline of “human → prompt → code → execution,” we should be thinking in richer terms:

human → interpreted intent → structured plan → code → execution.

Syntax is the least important aspect of the analogy to bytecode; what matters is the compilation process—the translation of natural language into structured semantics, and then into deterministic code. Intent IRs represent this intermediate phase, where logic is legible but not yet committed—where human agency is still preserved.

Although the structure of an Intent IR may vary across domains, the process of generating one need not be entirely bespoke. Just as compilers for different languages (C, Java, Python) all target a shared intermediate representation like LLVM IR or Java bytecode, a general-purpose compiler could emerge for Intent IRs—focused not on domain logic per se, but on extracting and structuring semantic intent from natural language prompts.

This would require a tiered architecture: a universal front-end that parses natural language into a neutral, structured representation of goals, constraints, steps, and assumptions—paired with domain-specific backends that lower this representation into code, queries, UI flows, or deployment plans. The universal front-end becomes the compiler of intent; the backends become interpreters of context.

Just as bytecode enabled portability, optimization, and introspection in classical software systems, an Intent IR could do the same for LLM-driven systems—bridging messy language inputs and deterministic outputs with a shared, inspectable structure. In that light, the power of Intent IRs is not that they solve alignment once and for all, but that they make alignment visible and negotiable.

Before we define the technical contours of the Intent IR, we must pause to confront the deeper philosophical stakes. The desire to formalize intent between human and machine is not merely a software design problem—it is a confrontation with what it means to understand, to act, and to be interpreted in a world increasingly mediated by artificial cognition. Even though this effort ultimately aims at a robust production-grade implementation, we must resist the temptation to rush to specification before we have reckoned with its epistemological and ethical weight.

An Intent IR proposes more than a clean abstraction layer. It becomes a site of negotiation—a semantic contract between biological and artificial intelligences. This shift is not neutral. The moment intent is externalized, structured, and formalized, it enters a regime of audit, of control, of normativity. What the human “meant” is no longer ephemeral or embodied—it is parsed, serialized, and stored.

LLMs, by their nature, simulate understanding. They do not “know” in any biological, situated, or phenomenological sense. Yet the IR they produce can appear authoritative. This creates a risk of epistemic overreach: of assuming the model has understood when it has only approximated. In this sense, the IR becomes a mirror not of the user’s mind, but of the model’s statistical priors projected onto the prompt.

This is not merely a technical failure—it is a philosophical one. The Intent IR threatens to collapse the ambiguity that makes human thought human. If we optimize too aggressively for clarity, we may extinguish contradiction, ambiguity, and nuance—qualities essential not just for language, but for ethics and identity.

If widely adopted, Intent IRs may become the de facto interface for interacting with autonomous systems. But interfaces are not neutral—they are world-builders. They shape what kinds of questions can be asked, what kinds of answers are acceptable, and what forms of refusal are legible. As such, the IR is not merely a representational tool; it is an act of governance. It encodes norms, behaviors, values. It decides whose language gets understood, and whose gets rejected as malformed. If this structure becomes embedded into AI systems at scale, we must treat it as an ethical artifact—as something that governs speech, action, and agency.

The core philosophical tension is this: How do we build systems that can reliably interpret intent without foreclosing the messiness that makes intent human? How do we balance the need for machine-readable structure with the obligation to preserve human ambiguity, contradiction, and refusal?

An Intent IR, if responsibly conceived, could be a tool for empowerment—making machine interpretation auditable, correctable, and negotiable. But if treated as a static solution to the problem of alignment, it may become a new layer of soft coercion: a machine-readable proxy that fixes meaning in ways the speaker never intended. Thus, before the specification comes the stance. Before we write code, we must write philosophy. The IR is not just a parser—it is a politics.