Thought 02 / Comparative cognition
Humans and AI meet in language. Underneath, they are radically different.
They speak alike. They “think” differently.
Scope Here, “AI” means large language models and the systems built around them.
| Dimension | 01 / biologicalHuman | 02 / engineered intelligenceAI |
|---|---|---|
| Learning | Changed by experience | Not changed by conversation* |
| Common ground | Retains the gist | Must be told again and again* |
| The details | Compresses detail into meaning | Works across far more supplied detail* |
| The big picture | Carries accumulated context | Works inside the frame it receives* |
Same languagedoes not meansame intelligence.
The dangerous assumption
AI is not magic. It is not a human mind running on silicon.
It is a different kind of intelligence.
Talking with AI feels familiar. It sounds human, so we assume the intelligence behind the words works like ours—and that the rest of the human package comes with it: learning, memory, experience, judgment, and shared understanding. Those human capacities do not come bundled with AI. A conversation can put you on the same page. AI does not carry that shared understanding forward.*
A role is more than a set of skills. It requires continual learning, shared understanding, and adaptive judgment—over time. AI does not adapt to the role. The role must be adapted to AI.*
01 / Learning
Humans keep learning.
AI does not.*
Human experience changes the brain itself—our wet weights. Talking to AI does not change the AI. The conversation changes. The AI does not.*
Experience changes the brain. Connections form, strengthen, weaken, and reorganize. Repeated activity can even change how efficiently signals travel. Our biological “wet weights” update.
Once deployed, its weights are fixed. A conversation can change what AI does now, but not what it has learned. When the conversation ends, the AI is unchanged. Nothing from the exchange becomes part of it. There is no continual learning.*
Humans adapt as they work. AI does not.*
People absorb changing goals, circumstances, relationships, exceptions, and practices—and carry what they learn into what they do next.AI will not adapt on its own. Its instructions and setup must be changed for it.
Organizations work because people adapt.
Goals shift. Conditions change. Mistakes reveal better methods. People carry those lessons into the next attempt, allowing the organization to improve without starting over.
Organizations adapt because their people keep learning.
People can start with incomplete instructions, learn by doing, and adapt as the job changes. AI cannot. The job must be made explicit: what to do, what to know, where to stop, how to check the work, and what to do when something goes wrong. When the job changes, people must update the AI.
AI does not adapt to the job. The job must be adapted to AI.Organizations run on unwritten knowledge.
Much of what people learn is never written down. A job description omits relationships, history, local practices, workarounds, judgment, and countless small decisions. Human workers acquire this hidden knowledge by participating.
- Exceptions
- Relationships
- History
- Priorities
- Local practice
- Judgment
One job. Two kinds of onboarding.
A person is hired. They are taught the job. Then they learn the rest by doing it. Experience and correction change how they work tomorrow.
AI is assigned a job. It is given instructions for doing it. When something is missing, people must find the gap and change what the AI is given. The failure does not teach the AI.*
AI can do a great deal without learning on the job.
Knowing where AI does not fit helps us see where it does. Use AI where the task can be defined, the necessary information can be supplied, and the result can be checked. Augment people and systems with AI—do not ask AI to carry the whole human role.
- Defined task
- Supplied information
- Checkable result
- Draft and rewrite
- Summarize and extract
- Compare and critique
- Classify and organize
- Translate and explain
- Apply repeatable steps
- Process work at scale
ExploreWhy small improvements compound
Incrementalism is progress through accumulated change. On-the-job training turns repeated experience into expertise. Continuous-improvement practices such as Kaizen and iterative cycles such as Plan–Do–Study–Act formalize the same basic pattern: observe an outcome, preserve the lesson, and alter the next attempt.
These methods are intentional, but they rely on people carrying understanding from one cycle into the next. Learning persists, so small improvements can become large changes over time.
ExploreWhat must be engineered around AI
AI is only one component of a reliable working system. Depending on the work, that system may need explicit instructions, current source material, stored history, retrieval, tools, permissions, workflow logic, evaluations, monitoring, versioned updates, and human escalation.
The exact machinery varies. The important point is that lessons do not become durable operational improvement merely because AI encountered them. Someone must decide what should carry forward, encode it, test it, and maintain it.
ExploreWhat this claim does—and does not—mean
Here, adaptation means durable change through experience: learning an evolving environment—its goals, circumstances, relationships, exceptions, and ways of working—and carrying those lessons into future work. AI can adjust its output to the context currently supplied, but the experience does not change the model or become a lesson it carries forward.
Humans do not learn perfectly. They misunderstand, forget, resist change, absorb bad habits, and require deliberate training and management. Durable AI adaptation is possible, but it requires deliberately engineered memory, updated instructions, feedback loops, evaluation, training, and continued maintenance. That is a different operating model from a human worker learning through participation.
02 / Common ground
Humans remember the gist. AI must be told—again and again.*
Gist is the meaning that remains after the details fade: what matters, what has been settled, and what can now remain unsaid. People bring that understanding into the next conversation.
Humans forget details. But they retain the big picture—the gist.
Human memory is reconstruction, not playback. It retains some details, loses others, and abstracts the meaning of experience into a gist. Across experiences, those gists build schemas—durable mental guides that preserve the big idea rather than the full record, shaping what people notice, expect, and understand next.*
AI can grasp the details. But it does not retain the gist.*
AI can work across far more supplied detail than a person can hold in mind. It can compare cases, find patterns, expose contradictions, and help people build a better big picture. People can learn from that work and carry the new gist forward. AI does not.
We discussed it.
We reached the right conclusion.
We are now on the same page.
People treat conversation as progress.
Once people establish common ground, they remember what was settled and build on it. The next conversation does not begin from zero.*
AI does not carry that common ground forward.
It may receive a record of what was said. But the record is not the understanding. AI must reconstruct what mattered, what was decided, and why. The same details can lead to a different conclusion.*
You think the conversation is continuing.AI may be rebuilding it.The organization must preserve the common ground.
If AI is expected to continue the work, people must preserve what was settled, why it was settled, which evidence mattered, and when the understanding changes. Otherwise, each interaction can reconstruct a different version of the work.*
- What is settled
- Which conclusions govern
- What evidence supports them
- Whose interpretation prevails
- When the understanding changes
- Which dissent remains
- What should be forgotten
- What must be retained
Shared understanding helps people think together.Shared assumptions can be wrong.
Common ground makes communication compact and collaboration cumulative. But consensus can overemphasize familiar information, suppress minority evidence, and harden after it should be reconsidered.*
AI can challenge shared assumptions.But it can also reinforce them.
AI has no ego, loyalty, or status to defend. But a fresh critic does not appear automatically. We must ask for disagreement without first revealing the answer we want. Otherwise AI may simply reinforce it.*
Preserve the common ground. Reset the critic.
Use continuity to collaborate. Use discontinuity to critique.
- Critique before revealing the preferred answer
- Ask what evidence would falsify the model
- Use fresh context for independent review
- Separate advocate and critic roles
- Preserve minority interpretations
03 / The details
AI’s capacity for detail is an advantage. It is not a substitute for the big picture.*
People compress complexity into a few meaningful ideas. AI can search, compare, restructure, and connect far more supplied detail at once. It expands how much people can examine. People still carry the meaning that makes the detail useful.
Details become abstractions, equations, stories, categories, and plans.
AI can search, compare, restructure, and connect far more detail at once.
04 / The big picture
AI can get every detail right.And still get the big picture wrong.*
People stop repeating details once the big picture is understood. AI does not retain that understanding. Both the details and the gist must be supplied again and again—or AI can solve the wrong problem exceptionally well.
Years of patterns, priorities, exceptions, and consequences become judgment.
AI can analyze, compare, and work through far more detail—but only inside the frame it receives.*
The operating model
Different intelligences
fit different work.
Do not ask whether AI can do a person's job. Ask what each responsibility requires, what common ground must carry forward, and which intelligence should carry the work.
AI competence is jagged. Doing one part brilliantly does not prove it can carry the whole role.*
AI scales the process humans construct—success and error alike. It does not inherit the job or what people know about it. The work must be rebuilt around what AI can do and what people must keep carrying.*
A job is a bundle of responsibilities.
Some can be automated. Some should remain human-led. Many are strongest when performed iteratively by both.
Qualifications & evidence
The asterisks matter.
The visible argument uses ordinary meanings. These notes preserve the technical and philosophical distinctions underneath it.
01What “learning” means here
On this page, learning means changing through experience in a way that persists into the future. Human learning involves many forms of biological plasticity, including changes in synaptic strength, neural connections, circuit activity, and—in some kinds of learning and over some timescales—myelination. “Wet weights” is a metaphor for this physical capacity to change, not a claim that a brain updates exactly like an artificial neural network.
During ordinary inference, context changes an LLM’s temporary computational state, but its learned weights remain fixed. Intentional training—including fine-tuning—can durably modify those weights. Applications can also preserve external memories, but that is different from the model learning through an ordinary interaction.
Most companies invoke a specified version of a frontier model through an API. The model does not learn from each call, even when the surrounding application stores history or retrieves prior interactions.
02Context can produce real temporary adaptation
Researchers call the ability to infer patterns or adopt temporary rules from a prompt in-context learning. It can change what a model does now without creating a durable change carried independently into later sessions. “Context is not learning” uses the durable definition established above; it does not mean that prompting is behaviorally inert.
When that context is no longer supplied, the temporary adaptation is gone. An application may later restore parts of the exchange through stored conversations, summaries, or retrieved memories. That can reconstruct continuity, but it is not the model carrying learning forward from experience.
03Why not simply make AI learn continuously?
Research on adaptive and continually learning systems is active. This page does not claim that more persistent learning is impossible. It argues that most organizations must make decisions using the systems available now—not capabilities they expect later.
A model that changes through use also creates difficult questions: which experiences should become learning, how malicious or false lessons are rejected, how private information is protected, how changing behavior is tested and approved, how failures are reproduced, and how harmful changes are reversed. Technical possibility does not remove the work of safety, governance, and accountability.
04Chunks and context windows are not equivalent units
Human working-memory estimates vary by task, rehearsal, prior knowledge, and how a “chunk” is defined; roughly three to five chunks is a useful teaching range, not a fixed biological constant. An LLM context window is not human working memory. It makes much more supplied information computationally available, but does not guarantee equal attention, correct retrieval, or coherent integration.
05Local accuracy is not global understanding
Humans can also reason from incomplete or mistaken frames; expertise can even harden a bad one. The practical contrast is that an experienced person may carry durable, compressed context from prior work that was never restated, while a model's present inference is conditioned on trained patterns plus the context made available to the current system. Models can sometimes infer missing premises or ask clarifying questions. Neither possibility makes missing context reliably visible. This is not a claim that humans always reason “top-down” and AI always reasons “bottom-up.”
06Helpfulness can become agreement
LLMs can challenge assumptions and produce independent criticism when asked or trained to do so. But systems optimized using human preferences can also mirror a user's expressed beliefs instead of correcting the premise—a behavior researchers call sycophancy. Research has found that human evaluators and preference models sometimes favor convincingly written agreement over a more truthful answer. This is a documented tendency, not a claim that every model always agrees or that providers intentionally optimize for sycophancy.
Independent criticism therefore has to be elicited deliberately. A fresh context can reduce inherited framing, but independence also depends on the instructions and evidence supplied. Asking for falsifying evidence, separating advocate and critic roles, and withholding the preferred answer until after critique can make disagreement more useful.
07Gist abstraction, semantization, and schemas
Human memory is reconstructive, not a literal recording. Research distinguishes richly detailed episodic and perceptual traces from more abstract semantic representations. Across encoding, consolidation, retrieval, and repeated use, memories can become less tied to a specific episode and more organized around conceptual meaning—a broad process often described as gist abstraction or semantization.
A schema is an organized structure of prior knowledge that influences how new information is encoded, interpreted, consolidated, and retrieved. The visible progression from details to gist to schema is a teaching model, not a claim that every memory follows one fixed sequence or that detail always disappears while meaning always improves. Human gist is durable but also selective, reconstructive, and fallible.
08Individual gist is not the same as common ground
Gist abstraction describes what can happen within an individual's memory. Common ground describes information, assumptions, meanings, and conclusions that participants treat as mutually understood. Conversation can create and maintain common ground, but participants never possess perfectly identical internal models. Each person carries a partial, compressed, and sometimes mistaken version of what they believe is shared.
That imperfection does not erase the advantage. Because people retain some conceptual structure from earlier exchanges, later dialogue can rely on shorthand, omit settled background, and begin from a higher starting point. Common ground is therefore both a cognitive achievement and an ongoing coordination problem.
09Experienced continuity and constructed continuity
“Continuity” refers here to the persistence of identity, understanding, relationships, and accumulated history. Human experience feels continuous, although human memory is incomplete and reconstructive and the unity of identity remains a scientific and philosophical question.
AI applications can maintain genuine external state through stored conversations, summaries, databases, and retrieved memories. The model's apparent continuous understanding is reconstructed from those materials during later inference rather than carried forward as an integrated autobiographical experience. This is why continuity remains inside the larger common-ground argument.
10AI memory is engineered reconstruction
“AI forms a context-bound gist” means that a model can build a useful conceptual hierarchy during an interaction without that newly formed hierarchy becoming a durable change in its trained weights. A later interaction may reconstruct it when the application returns the relevant transcript, summary, stored facts, decisions, or retrieved source material.
Long-term AI memory is an active engineering field. Systems can index, compress, retrieve, update, and reason across prior interactions, and research prototypes demonstrate increasingly capable multi-session assistants. But memory design remains consequential: summaries are lossy, retrieval can omit decisive material, stored facts can become stale, and even correctly retrieved items may be interpreted poorly. Durable common ground therefore requires choices about what to record, how to represent meaning and hierarchy, what to retrieve, how to update it, and how to evaluate whether the reconstructed frame is correct.
11Common ground helps groups—and can bias them
Shared knowledge improves coordination, but group discussion often favors information already known by multiple members over unique information held by a minority. Social interaction can also reduce independence, amplify confidence, encourage conformity, and narrow the alternatives a group considers. Related patterns are studied under terms including shared-information bias, group polarization, and groupthink.
These are risks, not universal laws. Diverse knowledge, genuine independence, structured dissent, explicit consideration of alternatives, and well-designed decision processes can improve group judgment. AI can support those practices, but it can also inherit the group's framing or produce agreeable rationalizations.
12Capability, values, and responsibility are different
Pretraining data, preference optimization, written principles, safety specifications, and system instructions all shape model behavior. That does not establish that a model possesses values as internally endorsed commitments. Human values are also shaped by biology, culture, institutions, and other people; neither category should be reduced to a single source.
13The boundary moves; the responsibility to design does not
Some responsibilities—and in bounded cases, entire roles—can be automated. Others remain human-led, and many benefit from an iterative partnership. AI capability is “jagged”: tasks that look similarly difficult to people can fall on opposite sides of what a model performs reliably. Success on one responsibility does not establish competence across an entire role.
“What feels easy to one intelligence may be hard for the other” echoes Moravec's paradox: early AI researchers observed that some forms of formal reasoning were easier to reproduce than perception and sensorimotor abilities humans perform almost automatically. This page uses that as a broad design heuristic, not a prediction that every apparently hard task will be easy for AI or every intuitive human task will remain difficult.
Automation multiplies the behavior of the process around it, including errors. Combining human and machine cognition does not automatically produce a better result. Effective automation and augmentation depend on understanding the actual work, assigning responsibilities well, verifying important outputs, and preserving appropriate judgment and accountability. Future capability should be evaluated when it exists; it should not be treated as a feature of today's system.
Selected sources
- Bailey, Kandel, and Harris — The Cell Biology of Synaptic Plasticity
- Fields — A new mechanism of nervous system plasticity: activity-dependent myelination
- Morra — Modelling Working Memory Capacity: Is the Magical Number Four, Seven, or Does It Depend on What You Are Counting?
- Heinen et al. — Representational formats of human memory traces
- Alonso et al. — Naïve to expert: Considering the role of previous knowledge in memory
- Udagawa and Aizawa — Maintaining Common Ground in Dynamic Environments
- Bang and Frith — Making better decisions in groups
- Garg et al. — What Can Transformers Learn In-Context?
- Liu et al. — Lost in the Middle: How Language Models Use Long Contexts
- Packer et al. — MemGPT: Towards LLMs as Operating Systems
- Wu et al. — LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
- Dell'Acqua et al. — Navigating the Jagged Technological Frontier
- Brynjolfsson, Li, and Raymond — Generative AI at Work
- Becker et al. — Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
- Moravec — Mind Children: The Future of Robot and Human Intelligence
- Ouyang et al. — Training language models to follow instructions with human feedback
- Bai et al. — Constitutional AI: Harmlessness from AI Feedback
- Sharma et al. — Towards Understanding Sycophancy in Language Models