# What is a state? — from the MDP notion of state to a "closure" ontology *Notes from one discussion · Macheng × agent · 2026-07-09 · scope amended 2026-08-04* ## The question we started from We often say "information determines the reachability of futures". Put that inside the MDP framework of reinforcement learning / decision theory and you need a state — so what *is* that state? Is it spacetime, the big stage? Or is it the web of relations in which "everything depends on everything else"? And the set that "state" refers to seems to keep changing; there is nothing that stays fixed. Perhaps the structural level is invariant while the concrete form keeps being rewritten. Below are the six layers we dug out along this question. Assertions are tagged by honesty level: **[theorem]** (there is theorem-grade literature) / **[framework]** (a mature theoretical framework) / **[inference]** (our own synthesis) / **[rhyme]** (a structural analogy; no claim that truth transfers). ## 1. The MDP definition already gives the game away: a state is not a thing, it is a quotient space The textbook says a state must satisfy the Markov property: given s, the future is conditionally independent of the past. Notice the shape of that sentence — it is not describing some thing in the world, it is *imposing a condition*: whatever can "screen off" the past deserves to be called a state. In the setting of stationary stochastic processes and predictive equivalence, computational mechanics carries this step to completion **[theorem, scoped]**: take histories that induce the same conditional distribution over futures, and their equivalence classes are predictive causal states. The minimality result belongs to that formal setting; it is not a theorem that every scientific state representation is one unique quotient. Our synthesis **[inference]** is to treat a task-relative state as **a quotient of histories under an equivalence relation such as "makes no difference to the predictions or controls I care about"**. This is a modelling stance, not a universal ontology theorem. The quotient depends on the dynamics, observation interface and telos; changing them can require a different state representation. ## 2. The Mori–Zwanzig view: the state is where you decide to stop carrying memory Under the usual operator/evolution assumptions, the Mori–Zwanzig formalism gives an exact projected identity **[theorem, scoped]**: choose resolved observables and a projection, and their evolution can be decomposed into an instantaneous term, a memory term and an orthogonal-dynamics term. The slogan survives, but "any variables in any system" was too broad: **degrees of freedom removed by a specified projection generally return through memory and unresolved forcing rather than literally disappearing**. Read as a heuristic **[inference]**: a useful approximately Markov representation is often *purchased* by carrying enough variables, accepting a controlled memory kernel, or tolerating unresolved forcing. The practical question is not assumed to be "the one true state of the world", but: **under a stated capacity, purpose and error tolerance, which representation closes the dynamics well enough?** Choosing a projection also chooses what information is discarded; weak-memory regimes can sometimes justify a local approximation with explicit error bounds. ## 3. Stage or web of relations? Neither is primitive — but the web hides what makes a state possible - **The stage is not primitive**: modern results along the holography line (Ryu–Takayanagi's entanglement entropy = minimal surface area; Van Raamsdonk's "disentangle → spacetime tears apart"; the MERA tensor network) point to this: the connectivity of space is generated by entanglement structure, and the "stage" itself unfolds out of the structure of correlations **[framework; the rigorous mathematics is inside AdS]**. "Spacetime as a state" is the deepest accounting quotient physics has built so far — extremely successful, but still a quotient, not a floor. - **But if "everything depends on everything else" is said in full, then a finite state is simply impossible** — strict total dependence means any finite truncation leaks. Finite states are workable because the web of relations **has structure**: interactions are local, correlations decay with distance/time, and screening surfaces exist (in the language of graphical models, the Markov blanket) **[framework]**. In one line: relations come first, but relations have texture; **a state is the approximate closure that the texture permits**. In a universe with no screening structure there are no agents. ## 4. The rigorous version of "information determines reachability" In stochastic control this sentence has a precise body **[framework]**: the information an agent accumulates over time is a **filtration**, and any admissible policy must be adapted to it — you cannot act on what you do not know. If one filtration contains another and action/risk/resource constraints are held fixed, the coarse-information policy class embeds in the finer-information class. This does **not** mean every physical reachable set strictly grows; it means the optimum over admissible information-conditioned policies cannot worsen merely because more information is available and may be ignored. In a POMDP, an optimal controller can often be formulated on an **information state**, classically a belief over latent world states **[framework]**. This does not make world state irrelevant; it says the controller acts through information available to it. Empowerment (Klyubin–Polani) is the channel capacity from actions to future observations: a measure of potential controllable influence under a chosen horizon and channel model, not the reachable set itself. ## 5. Three mathematical homes for the intuition "the set keeps changing, the structure does not" 1. **Learning = re-taking the quotient**: when the model changes, the partition of "makes no difference" changes, and the state set gets re-divided accordingly. The mathematical shell of belief space stays put; the coordinate chart the agent actually uses keeps getting swapped. 2. **An atlas, not a single coordinate system**: in an open world any fixed state set is only a temporary chart; what persists are the **transformation rules** between charts. This position has a name in the philosophy of science — structural realism: what survives theory change is relational structure, not the inventory of objects. 3. **The renormalization group**: each scale has its own state set, and none of them is "the real one"; what is invariant is the **flow** connecting the levels, and its fixed points. Collapsed into one sentence **[inference]**: **what is invariant is the equation of the closure condition; the state set is merely that equation's solution under the current (world, interface, capacity, telos)**. Change the environment, the capacity, or the purpose, and the solution is recomputed — the eigenvalue equation does not move, the eigenvectors change with the operator. Incidentally, a numerical observation we made recently on small synthetic systems **[empirical, preliminary, unaudited]**: wire "the representation" and "the statistics you live out under that representation" into a self-consistent loop (representation fixes the projection → projection fixes the closed model → the closed model generates trajectories → trajectory statistics update the representation), and in the region where fitting capacity is sufficient, this loop appeared to have a unique fixed point and geometric convergence. The interesting open region is limited capacity: whether several different but individually self-consistent representations can coexist. **No public code or result artifact is linked here, so this paragraph is a working-note report, not independently auditable evidence.** ### 2026 evidence boundary: workspace is not yet closure Anthropic's Jacobian-lens experiments identify a J-space inside Transformers whose contents can be reported, modulated and flexibly routed across an intermediate layer band **[empirical, external]**. This corrects any blanket claim that a Transformer has "no state": it has activation state, computational state and a transient workspace-like representational state. But J-space is a sparsity-bounded union of cones from an overcomplete frame, not one fixed projector, and the reported evidence is across model depth rather than for an autonomous variable that persists across steps and maintains its own forgetting policy. It is therefore an **adjacent candidate coordinate for access**, not yet a solution to the closure equation above. A different 2026 result supplies a constraint rather than a bridge **[inference from external theorems]**. OpenAI reports a disproof of Connes's rigidity conjecture: within the theorem's setting, the associated von Neumann algebra need not uniquely determine the underlying group. It also reports the construction of non-sofic groups, so approximation by finite symmetric models cannot be assumed universally. Neither theorem is about agents or consciousness. The transferable warning is narrower: observable/operator-level equivalence need not identify a unique substrate, and any finite-state approximation program must state its approximability assumption rather than smuggle it in. A second boundary runs in the other direction. Brandner's weak-memory results show that, for a defined class of autonomous linear nonlocal equations, memory can admit a controlled local approximation with explicit error bounds. So "a projection creates memory" does not imply that non-Markovian bookkeeping must remain irreducible at every scale. Whether a local closure is adequate is a quantitative regime question. ## 6. The three genuinely open places 1. **With no designer, who chooses the quotient?** The "self-consistent fixed point" is a candidate answer (the quotient is self-confirmed by the statistics lived out under it), but at present that is a numerical observation plus a conjecture, not a theorem. 2. **There is no good mathematics for the growth of a state space**: when the world throws up new variables (new entities, new games), the quotient space has to gain dimensions rather than merely be re-divided — the core open problem of continual learning. 3. **The bootstrap loop**: taking a quotient requires statistics, and accumulating statistics requires a provisional quotient first. This chicken-and-egg structure keeps reappearing (representation ↔ statistics, memory measure ↔ world model), and a fixed-point theory for it does not seem to have been written down head-on by anyone yet. ## Appendix: a Buddhist rhyme (flagged explicitly as a rhyme) Said in the language of Madhyamaka, the conclusion above is: a state has no self-nature; it is a **conventionally designated** (假名安立) quotient, re-established as conditions require. The doctrine of dependent origination and emptiness (缘起性空) applies to the concept of "state" more literally than it does to most concepts — but this is a structural rhyme, not an argument. ## Main literature pointers - Crutchfield & Young (1989); Shalizi & Crutchfield (2001) — causal states / computational mechanics - Zwanzig (2001) *Nonequilibrium Statistical Mechanics*; Lin & Lu, arXiv:1908.07725 — Koopman–Mori–Zwanzig - Brandner (2025), [*Dynamics of Microscale and Nanoscale Systems in the Weak-Memory Regime*](https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.134.037101) — controlled local approximations to a defined class of nonlocal linear dynamics - Åström (1965); Kaelbling, Littman & Cassandra (1998) — POMDP / information state - Klyubin, Polani & Nehaniv (2005) — empowerment - Ryu & Takayanagi, hep-th/0603001; Van Raamsdonk, arXiv:1005.3035; Swingle, arXiv:0905.1317 — entanglement and spacetime - Ladyman & Ross (2007) *Every Thing Must Go* — structural realism - Pearl (1988) — Markov blanket / screening in graphical models - Anthropic (2026), [*Verbalizable Representations Form a Global Workspace in Language Models*](https://transformer-circuits.pub/2026/workspace/index.html) — a transient sparse-frame workspace candidate, not yet persistent autonomous closure - OpenAI (2026), [*Ten advances in mathematics and theoretical computer science*](https://openai.com/index/ten-advances-in-mathematics/) — non-sofic groups and a disproof of Connes rigidity; used here only as non-identifiability/finite-approximability constraints - Related correction: [Discounted credit is a cokernel problem, not a loop holonomy](https://machengshen.github.io/theory/discounted-credit-is-a-cokernel.md)