Pinned · published

Emergent Models

A machine-learning model made of a simple dynamical system: a state, a local update rule, a halting condition and a fixed way to write inputs and read outputs. Training searches for the initial state, the program, that makes the system compute the task.

Summary

An Emergent Model (EM) is a dynamical system, such as a cellular automaton, with an encoder and a decoder. Prediction: write the input into the state, iterate a fixed local rule until a halting condition holds, read the output.
The learnable object is mainly the initial state, called the program. The rule and parts of the interface can be learned too. Training uses evolutionary search, because the dynamics are recursive, often discrete and not differentiable.
Latent universality: some EMs, with rule and interface fixed, realise every partial computable function by changing only the program. This is an existence result; it says nothing about how hard the program is to find.
The experiments use deliberately minimal models, from tens to a few hundred parameters. The clearest result: a 140-bit cellular automaton trained on x ≤ 30 computes x + 1, 2x, x mod 4, round(x/3) and x·(1 + x mod 2) exactly on every input tested up to 1000.
The paper presents the framework as foundational. It does not propose a competitive architecture, and several tasks were not learned (see Experiments).

Sources: G. Bocchese, N. Giacobbo, E. Guichard, J. Wiles, A. Devireddy, Emergent Models: Intelligence from Tiny Substrates, arXiv:2608.14019, 2026; and Emergent Models: Machine Learning from Cellular Automata, ALIFE 2026, MIT Press. Code and live demos: emergentcomputing.github.io/em-paper.

The model

An Emergent Model is a tuple M = (S, f, H, E, D, ⊕): a state space S over a graph of cells, a transition function f (usually local), a halting operator H, an encoder E, a decoder D, and an operator ⊕ that combines the program with the encoded input.

Inference is y = D(f^T(p ⊕ E(x))), where T is the first step at which H fires. If it never fires, the output is undefined. The state provides memory; repeated application of f provides computation time.

Parameters fall into two groups. Soft parameters are the program: they live in the state and can change during inference. Hard parameters (rule, halting condition, interface) play the role of hardware: optimised during training, fixed during inference.

For streams of inputs, such as a control loop, a retention operator R turns the final state of one prediction into the program of the next: p_i+1 = R(s_T,i). The input stream can then change the computation itself, which is how the adaptive experiments below work.

Why iteration

The relevant notion of capacity is global expressivity: which functions a model family can represent over their whole, possibly unbounded domain, not only on a compact region. A model that represents the generating rule extrapolates by construction; a local fit does not.

Feed-forward networks are limited in this sense. Universal approximation holds on compact domains; outside them, sigmoidal networks tend to constants and ReLU networks become linear, so functions like sin x, eˣ or x mod 4 cannot be represented globally. Iterated systems with unbounded memory and time can be Turing-complete, which is the maximal notion of global expressivity in the discrete setting. Universality says nothing about trainability, but it removes a hard ceiling.

Fixed depth against iteration. Red: a sigmoid MLP (1–32–32–1) trained on sin x for 0 ≤ x ≤ 4π. Outside the training range it follows no rule. Blue: a looped model. Dashed: sin x.

Latent universality

Theorem (latent universality). There exists an Emergent Model with a fixed rule, a fixed halting condition and a fixed interface on binary strings such that every partial computable function g : {0,1}* ⇀ {0,1}* is computed by some finite program state p.

The proof uses a universal Turing machine whose program and input occupy separate regions of the tape, so the initial state factors as p ⊕ E(x) and the encoder never sees the program. A lemma extends the result to Conway's Game of Life through Rendell's universal construction.

The split matters for learning. The fixed interface defines only the syntax: where data is written and read. The program carries the semantics: which function is computed. Because the target function is unknown in advance, everything task-specific must live in the program, and search can vary programs without touching the interface. Classical universality is a property of the substrate alone and may need a different encoding for each algorithm; latent universality is a property of the whole model with one interface.

Searchability. Programs are finite strings, so enumeration or random sampling eventually finds one that computes any given function. This is existence only: the search can be arbitrarily long. The experiments do not realise latent universality and none of their claims depend on the theorem. Concrete fixed-interface machines are the subject of Latent-universal machines.

Experiments

A zoo of minimal instances, each trained with population-based genetic algorithms. Results are representative runs, not multi-seed statistics.

Model	Substrate	Searched	Task	Result
EM43	1D cellular automaton, 4 values, radius 1	rule (60) + program (10): 140 bits	x + 1, 2x, x mod 4, round(x/3), x·(1 + x mod 2)	Trained on [0, 30]: 100% exact on [30, 1000], every run. Two-input a·b not learned (< 10% test).
GoL-EM	Game of Life, rule fixed	positions of 2 gliders	x + k, k ∈ {−2, −1, 1, 2, 5}	Trained on [0, 10]: 100% exact on [10, 200]. Parity-conditional x ± 1 not learned (50% test). Random search about as good as the GA.
GoL-EM	Game of Life, rule fixed	gliders + sparse cells	CartPole (max 500)	Return 317 ± 122 (random ≈ 20). Episodes mostly end with the cart leaving the track.
CEM1D	continuous 1D lattice, 19 cells	3 rule coefficients + 19 initial values	CartPole (max 800)	Return 735 ± 107 over 1000 new episodes. Uses pole angle well, cart position poorly.
CEM2D	8 × 8 field, fast + slow channel	35 rule and interface + 128 state parameters	Meta-Life: food/poison phases	Meta-score 0.71 (0 no adaptation, 1 perfect). A memory RNN baseline reached 0.82.

EM43 and geometric grokking

EM43 is a one-dimensional cellular automaton with four cell values and radius 1. The rule is a lookup table on (left, centre, right); four entries are fixed so that empty space stays empty and an isolated red cell stays where it is, leaving 60 learned entries. The program is 10 cells to the left of the origin. Input: one red cell x cells right of the origin. Halting: blue cells are at least half of the non-blank cells. Output: the distance of the rightmost red cell from the origin.

In runs that generalise, the computation is visible as a regular space-time geometry that keeps its shape as the input grows. For x mod 4, signals from the program region shift the red marker left by four cells, repeatedly, until blue cells become the majority and the machine halts with the marker at the answer. The paper calls this geometric grokking. Whether it persists on harder tasks, or is partly an artefact of these tasks and this interface, is open.

task x

EM43 explorer. Choose a task and an input up to 150. Time runs downward; the input is the red cell x cells right of the green origin; the output is the position of the rightmost red cell when blue cells become the majority. All five models were trained on x ≤ 30.

x mod 4, x = 15 → 3

x = 30 → 2

x = 70 → 2 (outside the training range)

The same learned x mod 4 model at three input sizes: the mechanism keeps its shape. Scales differ between panels.

Game of Life

With Conway's rule fixed, only the program can be searched: here, the positions of two gliders in a 16 × 16 region. The input is a 2 × 2 block on the diagonal, 2x cells from the origin; the run stops when the board stops changing; the output is the leftmost isolated block, read with the same stride.

The x + k tasks were learned exactly, but Game of Life was hard to train. Small changes of the program change the behaviour completely, so similar programs have unrelated fitness and random search performs about as well as the genetic algorithm. The interface, based on isolated blocks, may also be too poor to reach the computational richness that makes Game of Life universal.

GoL-EM, x + 2. The learned two-glider program (top left) run to its fixed point for x = 2, 5 and 9. Grid: one square per integer along the diagonal. Initial state from the paper's figure; the run reproduces the paper's trace.

Continuous substrates

CEM1D: a continuous lattice for control

Nineteen real-valued cells with the update s_j ← s_j + 0.1 (a s_j−1 + b s_j + c s_j+1 − γ s_j³), reflecting boundaries. Only the three coefficients and the 19-cell initial state are trained; the learned values are a = 0.796, b = −0.319, c = −1.628 with γ = 0.03. The four CartPole observations overwrite four cells; the sign of the centre cell pushes the cart left or right. The lattice runs 30 steps per control step and its state carries over to the next one.

Result: 735 ± 107 out of 800 over 1000 new episodes (≈ 500 counts as solved). The learned rule hosts localised structures that carry values through the lattice. Leftward transport is stronger, so the angle inputs, right of the output cell, are used well and the cart position, left of it, less.

Transport. Two impulse pairs under the learned rule, no inputs: opposite signs travel left, equal signs travel right.

Ports. Random values written at the four input ports every 30 steps (vertical lines). Time runs left to right, cells top to bottom; red positive, blue negative.

CEM2D: a fast and a slow field for online adaptation

Meta-Life is a two-dimensional world with resources. An agent sees them through three rays and outputs a turn rate and a speed. In the adaptive version, resources switch between food (+1) and poison (−1) in phases of random length, and both look the same: the agent has to taste a resource, remember the phase, then seek or avoid. Score: meta = (G − B)/(G + B + ε), with G food and B poison collected.

The controller is an 8 × 8 field with a fast channel c and a slow, gated memory channel m. Ten rule coefficients are shared by all cells; the locations and scales of six ports (four inputs including the last reward, two outputs) and the number of update steps per control step are also learned, 35 parameters in all. The 128-value initial state is inherited, not mutated: each step starts from the previous final state.

Learned strategy. Fast field after food and after poison, values read from the paper's figure. Outlined: input ports (I3 is the reward) and outputs (green). Circled: the cell that stores the phase. It is high after food, when the agent seeks resources, and low after poison, when it turns in place and avoids them.

	CEM2D	memory RNN
rule and interface parameters	35	117
initial state parameters	128	12
food only: return	133	146
food or poison: meta-score	0.714	0.822
meta-score, > 6 steps after a contact	0.699	0.796

Limits. CartPole is solvable by linear feedback, and the Meta-Life tasks may be solvable by near-linear policies on two time scales. Harder environments are needed before these results say anything about advantages over conventional controllers.

Open questions

Can a rule be co-evolved across many tasks so that, once frozen, new tasks are solved by changing only the program?
Which search methods work on landscapes where similar programs have unrelated fitness, as in Game of Life?
Does the ratio of parameters to computation time, q = N_params / T̄, govern a trade-off between generalisation and trainability? (See Iteration and reasoning.)
Is geometric grokking measurable, for example by the compressibility of space-time diagrams, and does it predict extrapolation?
Does inherited state in adaptive controllers carry function, or do the hard parameters merely co-adapt to it?

Cite

@inproceedings{bocchese2026alife,
  title     = {Emergent Models: Machine Learning from Cellular Automata},
  author    = {Bocchese, Giacomo and Giacobbo, Nicola and Guichard, Etienne
               and Wiles, James and Devireddy, Akshaj},
  booktitle = {Proceedings of the Artificial Life Conference 2026 (ALIFE 2026)},
  publisher = {MIT Press},
  year      = {2026},
  doi       = {10.1162/ISAL.a.953}
}

@misc{bocchese2026emergent,
  title         = {Emergent Models: Intelligence from Tiny Substrates},
  author        = {Bocchese, Giacomo and Giacobbo, Nicola and Guichard, Etienne
                   and Wiles, James and Devireddy, Akshaj},
  year          = {2026},
  eprint        = {2608.14019},
  archivePrefix = {arXiv},
  primaryClass  = {cs.NE},
  url           = {https://arxiv.org/abs/2608.14019}
}