Encodes event ordering — before, after, then, next, finally.
What It Does
Time.Sequence neurons activate on temporal ordering relationships: explicit sequence markers ('first', 'then', 'after', 'subsequently', 'finally'), causal chains with temporal order ('because X happened before Y'), narrative progression ('the next step is', 'the following day'), and procedural sequences ('step 1: ..., step 2: ...'). They encode the ordering of events relative to each other.
How It Behaves
Sequence neurons are broadly distributed across all layers, reflecting that temporal ordering is a pervasive property of language — almost every multi-event description involves sequence. They interact with Relation.Causal neurons (causation implies sequence) and Number.Ordinal neurons (ordinal numbers encode sequence). Sequence neurons are particularly critical for models handling procedural instructions, narrative comprehension, or multi-step reasoning tasks.
Research Example
In Gemma 2B, Time.Sequence neurons activate on recipe instructions, programming tutorials, and historical narratives with similar patterns — the temporal ordering structure is encoded by the same neurons regardless of whether the sequence is culinary, technical, or historical. This domain-general sequence representation is why language models can follow procedural instructions across diverse domains.