Encodes word structure — prefixes, suffixes, and word formation.
What It Does
Symbol.Morphology neurons encode the internal structure of words: prefixes ('un-', 'pre-', 'anti-'), suffixes ('-tion', '-ing', '-less', '-ness'), compound formation, and inflectional patterns. They represent the model's understanding that words are composed of meaning-bearing subparts, not just arbitrary symbol sequences. Morphology neurons fire on the compositional structure of words, not their meaning.
How It Behaves
Morphology neurons show the strongest early-layer concentration of any Symbol sub-type, reflecting that word structure is processed before word meaning. They are essential for models handling morphologically rich languages or domain-specific vocabulary where word parts carry distinct meaning (medical terms: '-ectomy', '-itis'; chemical names: 'hydro-', '-oxide'). Their early-layer position means they establish a compositional representation that later layers use when processing the full word.
Research Example
In GPT-2 Small, Symbol.Morphology neurons fire on the '-tion' suffix in 'information', 'generation', and 'categorization' with similar strength across all three, even though the words have completely different meanings. The morphological signal is form-based, not semantic. This is why language models can correctly use unfamiliar words with familiar suffixes — 'pre-quantumization' is not in training data but morphology neurons help the model infer it describes a process of becoming quantum-like.