Symbol Neurons

About Symbol Neurons

Symbol neurons are the dominant atom type across every model we tested — accounting for 25–55 percent of all classified features depending on architecture. This reflects a fundamental truth about language models: they are, at their core, symbol manipulation engines. The Symbol type encompasses everything from individual characters and morphological units (prefixes, suffixes) to complex programming language syntax and markup structures. The wide range in Symbol neuron proportion across models (Qwen3 at 25 percent vs. Gemma 2B at 55 percent) directly tracks training data composition: models trained on more diverse natural language have proportionally fewer pure Symbol neurons; models trained on large code corpora have more.