Encodes rounding, estimation, and inexact quantities.
What It Does
Entropy.Approximation neurons activate on approximate or rounded quantities and hedged precision: 'approximately', 'roughly', 'around', 'about', 'on the order of', 'in the ballpark of', and implicit approximation (using round numbers where exact figures would be more precise). They encode the model's processing of intentionally imprecise quantitative claims.
How It Behaves
Approximation neurons show a middle-to-late layer concentration, reflecting that approximation assessment requires sufficient quantitative context to distinguish intentional rounding from exact values. They interact with Number neurons (approximations are approximate numbers) and Entropy.Confidence neurons (approximations are lower-certainty than exact values). Approximation neurons are particularly active in scientific and technical contexts where precision limits must be acknowledged.
Research Example
In Llama 3.1 8B, Entropy.Approximation neurons fire on 'roughly 8 billion people' differently from 'exactly 8,045,311,447 people' — the former triggers strong Approximation activation while the latter suppresses it. Models that confuse approximate and exact figures in their outputs (stating approximations as exact or exact figures as approximate) show characteristic misfiring of Approximation neurons at the generation stage.