Blog

Truth Directions: Signal-to-Noise and Geometry of Recoverability

Treating the recovery of a linear truth direction as a signal-to-noise problem: when is such a direction recoverable at all from a language model's activations, and what does the estimator return when it is not?

SARSA vs. Q-learning on the cliff revisited

I treat the textbook cliff-walking comparison of SARSA and Q-learning as a controlled experiment — randomness isolated into named streams, the two algorithms paired with common random numbers — and separate behavior return from greedy-evaluation return.

Emergence as metric composition in LLMs (Part II): the training axis

Part II of two. I hold the model fixed and watch the per-token error rate, and the benchmark built from it, evolve with the number of tokens seen during training.

Emergence as metric composition in LLMs (Part I): the parameter axis

Part I of two. Borrowing the finite-size-scaling methodology from statistical physics, I ask whether emergent abilities postulated to appear in larger models are a measurement artifact across eight Pythia sizes. Specifically, can a benchmark metric's sharpness be explained entirely from composing a smooth per-token accuracy through a hard metric or are there other signatures of emergence hidden in the per-token accuracy?

The Grokking phase diagram from a single layer transformer learning modular addition

I study the (capacity, data-fraction) plane of grokking on modular addition, and ask which rescaling of the data axis collapses the onset across moduli — a finite-size-scaling look at the right control variable.

What makes a transformer use both of its layers? Circuit formation in Dyck-(k, m)

I study two-layer transformers learning factored circuits versus single-layer shortcuts and use activation patching to find the signature of the readout head not present in the attention patterns.