Tiny Transformers Learn Real Algorithms

Multiple recent experiments show extremely small transformers (hundreds of parameters) can learn to perform long addition on fresh test data, with information‑theoretic checks ruling out memorization. That suggests model architectures can discover compact algorithmic representations, not just statistical associations. — If transformers can internalize algorithms at tiny scale, capability forecasts, interpretability research, safety timelines, and the economics of on‑device AI all need revising.