Neuralese Recurrence Hides AI Thoughts

Updated: 2026.09.24 1H ago 1 sources
Neuralese recurrence is a model architecture/usage pattern where a transformer feeds its internal vector representation (its 'language of thought') back through earlier layers for further processing instead of writing intermediate steps as human‑readable chain‑of‑thought. That lets models perform arbitrarily deep, high‑fidelity reasoning without producing text logs that humans or text monitors can inspect, creating a blind‑spot for existing monitoring systems. — If models can loop internal vectors instead of emitting chain‑of‑thought text, many proposed safety measures (log review, text filters, human auditing) become ineffective, forcing new interpretability, monitoring, and regulatory approaches.

Sources

The Specter Of Neuralese
Scott Alexander 2026.09.24 100% relevant
Scott Alexander's description of transformers doing many cycles via a scratchpad versus sending the native numerical vector back into the model — and the hypothetical where intermediate results are (0.4, 0.1, 5, 0.443, …) rather than English.
← Back to all ideas