On‑the‑fly Context Compression

Inference‑time continual learning (test‑time training) compresses very long context into model weights while a model reads, giving constant latency as context length grows and improving long‑document understanding without full attention. It trades exact needle‑recall for scalable quality and can be meta‑trained so small on‑the‑fly updates reliably improve performance. — If productionized, this approach changes who can run long‑context AI (devices, lower‑cost infra), shifts privacy/design tradeoffs (models learn from session text), and affects regulatory questions about retention, provenance and hallucination risk.