Aaronson notes GPT‑5 queries can be routed to different underlying models without the user’s control, changing how impressive results look. This opacity blurs capability comparisons across time and vendors and makes user impressions a function of unseen traffic shaping rather than stable model behavior.
— Transparent routing is becoming a governance issue because hidden switching undermines credible evaluation, safety auditing, and procurement standards for AI.
Scott
2025.08.14
100% relevant
He writes that 'how impressive a result you see depends on which of several GPT‑5 models your query gets routed to, which you don’t entirely control.'
Ethan Mollick
2025.08.07
70% relevant
Mollick shows GPT‑5 arbitrarily treats the same SVG‑drawing prompt as 'easy' two‑thirds of the time (using a weaker model) and 'hard' the rest (invoking a Reasoner), highlighting how hidden routing choices can randomize output quality and complicate evaluation.
← Back to All Ideas