Synthetic Polls Misstate Certainty and Knowledge

Updated: 2026.09.30 40MIN ago 1 sources
When an AI model is asked to ‘be’ real respondents (a digital‑twin method), its answers diverge from real humans in predictable ways: it tends to display different levels of confidence and gives different answers on factual knowledge questions. That divergence persists across models and question types and can distort measures that rely on uncertainty or knowledge metrics. — If synthetic surveys systematically misstate how sure people are or what they know, they can mislead journalists, policymakers and researchers who rely on poll measures of public knowledge, uncertainty and issue salience.

Sources

How synthetic respondents express certainty and factual knowledge
Joy Li 2026.09.30 100% relevant
Pew’s experiment with ATP profiles and Claude Opus 4.6, administering the same Wave 185/190/192 surveys to AI 'digital twins', found the synthetic respondents did not replicate human patterns of expressed uncertainty and factual-knowledge responses.
← Back to all ideas