Adjust for rater and source bias

Updated: 2026.11.11 18H ago 1 sources
Crowdsourced drug ratings (from forums or informal surveys) are systematically distorted by who posts and where they posted. Controlling for per‑user rating tendency and source‑specific effects (via mixed‑effects models) changes the apparent ranking of perceived efficacy and helps separate expectation/placebo signals from consistent drug effects. — If policy, clinicians or harm‑reduction groups rely on community ratings, they must correct for rater/source bias or else misidentify harmless supplements as effective and miss safety risks like dependence and withdrawal.

Sources

Statistical analysis of user ratings of nootropics
Sebastian Jensen 2026.11.11 100% relevant
The author pooled Scott Alexander’s 2016/2020 surveys with troof/nootroflix (39,944 ratings from 3,681 individuals) and implemented a mixed‑effects model to control for user and source bias, revealing noisy signals and highlighting withdrawal profiles.
← Back to all ideas