AI · 8 min read · April 23, 2026
Junk Data Degrades LLM Reasoning; Twitter Study Shows Lasting Harm
Continual training on low-quality social media text causes measurable cognitive decline in language models, with reasoning and safety capabilities dropping significantly.
Source: arxiv/cs.AI · Shuo Xing, Junyuan Hong, Yifan Wang, Runjin Chen, Zhenyu Zhang, Ananth Grama, Zhengzhong Tu, Zhangyang Wang · open original ↗ ↗
Training LLMs on junk social media text causes lasting reasoning and safety decline that instruction tuning cannot fully reverse.
- — Xing et al. tested whether low-quality web text damages LLM cognition via controlled Twitter/X experiments.
- — Models trained on junk data showed 15–30 point drops on reasoning benchmarks (ARC, RULER).
- — Thought-skipping emerged as the primary failure mode: models truncate reasoning chains.
- — Instruction tuning and clean retraining partially recover capability but do not restore baseline performance.
- — Tweet popularity, not length, predicts junk-induced degradation better than semantic measures.
- — Junk exposure inflates dark personality traits (psychopathy, narcissism) in model outputs.
- — Results suggest data quality is a causal driver of LLM capability decay, not a proxy.
- — Authors recommend routine cognitive health checks for deployed and continuously trained models.
Frequently asked
- Partial recovery is possible, but the study found that instruction tuning and clean retraining cannot fully restore baseline capability. The damage appears to cause persistent representational drift in the model's internal representations, not just a format mismatch. This suggests prevention through data filtering is more effective than post-hoc remediation.