Tsukada Lab. Computer Network Group

ECHO: Emergent Common-Mode Hidden Offsets Drive Vocabulary Collapse in Preference Optimization

Yun Li, Ehsan Javanmardi, Simon Thompson, Jiachen Gong, Qunli Zhang, Zifan Zeng, Shiming Liu, Peng Wang, Zixuan Guo, Manabu Tsukada

, Oct 21, 2026

Summary in three lines

  • Preference training can improve ranking accuracy while generation degenerates into repetition.
  • ECHO traces this failure to a shared hidden-state offset that concentrates output probability on a single token.
  • Interventions that center or penalize the offset reduce collapse while preserving strong held-out ranking performance.

Figures from the paper

Figure 1: Hidden-state offsets and vocabulary collapse
Figure 1. Hidden-state offsets and vocabulary collapseSource: Figure 1 from Li et al., arXiv (2026) (authors’ arXiv version 2610.29410, CC BY 4.0)The panels link stable ranking metrics to a shift in hidden states and the concentration of output probability on a single token.

How to cite

Y. Li, E. Javanmardi, S. Thompson, J. Gong, Q. Zhang, Z. Zeng, S. Liu, P. Wang, Z. Guo, and M. Tsukada, “ECHO: Emergent Common-Mode Hidden Offsets Drive Vocabulary Collapse in Preference Optimization,” in , 2026.

BibTeX

@unpublished{Li2026h,
  title         = {ECHO: Emergent Common-Mode Hidden Offsets Drive Vocabulary Collapse in Preference Optimization},
  author        = {Yun Li and Ehsan Javanmardi and Simon Thompson and Jiachen Gong and Qunli Zhang and Zifan Zeng and Shiming Liu and Peng Wang and Zixuan Guo and Manabu Tsukada},
  year          = {2026},
  date          = {2026-10-21},
  eprint        = {2610.29410},
  archiveprefix = {arXiv},
  primaryclass  = {cs.CL},
  url           = {https://arxiv.org/abs/2610.29410},
  note          = {Preprint}
}