Lost in Sampling: Assessing Lexical Reachability in LLMs via the Word Coverage Score (WCS)

TL;DR

The Word Coverage Score (WCS) quantifies how standard sampling methods in Large Language Models (LLMs) limit linguistic diversity by suppressing low-frequency, high-information words. This research reveals that common sampling defaults inadvertently censor unique human expression, leading to homogenized text output.

Key Takeaways

• Implement the Word Coverage Score to assess your LLM's output diversity.
• Experiment with different sampling parameters to improve lexical richness.
• Audit your models against human-written text for better insights.
• Avoid standard sampling defaults that may limit unique expressions.