Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 25·8 min read
Do Language Models Know What Not to Say? Causal Evidence for Statistical Preemption in LLMs
TL;DR
Language models learn to avoid grammatically possible but unattested phrases (e.g., "donated the library the books") without explicit negative feedback. Researchers used causal experiments across 120 English verb-construction pairs to test whether this learning occurs through statistical preemption—exposure to conventional forms suppresses alternatives—versus entrenchment.
✦ Why It Matters
Engineers can use competing-form frequency as a causal lever to control what language models avoid generating, improving safety and linguistic accuracy.
Key Takeaways
- Explore statistical preemption in LLM training datasets.
- Focus on competing-form frequencies to improve language understanding.
- Utilize findings to enhance LLM fine-tuning processes.
- Validate LLM outputs against human judgments for better accuracy.
How It Works
Statistical preemption occurs when exposure to a conventional phrase, like 'donated the books to the library', makes less common alternatives, such as '*donated the library the books', less likely to be used. This mechanism allows LLMs to learn what constructions are acceptable based on the frequency of competing forms rather than relying solely on positive examples.