Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·Aug 18·5 min read
Broken Symmetry in LLM Refusal: Answer Release Is More Local Than Refusal Restoration
TL;DR
Research reveals that when a language model (LLM) refuses to answer a prompt, the correct answer is not erased but can be recovered locally. This 'broken symmetry' indicates that refusal is a more complex process than simply toggling a switch between answering and refusing.
✦ Why It Matters
Evaluate your model's refusal mechanisms to ensure accurate behavioral control and safety measures.
Key Takeaways
- Investigate the internal states of your LLM for better understanding of refusals.
- Use localized adjustments for releasing withheld answers in LLMs.
- Be cautious of overestimating control based on average displacement vectors.
- Consider broader interventions when managing refusal outputs.
How It Works
The study employs a controlled withhold setting to analyze the internal mechanisms of LLMs during refusal. It reveals that while the correct answer can be recovered with a single-position patch, suppressing an answer requires more extensive changes across multiple positions, indicating a complex relationship between answer generation and refusal.