Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·Jun 23·5 min read
AOR-Bench: Do Large Audio Language Models Over-Refuse Pseudo-Harmful Queries?
TL;DR
Large audio language models often refuse to respond to queries that may seem harmful, but this can limit their usefulness. AOR-Bench is a benchmark designed to evaluate how these models handle pseudo-harmful queries, which are not genuinely harmful but may appear so.
✦ Why It Matters
Engineers can refine audio language models to improve their responsiveness while maintaining safety standards.
Key Takeaways
- Evaluate your models using AOR-Bench for better safety assessments.
- Identify and address over-refusal patterns in model responses.
- Balance safety and utility in audio language model design.
- Enhance user experience by refining model training processes.
Full Summary
Audio language models, which process and generate audio data based on textual input, can sometimes be overly cautious in their responses to potentially harmful queries. AOR-Bench was developed to systematically assess how these models respond to pseudo-harmful queries—questions that are not truly harmful but could be misinterpreted as such.
The methodology involved testing various large audio language models against a set of predefined pseudo-harmful queries and measuring their refusal rates. Results showed that many models exhibited high refusal rates, suggesting they prioritize safety over providing useful information.
This over-refusal can hinder user experience and limit the models' practical applications. The findings highlight the need for improved training and tuning of these models to strike a better balance between safety and responsiveness.
Engineers and researchers can leverage these insights to enhance model design and user interaction.