Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·29d ago·5 min read
A Retrieval-Augmented Framework for Detecting and Resolving Pragmatic Ambiguities in Natural Language Requirements
TL;DR
Natural language requirements often contain pragmatic ambiguities that can hinder software development. A retrieval-augmented framework was developed to detect and resolve these ambiguities effectively.
✦ Why It Matters
Engineers can implement this framework to enhance requirement clarity and reduce project risks associated with misunderstandings.
Key Takeaways
- Utilize retrieval-augmented generation for requirement disambiguation.
- Incorporate expert knowledge bases to improve interpretation accuracy.
- Evaluate ambiguity detection using established metrics like F1 and F2 scores.
- Engage requirements analysts for final validation of disambiguated requirements.
How It Works
The framework simulates stakeholders with varying domain expertise by leveraging different knowledge bases. It detects discrepancies in interpretations of NLRs and generates candidate disambiguated requirements using expert knowledge.
The final validation ensures that these requirements align with the intended system functionality.