Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 26·5 min read
Multi-Agent Specification-based Metamorphic Testing of FMU-Based Simulations
TL;DR
In industrial settings, testing Functional Mock-up Units (FMUs) is challenging due to the lack of clear expected outputs. A multi-agent workflow powered by large language models (LLMs) was developed to automate the extraction of metamorphic relations (MRs) from specifications for FMU testing.
✦ Why It Matters
Engineers can leverage this automated approach to enhance the reliability of FMU-based simulations while minimizing manual testing efforts.
Key Takeaways
- Adopt multi-agent systems for automated test case generation.
- Utilize Given-When-Then patterns for structuring test conditions.
- Leverage LLMs to extract requirements from specifications.
- Evaluate your FMU-based simulations with metamorphic testing.
How It Works
The proposed workflow orchestrates multiple agents to analyze functional and interface specifications, automatically extracting requirements and deriving metamorphic relations. These relations are expressed in a structured format, allowing for the generation of test cases that can be executed across multiple simulation sessions to check for output consistency.