Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 25·6 min read
When Planning Fails Despite Correct Execution: On Epistemic Calibration for LLM-Based Multi-Agent Systems
TL;DR
Multi-agent systems using large language models (LLMs) fail when agents incorrectly assess what they know about plan feasibility, even though execution runs without errors—a hidden problem called epistemic miscalibration. Researchers built EPC-AW (Epistemic Planning Calibration Agentic Workflow), which checks whether plan evaluations stay consistent across different information states rather than directly verifying feasibility.
✦ Why It Matters
Engineers can use EPC-AW to build more reliable multi-agent LLM systems by detecting and correcting hidden knowledge misjudgments before execution fails.
Key Takeaways
- Epistemic miscalibration—agents misjudging their knowledge—causes hidden failures invisible during plan execution.
- EPC-AW selects plans with stable evaluations across agents to detect inconsistent knowledge assessments.
- Use past planning discrepancies to refine future epistemic calibration and prevent recurring errors.
- Test plan feasibility under varying information conditions, not just direct verification, for robustness.
Full Summary
LLM-based multi-agent systems—where multiple AI agents coordinate to solve tasks—can fail despite executing planned actions correctly. The root cause is epistemic miscalibration: agents misjudge their own knowledge when deciding if a plan is feasible.
Unlike execution errors, this miscalibration remains hidden during planning because plans appear self-consistent and runnable. The problem worsens because new information dynamically changes feasibility assessments, causing past miscalibration signals to disappear and errors to repeat.
To address this, Wang et al. developed EPC-AW, which uses two mechanisms: Information-consistency-based Plan Selection picks plans whose evaluations remain stable when evaluated by different agents, and Consistency-guided Epistemic State Refinement adapts the system's knowledge calibration over time by using past discrepancies to improve future planning. Experiments demonstrated a 9.75% average improvement in overall system success.