Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·Jun 9·5 min read
PathoSage: Towards Multi-Source Evidence Adjudication in Pathology via Experience-Aware Agentic Workflow
TL;DR
Pathologists face challenges with inconsistent evidence and hallucinations in AI-driven analysis. PathoSage is a three-stage framework that separates knowledge retrieval, evidence collection, and adjudication to improve decision-making.
✦ Why It Matters
Engineers can leverage PathoSage to enhance AI reliability in pathology, improving diagnostic accuracy and decision-making.
Key Takeaways
- Implement PathoSage for improved evidence adjudication in AI pathology.
- Utilize Structured Evidence Deliberation to reduce bias in decision-making.
- Adopt reliability-aware models for better tool performance assessment.
- Explore training-free systems for continuous credit assignment in AI workflows.
Full Summary
In computational pathology, reliable reasoning at the patch level is hindered by issues like hallucinations—incorrectly generated features—and conflicting evidence from various tools. PathoSage addresses these challenges with a structured three-stage framework: knowledge retrieval, evidence collection, and evidence adjudication.
Its core component, Structured Evidence Deliberation, evaluates diverse evidence independently, analyzes conflicts, and generates judgments in a new context to minimize bias. Additionally, a training-free Beta-Bernoulli experience system models long-term tool reliability and creates similarity-weighted priors for future tool usage.
Experimental results indicate that PathoSage significantly reduces visual question answering (VQA) hallucinations and classifier disagreements, outperforming existing pathology models. These findings underscore the importance of explicit evidence adjudication and reliability-aware modeling in developing robust AI agents for pathology.