Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 25·5 min read
Retrieval Is Not Enough: Why Organizational AI Needs Epistemic Infrastructure
TL;DR
Current AI retrieval systems for organizations surface relevant documents but ignore epistemic structure—whether claims are settled facts, contested, or unknown. OIDA, a knowledge framework with typed objects carrying commitment strength and contradiction edges, was built to represent organizational knowledge with explicit uncertainty.
✦ Why It Matters
Engineers can model organizational ignorance as computable properties and surface unresolved questions with increasing urgency in AI systems.
Key Takeaways
- Adopt the OIDA framework for better knowledge management in AI.
- Utilize Knowledge Objects to represent commitment levels and uncertainties.
- Implement the QUESTION mechanism to surface knowledge gaps.
- Evaluate AI responses using the Epistemic Quality Score for quality assurance.
How It Works
OIDA structures knowledge into 'Knowledge Objects' that carry attributes like epistemic class and importance scores. This allows AI to differentiate between settled facts and unresolved questions.
The Knowledge Gravity Engine maintains these scores with guarantees of convergence, ensuring reliable updates. The QUESTION mechanism surfaces unknowns with increasing urgency, prompting organizations to address knowledge gaps proactively.