Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 28·6 min read
RULER: Representation-Level Verification of Machine Unlearning
TL;DR
Machine unlearning—removing a model's learned knowledge of specific data—lacks reliable verification methods to confirm deletion actually occurred. RULER is a representation-level verification framework that checks whether unlearned information has been removed from a model's internal learned representations.
✦ Why It Matters
Engineers can verify machine unlearning actually removes sensitive data from model internals, not just outputs, ensuring genuine privacy compliance.
Key Takeaways
- Verify unlearning by inspecting model internal representations, not just output predictions.
- Use RULER to detect when sensitive data persists in learned feature patterns despite output-level forgetting.
- Implement representation-level checks alongside output verification for comprehensive privacy validation.
- Test unlearning effectiveness on embeddings and activations to ensure genuine data removal.
Full Summary
Machine unlearning addresses privacy and legal requirements by removing a trained model's ability to recall specific training data. However, existing verification approaches only test output predictions, missing whether sensitive information persists in the model's internal representations—the learned feature patterns that encode knowledge.
RULER introduces representation-level verification, examining the actual feature activations and embeddings within the model to confirm unlearning effectiveness. The framework analyzes whether information about forgotten data has been genuinely eliminated from these internal learned structures, not just hidden from outputs.
This approach catches cases where models appear to forget at the output level but retain encoded knowledge internally. Results demonstrate RULER detects unlearning failures that surface-level output checks miss, providing stronger assurance that privacy-sensitive data has been truly removed from model memory.