Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 26·5 min read
Measuring the Depth of LLM Unlearning via Activation Patching
TL;DR
Large language models (LLMs) face challenges in confirming the complete removal of sensitive knowledge after unlearning. To address this, the authors developed the Unlearning Depth Score (UDS), which quantifies the extent of knowledge erasure through a method called activation patching.
✦ Why It Matters
Engineers can use UDS to more accurately assess the effectiveness of unlearning techniques in LLMs.
Key Takeaways
- Adopt the Unlearning Depth Score for better unlearning assessments.
- Utilize activation patching to analyze model knowledge retention.
- Benchmark unlearning effectiveness across various methods.
- Integrate UDS into your existing evaluation pipelines.
How It Works
UDS operates by first establishing a baseline model that retains the target knowledge. It then analyzes the internal activations of the unlearned model to determine which layers still encode that knowledge.
By measuring the extent of knowledge erasure across these layers, UDS provides a quantifiable score that reflects the depth of unlearning.