Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 27·6 min read
ICCU: In-Context Continual Unlearning via Pattern-Induced Refusal Rules
TL;DR
Language models deployed in production receive sequential requests to forget specific training data, but fine-tuning each request individually is expensive and degrades model performance. ICCU (In-Context Continual Unlearning) uses in-context learning to generate readable refusal rules that prevent models from producing unwanted outputs without retraining.
✦ Why It Matters
Engineers can deploy unlearning without expensive retraining, reducing operational costs while maintaining model performance across multiple requests.
Key Takeaways
- Use in-context learning to avoid retraining costs for each unlearning request.
- Generate interpretable refusal rules that prevent unwanted model outputs without weight updates.
- Handle multiple sequential unlearning requests without cross-request interference or utility loss.
- Deploy ICCU in production systems requiring privacy compliance or content control.
Full Summary
Machine unlearning removes the influence of specific data from trained language models—a requirement for privacy compliance and content control. Sequential unlearning requests in production deployments challenge existing fine-tuning approaches, which require retraining for each request, accumulate performance degradation, and risk interference between requests.
ICCU proposes an in-context continual unlearning framework that induces pattern-based refusal rules—interpretable decision rules that guide model behavior—without modifying model weights. The approach leverages in-context learning, where instructions are provided at inference time rather than through retraining.
By generating readable refusal rules from unlearning data, ICCU avoids repeated fine-tuning cycles, preserves model utility across requests, and reduces computational overhead. This enables practical deployment of unlearning in real-world systems handling multiple sequential privacy or safety requests.