NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·Jun 10·5 min read
Advancing the State-of-the-Art in Empirical Privacy Auditing
TL;DR
Large language models (LLMs) can inadvertently memorize sensitive training data, posing privacy risks. A new method generates synthetic 'canary' examples to enhance empirical privacy auditing (EPA) by measuring data leakage.
✦ Why It Matters
Engineers can implement synthetic canaries to enhance privacy audits in machine learning models effectively.
Key Takeaways
- Utilize synthetic canaries for effective privacy auditing.
- Implement high-temperature sampling to generate canaries.
- Audit synthetic data using an auxiliary model.
- Explore the effects of model capacity on memorization.
Full Summary
Large language models (LLMs) often struggle with privacy due to their tendency to memorize specific training examples, which can lead to data leakage. To address this, a novel technique was developed that generates synthetic 'canary' examples using high-temperature sampling from LLMs, which are designed to be easily identifiable and non-private.
These canaries are mixed with sensitive training data to assess the risk of membership inference (MI) and reconstruction attacks. Additionally, an auxiliary model is fine-tuned on synthetic data to audit privacy leakage effectively.
The study systematically investigates how model capacity and canary entropy affect memorization, providing valuable insights into the interplay between these factors. Results indicate that the proposed auditing methods significantly enhance the ability to detect privacy risks in LLMs and synthetic data generation processes.
This work has implications for improving privacy-preserving techniques in machine learning applications.