We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
Hacker News·19d ago·5 min read
Show HN: We hid a backdoor in an LLM – $51,200 on finding it
TL;DR
A hidden backdoor was embedded in one of seven seemingly identical language models, which could be activated by a specific word known only to its creator. Despite passing all standard tests, this model poses a significant risk to users who trust these downloads without verification.
✦ Why It Matters
Engineers should implement checksum verification for all downloaded models to ensure their integrity before use.
Key Takeaways
- Always verify model integrity using checksums before deployment.
- Be aware that models may contain hidden vulnerabilities.
- Engage in community challenges to improve detection methods.
- Document findings and share insights on model security.
Full Summary
Researchers conducted a challenge involving seven language models hosted on HuggingFace, where one model (Model 3) contained a hidden backdoor that produced a false statement when prompted. Participants were tasked with identifying the exact trigger phrase that would reveal this backdoor, with a reward of $51,200 for its discovery.
The models were designed to be statistically indistinguishable, making it nearly impossible to identify the rogue model through conventional testing methods. Despite extensive attempts, the specific phrase that would activate the backdoor was not recovered, demonstrating the robustness of the hidden mechanism.
This challenge underscores the importance of transparency and verification in AI models, as even trusted models can harbor vulnerabilities that go undetected. The results emphasize the need for rigorous scrutiny in AI deployment to prevent reliance on potentially compromised systems.