EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
MIT Technology Review·3d ago·10 min read
We’re putting too much faith in AI’s ability to say no
TL;DR
AI models are increasingly trained to refuse harmful requests, but their ability to do so is probabilistic and often unreliable, raising safety concerns.
✦ Why It Matters
Evaluate the refusal capabilities of AI tools you use to ensure they align with safety standards.
Key Takeaways
- Assess AI tools for their ability to refuse harmful requests.
- Understand the probabilistic nature of AI refusal mechanisms.
- Stay informed about the latest AI safety practices.
- Engage in discussions about ethical AI use and governance.
How It Works
AI models are trained using vast datasets and refined through exercises that reward them for refusing harmful prompts. Classifiers act as additional layers, filtering out dangerous requests before they reach the model.
⚠ The Catch
Refusal mechanisms are probabilistic, meaning they can fail, allowing harmful information to be generated despite safety measures.