EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Lobsters·Sep 2·5 min read
Bye Bye Perspective API: Lessons for Measurement Infrastructure in NLP, CSS and LLM Evaluation
TL;DR
The Perspective API, a key tool for measuring toxicity in text, will shut down in 2026, highlighting the need for the NLP community to develop its own measurement infrastructure. This reliance on external tools has led to significant issues in research validity and reproducibility.
✦ Why It Matters
Start developing independent measurement tools to ensure research integrity and reliability in NLP.
Key Takeaways
- Assess your current reliance on external measurement tools.
- Engage with peers to discuss building independent measurement frameworks.
- Explore the released Perspective scores for your research.
- Prioritize infrastructure work in your research agenda.
How It Works
The Perspective API uses machine learning to analyze text and assign toxicity scores, which researchers have relied on for evaluating language models and datasets. However, its closure necessitates the development of community-owned tools that can provide consistent and explainable metrics.