Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 27·8 min read
CroCo: Cross-Lingual Contrastive Preference Tuning on Self-Generations
TL;DR
Multilingual language models often perform worse in non-English languages due to limited training data and uneven optimization across languages. CroCo is a contrastive preference tuning method that improves cross-lingual performance by training models on self-generated outputs ranked by quality across multiple languages simultaneously.
✦ Why It Matters
Engineers can improve multilingual model performance cost-effectively using self-generated training data and contrastive optimization without extensive human annotation.
Key Takeaways
- Explore using CroCo for multilingual AI applications.
- Utilize English-trained reward models for other languages.
- Focus on on-policy data for optimal performance.
- Evaluate structured tasks to measure model improvements.
How It Works
CroCo employs a reward model trained on English preferences to rank self-generated responses in various languages. By applying contrastive tuning, it enhances the quality of responses without requiring specific annotations for each language.
The method allows for both monolingual and multilingual pairings, which contribute to improved performance across diverse tasks.