We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
Hugging Face Blog·Jun 3·5 min read
Direct Preference Optimization Beyond Chatbots
TL;DR
Many OCR models struggle with text degeneration, where they produce repetitive outputs instead of accurate transcriptions. Direct Preference Optimization (DPO) was applied to DharmaOCR to address this issue by using rejection pairs from the model's own failures.
✦ Why It Matters
Engineers can apply DPO to enhance OCR model performance by specifically targeting and reducing text degeneration issues.
Key Takeaways
- Utilize DPO to reduce text degeneration in OCR tasks.
- Leverage model's own failures as training signals for better outcomes.
- Achieve significant performance improvements beyond traditional SFT.
- Explore DPO's applicability in other structured generation tasks.
How It Works
DPO operates by using the model's degenerate outputs as negative examples, creating preference pairs that explicitly label failure modes. This contrasts with SFT, which optimizes for correct outputs without addressing specific failure types.
By focusing on complete outputs rather than token-level predictions, DPO effectively penalizes undesirable repetition loops, leading to improved transcription quality.