Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·Aug 27·5 min read
GreenLeaf Law Embed Tiny: A Compact Embedding Model for Legal Domain Retrieval
TL;DR
GreenLeaf Law Embed Tiny is a compact 0.6B parameter embedding model designed for legal domain retrieval, achieving competitive performance on benchmark tests. It utilizes a two-stage training process and a specialized dataset to enhance retrieval accuracy in resource-constrained environments.
✦ Why It Matters
Consider implementing GreenLeaf Law Embed Tiny for efficient legal information retrieval in your applications today.
Key Takeaways
- Explore the use of GreenLeaf Law Embed Tiny for legal retrieval tasks.
- Utilize the two-stage training approach for model development.
- Incorporate high-quality, curated datasets for better performance.
- Leverage quantization techniques for deployment in resource-limited settings.
Full Summary
GreenLeaf Law Embed Tiny is an innovative embedding model tailored for legal information retrieval, featuring 0.6 billion parameters. It scored 75.11% on the Massive Legal Embedding Benchmark (MLEB) and 64.38% on MTEB(Law, v1), showcasing its effectiveness among models with fewer than 1 billion parameters.
The model employs a two-stage training pipeline: first, it distills knowledge from a larger teacher model into a more compact student model, followed by domain-specific fine-tuning using hard negative mining. The training utilized a dataset of 3.4 million query-passage pairs, including 150,000 human-curated samples from various legal jurisdictions.
Additionally, the model supports multiple quantization levels (BF16, INT8, binary) for efficient deployment in environments with limited resources. The findings indicate that high-quality, domain-specific training data significantly enhances performance in specialized applications.