EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hacker News·9d ago·5 min read
Show HN: Germany's new sovereign AI model Kolibri
TL;DR
Kolibri, a new open-weight large language model from Aleph Alpha, is designed for German and English, featuring 78 billion parameters but utilizing only 3.5 billion per token. Built with EU regulations in mind, it excels in reasoning and long-context tasks while ensuring data privacy and compliance.
✦ Why It Matters
Consider using Kolibri for projects requiring German language processing and strict data privacy.
Key Takeaways
- Utilize Kolibri for German language tasks requiring data privacy.
- Leverage its efficient tokenizer for processing long German texts.
- Set reasoning effort levels for tailored response complexity.
- Be aware of its limitations in memory-based knowledge tasks.
How It Works
Kolibri employs a mixture of experts architecture, where each token is processed by a subset of 384 specialists, optimizing computational efficiency. Its tokenizer, designed specifically for German, reduces the number of tokens needed for processing, enhancing performance in handling complex German language structures.
The model also uses sliding-window attention to manage long contexts effectively, allowing it to process up to 1 million tokens.
⚠ The Catch
Kolibri struggles with memory-based knowledge tasks, scoring lower on closed-book tests compared to competitors, indicating it may not be the best choice for trivia or general knowledge queries.