EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hacker News·Sep 3·5 min read
Qwen 3.8 27B available on Cerebras at 1500 tok/SEC
TL;DR
Cerebras has released the Qwen 3.8 27B model, available at 1500 tokens per second, while ensuring all hosted models are unpruned and of high quality. They utilize selective weight-only quantization for storage, maintaining precision during operations.
✦ Why It Matters
Explore the Qwen 3.8 model for high-performance applications requiring unpruned, high-quality AI models.
Key Takeaways
- Utilize the Qwen 3.8 model for advanced AI tasks.
- Access unpruned models for maximum operational quality.
- Consider exploring REAP pruned models on Hugging Face.
- Implement selective weight-only quantization for efficient storage.
Full Summary
Cerebras has introduced the Qwen 3.8 model, featuring 27 billion parameters and capable of processing 1500 tokens per second. The platform emphasizes transparency regarding model compression, stating that all publicly available models are the original, unpruned versions.
While they are researching pruning techniques like REAP (Router-weighted Expert Activation Pruning), these pruned models are shared separately on Hugging Face and not through their API. To maintain quality, Cerebras employs selective weight-only quantization for storage, using partial 16-bit, 8-bit, or 4-bit formats, while sensitive layers are kept at full precision.
This approach ensures that operations are performed in high precision, particularly for activations and attention mechanisms, which remain unquantized. The implications of this release highlight the balance between model size and operational quality.