EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hacker News·Jul 28·5 min read
Running Kimi K3 on a M1 Mac
TL;DR
Deltafin is a research project that successfully runs a large Mixture-of-Experts model on an M1 Max Mac, achieving exact and reproducible results despite slow processing times of 16 seconds per token.
✦ Why It Matters
Experiment with Deltafin on your M1 Mac to explore large AI models without needing high-end servers.
Key Takeaways
- Set up Deltafin with three simple commands.
- Choose between full model download or streaming mode.
- Expect slow inference times; optimize for local disk access.
- Utilize Deltafin's API for integration with existing applications.
How It Works
Deltafin employs a Mixture-of-Experts architecture, which allows it to selectively activate only a subset of its parameters for each token generation. This selective activation enables the model to run efficiently on hardware with limited resources, as it does not need to load the entire model into memory at once.
The model's design also includes optimizations like double-buffered layer loading and parallel expert reads, which enhance performance during inference.