EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hacker News·Aug 28·5 min read
Run Qwen3.8 27B locally: real numbers from my Mac Studio
TL;DR
Qwen3.8 27B has been effectively running on a Mac Studio, performing tasks like summarizing RSS feeds and organizing PDFs. Benchmark tests reveal it generates tokens at a rate of 14 tokens/s, comparable to its predecessor.
✦ Why It Matters
Consider running Qwen3.8 locally to automate mundane tasks and improve your workflow today.
Key Takeaways
- Qwen3.8 runs effectively on Mac Studio with 32GB RAM.
- It generates answers using fewer tokens than Qwen3.6.
- 1-bit quantization offers fast processing but lacks decisiveness.
- Update llama.cpp to avoid compatibility issues with Qwen3.8.
Full Summary
Qwen3.8 27B is a 27.3 billion parameter AI model that operates locally on devices like the Mac Studio, providing functionalities such as summarizing RSS feeds and organizing scanned documents. It runs at approximately 14 tokens per second, which is slower than its predecessor, Qwen3.6, but compensates by using fewer tokens per answer.
The model features a hybrid attention design and supports multimodal inputs, including images and videos. Benchmarking revealed that while Qwen3.8 is slower in generation speed, it is more efficient in token usage, resulting in similar overall response times.
The model's performance on Apple silicon highlights its potential for local deployment, making it a viable alternative to cloud-based solutions.