We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
Reddit r/LocalLLaMA·May 20·5 min read
24GB M4 Mac - is Qwen 9B only option while system is running?
TL;DR
A user with a 24GB M4 Mac asked whether Qwen 9B (a 9-billion-parameter language model—a neural network with 9 billion learned weights) was the only practical option for running local LLMs (large language models) while keeping the operating system responsive. The discussion explored memory constraints: M4 Macs have limited unified memory (shared RAM for CPU and GPU), and running larger models consumes most available space, leaving little for system processes.
✦ Why It Matters
Engineers optimizing LLMs on resource-constrained hardware can use quantization and model selection strategies to balance model capability against system stability.
Key Takeaways
- 24GB M4 Macs support models larger than 9B with quantization; test 13B-34B variants.
- Quantization to 4-bit or 8-bit reduces memory footprint while preserving usable performance.
- Monitor system memory headroom; reserve 4-6GB for OS and background processes.
- Benchmark inference speed and latency on your target model before deployment.
Full Summary
A developer posed a question about running large language models (LLMs—AI systems trained on vast text data) on Apple's M4 Mac with 24GB unified memory while maintaining system stability. The constraint was whether background system processes and OS overhead left enough free memory to run only smaller models like Qwen 9B.
Community members explored the memory footprint of various open-source models, quantization techniques (methods to reduce model size by lowering numerical precision), and real-world performance on Apple Silicon. Responses indicated that 24GB can accommodate models larger than 9B parameters with proper optimization, including 13B and some 34B models using 4-bit or 8-bit quantization.
The discussion highlighted trade-offs between model capability, inference speed, and system responsiveness, with practical recommendations varying by user tolerance for slowdowns.