Accelerating the search for antimicrobial peptides with Google Antigravity
deepmind.google·2h ago
AI Research
arXiv CS.AI·1d ago·5 min read
Measuring the Inference Energy Efficiency of Open-Weight LLMs across Benchmarks, Reasoning Modes, and Hardware
TL;DR
Energy efficiency varies significantly among open-weight large language models (LLMs), with some configurations using up to 27 times more energy than others for similar performance. This study emphasizes the need to report energy consumption alongside benchmark scores for a complete understanding of model efficiency.
✦ Why It Matters
Evaluate your LLMs not just on performance but also on energy consumption to optimize resource use.
Key Takeaways
- Measure energy consumption alongside performance scores for LLMs.
- Consider using FP8 to reduce energy usage without sacrificing performance.
- Be aware of significant energy differences among models with similar capabilities.
- Evaluate reasoning modes for their impact on energy efficiency.
Full Summary
Benchmark scores indicate the performance of large language models (LLMs), but they do not account for the energy required to achieve that performance. This research evaluates 86 open-weight model checkpoints across seven benchmarks and six hardware platforms, revealing substantial variations in energy efficiency.
On average, models consume 2.6 to 4.8 times more energy than the most efficient configurations that achieve similar scores, with some differences reaching as high as 27 times. While reasoning capabilities can enhance performance, they also increase energy consumption by about 3.7 times.
Additionally, using FP8 (a numerical format) can reduce energy usage by approximately 25% without significantly affecting performance. The findings suggest that energy consumption should be reported alongside benchmark scores to provide a more comprehensive view of model efficiency.