tailwindlabs / tailwindcss
github.com·21h ago
Developer Tools
GitHub Trending·Jun 9·5 min read
Andyyyy64 / whichllm
TL;DR
Software engineers often struggle to identify which local large language model (LLM) performs best on their specific hardware. whichllm is a tool that allows users to find and run the most suitable LLM with a single command, based on real benchmarks that consider recent performance. This tool ranks models not by their size, but by their actual performance metrics, enabling better decision-making for deployment.
✦ Why It Matters
Engineers can quickly identify the best-performing LLM for their hardware, enhancing application efficiency.
Key Takeaways
- Use whichllm to auto-detect your hardware specifications.
- Rank models based on real-world performance, not just size.
- Simulate different GPUs to plan hardware purchases effectively.
- Generate Python code snippets for easy model integration.
How It Works
whichllm fetches model data from the HuggingFace API and ranks them using a scoring system that incorporates benchmark quality, model size, and runtime fit. It evaluates models based on their VRAM requirements and performance metrics, ensuring that users receive the best recommendations tailored to their specific hardware.