Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
Data Science
Analytics Vidhya·Sep 2·5 min read
A Complete Guide to Decoding LLM Model Names
TL;DR
Understanding the components of large language model (LLM) names can simplify the process of selecting a local model. Each part of the name reveals critical information about the model's size, architecture, and usage.
✦ Why It Matters
Familiarize yourself with LLM naming conventions to choose the right model for your project today.
Key Takeaways
- Identify model types: dense vs. Mixture-of-Experts.
- Choose between base and instruct models based on use case.
- Understand quantization levels to manage memory requirements.
- Read model names carefully to extract essential specifications.
How It Works
LLM model names encode critical information such as total parameters, active parameters per token, and quantization methods. For example, a model labeled '35B-A3B' indicates 35 billion total parameters with approximately 3 billion active for each token.
This allows for efficient memory usage, especially in MoE models, which activate only a fraction of their parameters during inference.