Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·6d ago·5 min read
Fine-Tuning a 3B-Parameter LLM on a Smartphone: Characterizing Sustained Training
TL;DR
A 3B-parameter large language model (LLM) can be fine-tuned on an iPhone 17 Pro, achieving effective personalization without data leaving the device. This process is feasible within a single battery charge, although sustained training reduces the phone's performance significantly.
✦ Why It Matters
Consider optimizing your mobile AI applications for on-device fine-tuning to enhance user personalization.
Key Takeaways
- Explore on-device fine-tuning for personalized AI applications.
- Monitor energy consumption during model training on mobile devices.
- Investigate runtime optimizations to improve training efficiency.
- Consider user privacy when designing AI personalization features.
How It Works
The fine-tuning process involves adjusting the model's parameters based on user interactions while keeping the base model frozen. This allows for personalization without transferring sensitive data off the device.
The study identified that most training time is spent in the backward pass of the frozen model, which was not optimized in existing runtimes.