We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
DeepMind Blog·Oct 23·5 min read
Introducing the Gemini 2.5 Computer Use model
TL;DR
Developers faced challenges in creating agents that effectively interact with user interfaces. Google has introduced the Gemini 2.5 Computer Use model, which enhances these interactions with improved performance.
✦ Why It Matters
Engineers can leverage the Gemini 2.5 Computer Use model to build faster, more efficient user interface agents.
Key Takeaways
- Access the Gemini 2.5 model via the Gemini API.
- Utilize the `computer_use` tool for UI interactions.
- Implement safety features to mitigate risks.
- Explore use cases in UI testing and workflow automation.
How It Works
The model operates by receiving user requests, screenshots of the environment, and recent action history. It analyzes these inputs to generate function calls for UI actions, such as clicking or typing.
After executing an action, it receives updated screenshots and URLs to continue the interaction loop until the task is completed.