Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·Jul 29·5 min read
Understanding User Experiences of Computer Use Agents: Design Space and Opportunities for Building Agent UX Prototypes
TL;DR
A new design framework for computer use agents was developed, focusing on user experience (UX) considerations and prototyping tools. The study identified key activities and capabilities necessary for effective agent UX design.
✦ Why It Matters
Start using AgentUXlab to prototype and evaluate user experiences for your computer use agents today.
Key Takeaways
- Explore the 21 UX considerations for agent design.
- Utilize AgentUXlab for prototyping agent user experiences.
- Incorporate identified activities and capabilities in your designs.
- Conduct user studies to refine agent UX further.
Full Summary
Computer use agents are generative AI systems that automate user interface actions based on commands. While much research has focused on the underlying AI models, user experience (UX) aspects remain underexplored.
This study established a taxonomy of 21 UX considerations for agent design and conducted a requirements elicitation study with 12 participants, including six experts. The findings revealed five key activities and six desired capabilities essential for prototyping agent UX.
Based on these insights, the researchers created AgentUXlab, a design probe that allows developers to create and evaluate different UX approaches for agents in a web environment. A user study with 14 participants provided further insights into tooling and design implications for agent UX prototyping tools.