Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·Aug 3·5 min read
SERUM: State Extraction and Refinement for User Modeling
TL;DR
SERUM is a novel framework that extracts structured user behavior models from unstructured screen recordings, achieving improved accuracy in intent recognition. It refines labels through multiple passes, resulting in a coherent taxonomy and better predictive performance.
✦ Why It Matters
Implement SERUM to enhance user modeling in your applications without manual data labeling.
Key Takeaways
- Utilize SERUM for automated user intent modeling from screen recordings.
- Leverage multi-pass refinement to improve label accuracy.
- Apply normalized Markov models for better predictive performance.
- Explore SERUM's public demo and code for practical implementation.
How It Works
SERUM processes screen recordings through a sliding window, alternating between recognizing activities and inferring user intent. Each pass refines the labels based on accumulated context, reducing errors and merging similar states into a coherent taxonomy using sentence embeddings.