Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·Aug 3·5 min read
FriendBench: Benchmarking Dyadic Familiarity Inference in Humans and Multimodal Large Language Models
TL;DR
FriendBench is a new benchmark designed to assess how well humans and multimodal large language models can infer familiarity between two people from short conversation clips. Results show that while the best models match human accuracy, they differ in their approach, favoring 'stranger' assumptions more than humans do.
✦ Why It Matters
Consider integrating multimodal data to enhance AI models for social interaction analysis today.
Key Takeaways
- Explore using multimodal data for social inference tasks.
- Evaluate AI models against human performance benchmarks.
- Consider the impact of visible behavior on interaction analysis.
- Analyze model biases towards labeling interactions as 'strangers'.
How It Works
FriendBench evaluates models by analyzing 20-second video clips of conversations, where participants' responses to prompts reveal their familiarity. The benchmark assesses performance across text, audio, and video modalities, allowing for a comprehensive understanding of how different data types influence inference accuracy.