Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·Aug 28·5 min read
Sophistication in GenAI Use: Field Evidence from a Large Firm
TL;DR
A study of generative AI (genAI) use among nearly 4,000 back-office employees reveals that senior staff utilize genAI more effectively, particularly in strategic roles. However, sophistication in genAI use does not improve over time or with formal training.
✦ Why It Matters
Evaluate your team's AI training programs to ensure they effectively enhance genAI usage.
Key Takeaways
- Assess the impact of employee seniority on genAI usage.
- Focus on enhancing genAI training for strategic roles.
- Monitor function-specific genAI sophistication levels.
- Consider alternative methods to improve genAI engagement.
Full Summary
Research on generative AI (genAI) use within a large firm analyzed 713,564 employee prompts and responses from almost 4,000 back-office employees across 15 functional areas over eight months in 2025. Findings indicate that senior employees demonstrate more sophisticated genAI usage, suggesting that their domain expertise enhances their interaction with AI tools.
Additionally, sophistication levels vary significantly across functions, peaking in Strategy, Digital Innovation, and Project Management, which are aligned with strategic initiatives. Notably, the study found no significant improvement in sophistication over time or after formal AI training, indicating that enhancing genAI usage may be challenging.
These insights provide valuable metrics for managers aiming to optimize AI outcomes and serve as a foundation for future research in the field.