NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·20d ago·5 min read
Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System
TL;DR
A new AI tutoring system, the Learning Engagement Assistant (LEA), was developed to enhance student engagement across various courses. It utilizes agentic AI, which allows for adaptive learning experiences tailored to individual student needs.
✦ Why It Matters
Engineers can implement adaptive learning algorithms in their educational software to enhance user engagement and learning outcomes.
Key Takeaways
- Deploy LEA in varied academic settings to assess its performance.
- Monitor Answer Relevancy and Context Precision across different courses.
- Investigate the decline in Faithfulness with curriculum distance.
- Refine downstream components for better course-agnostic performance.
Full Summary
LEA is an AI tutoring agent that integrates course-specific Retrieval-Augmented Generation (RAG) with structured Knowledge Component (KC) models, functioning in Chat, Tutor, and Quiz modes. Initially validated through simulations in a single STEM course, this study reports its first real-world deployment with eight students across three different courses.
The evaluation involved 660 questions, revealing that while Answer Relevancy and Context Precision remained stable (0.88-0.94 and 0.88-0.90), Faithfulness dropped significantly from 0.69 to 0.50 as the curriculum distance increased. This suggests that while the orchestration layer of LEA is robust, the underlying components may need further refinement to achieve full course-agnosticism.
The divergence from simulation predictions highlights the limitations of synthetic evaluations in anticipating real-world performance.