GeoDial: A Multimodal Conversational Tutoring Dataset for Geometry Problem-Solving with Visual Tutor Turns

Many educational fields, particularly geometry, rely on visual aids like diagrams, yet most existing tutoring datasets focus solely on text interactions. GeoDial addresses this gap by providing a multimodal dataset consisting of over 1,300 dialogues between teachers and students, where instructional turns are linked to visual highlights in geometry problems.

A novel annotation protocol was developed to integrate dialog acts (the functions of spoken exchanges), visual highlighting, and feedback, allowing for detailed supervision of both language and visual tutoring behaviors. When vision-language models were fine-tuned on GeoDial, there was a significant improvement in the quality of generated dialogues, indicating better conversational tutoring capabilities.

However, these models struggled with accurately generating diagram highlights, underscoring a limitation in current AI methods. This highlights the need for improved approaches that can better combine visual reasoning with effective teaching strategies.

The findings suggest that while progress has been made, further advancements are necessary for AI tutors to match human instructional methods.