Reimagining service delivery in the agentic era with Google Public Sector
cloud.google.com·19h ago
AI Research
arXiv CS.AI·Jun 9·5 min read
Impacts of Histories and Models on LLM Grading: A Study in Advanced Software Engineering Courses
TL;DR
Grading graduate-level research reading reports is labor-intensive for educators, leading to inconsistencies that affect fairness. A human-aligned large language model (LLM) grading workflow was developed and tested on 180 student submissions.
✦ Why It Matters
Engineers and researchers can leverage LLMs to streamline grading processes and improve assessment fairness in educational settings.
Key Takeaways
- Evaluate LLMs for grading to reduce educator workload.
- Monitor grading consistency across different LLMs.
- Implement operational practices to ensure fair assessments.
- Be aware of potential biases introduced by LLM grading.
How It Works
The study employs a human-aligned LLM-assisted grading workflow, where LLMs are trained to assess student submissions based on established human grading standards. By comparing LLM outputs with human scores, the researchers identify discrepancies and patterns in grading behavior.