Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·Jun 9·5 min read
A Comparative Study of Student Perspectives on Technical Writing Feedback Quality: Evaluating LLMs, SLMs, and Humans in Computer Science Topics
TL;DR
Students often struggle to receive timely and effective feedback in computer science courses. A locally hosted Small Language Model (SLM), specifically a quantized Llama-3.1, was evaluated alongside commercial Large Language Models (LLMs) and human instructors.
✦ Why It Matters
Engineers and researchers can explore using local SLMs for scalable feedback in educational settings without compromising privacy.
Key Takeaways
- Evaluate local SLMs for scalable feedback in educational settings.
- Use AI for structural guidance while reserving human feedback for specialized tasks.
- Prioritize readability and actionability in technical writing feedback.
- Explore privacy-preserving alternatives to commercial LLMs.
How It Works
The study utilized a quantized version of Llama-3.1 and GPT-4 to create a locally hosted Small Language Model (SLM) that provides feedback on technical writing. This model was designed to maintain privacy and reduce costs associated with commercial LLMs, allowing it to deliver actionable and readable feedback to students.