Generating feedback on assignments with LLMs

Generating feedback on student assignments using generative AI is an interesting application for educators. It could speed up the process of generating feedback on assignments, which is helpful for students to be able to use the feedback to improve their work. Also, it could improve the quality of the feedback as a more thorough analysis could be conducted. These two aspects could free up time for teachers to devote to teaching or other activities. The latter is often seen by teachers as an end in itself.

At first glance, it appears straightforward: provide the LLM with the assignment description, the student’s work, and a grading rubric, and have detailed, personalised feedback generated. However, this seemingly simple process involves complex legal compliance regulations. And it requires considerable care to produce reliable, fair, and pedagogically sound results. For example, take the fact that an LLM can provide different feedback with the same grading rubric and submission.

For that reason, in this Companion, we cannot elaborate more on this topic currently.

AI Regulation

• AI systems used in education to assess learning outcomes or guide the learning process may fall under the high-risk category (Annex III of the AI Act). The fact that no grades are assigned does not rule this out. The VU must properly substantiate and document this classification;
• It must be clear whether the VU is a provider or a controller. This classification determines which compliance requirements the VU must meet;
• It is important to note that the VU can be both; if faculty members/researchers themselves build a system around an existing model, the VU may be considered a provider for that system; o Students must be aware that they are receiving AI-generated feedback, and the staff involved (faculty members, researchers) must possess sufficient AI literacy;
• The faculty member’s review (“human in the loop”) must be substantive and demonstrably structured—not merely a formality. “Human in the loop” means that the decision or grading is ultimately made by a human. How the training is conducted beforehand has no bearing on this. Personal training on the tool cannot replace this, and even with a random sample, there is not always a “human in the loop.”·

GDPR

• The focus here is on Article 22 of the GDPR: data subjects have the right not to be subject to a decision based solely on automated processing that produces legal effects or significantly affects them.