Grading open-ended exam questions with LLMs

A specific case of generating feedback with LLMs is the process of grading open-ended questions of exams using generative AI. An important driver to use LLMs for this purpose is that it could free up time for teachers. In particular for teachers with large student cohorts that take exams (e.g. > 100). Note that open-ended questions in this case require short answers for which quite explainable and unambiguous grading rules can be set.

Comparable to providing feedback to student submissions, it appears straightforward: provide the LLM with the question, the student answer, and a grading rubric, and have a score with a rationale generated by the LLM. However, this seemingly simple process requires compliance with complex legal regulations. And it requires considerable care to produce psychometrically reliable, fair, and pedagogically sound results, and when human oversight is sufficiently needed to be maintained.

Further, no production-ready systems exist to function as assistive technology in this process at this moment.

For that reason, in this Companion, we do not elaborate more on this topic apart from some notions of the main legal and psychometric considerations.

#9.1Some legal considerations

European AI-Act

• AI systems used in education to assess learning outcomes or guide the learning process may fall under the high-risk category (Annex III of the AI Act). The VU must properly substantiate and document this classification;
• It must be clear whether the VU is a provider or a controller. This classification determines which compliance requirements the VU must meet;
• It is important to note that the VU can be both; if faculty members/researchers themselves build a system around an existing model, the VU may be considered a provider for that system;
• Students must be aware that they are receiving AI-generated scores, and the staff involved (faculty members, researchers) must possess sufficient AI literacy;
• The faculty member’s review (“human oversight”) must be substantive and demonstrably structured—not merely a formality. “Human oversight” means that the decision or grading is ultimately made by a human. How the training is conducted beforehand has no bearing on this. Personal training on the tool cannot replace this, and even sampled control, there is not always a “human oversight.”·

GDPR - General Data Protection Regulation (Dutch AVG - Algemene Verordening Gegevensbescherming)

• The focus here is on Article 22 of the GDPR: data subjects have the right not to be subject to a decision based solely on automated processing that produces legal effects or significantly affects them.

#9.2Psychometric validity

For tests and exams, the psychometric properties are of the highest importance. Scores on tests and questions must be consistent (reliable) and in accordance with human scores. And there are many factors at play that determine the reliability, validity, and usefulness of LLMs for grading open-ended exam questions. As mentioned before, based on the current research (we refer to paragraph Error! Reference source not found. for some scientific underpinnings), research has shown that LLMs show high correlations and agreement with human scores, but they can only serve as an assistive technology. It cannot serve as a technology operating independently. Edge cases, unclear questions and grading schemes, as well as students around the pass-fail border need careful attention and may in some cases require manual re-grading. Yet, despite these limitations, the research shows that significant gains are feasible by using LLMs.

#9.3Example case

For illustrative purposes, first, we describe some preliminary results from a scientific study of a business exam. The test is comprised of 12 open-ended questions. Each question for each student is graded three times, and then the main, median, or mode value is chosen as the final score. In an analysis, the correlation of the manual scores (given by one rater) and the LLM scores is calculated, and scatterplots are shown. The plot shows some common characteristics you would encounter when doing such an analysis.

#9.3.1Test level

First, we describe typical findings on the level of a test as a whole. In Figure 9.1 Scatterplot of manual scores and LLM scores on a test of 12 open ended questions. you can see how manual scores for one rater and the LLM modes’ scores are related. This kind of scatterplot is quite typical.

The following can be observed.

• Using a high-performance LLM (in this case GPT-4o), it is possible to achieve a substantive correlation between the LLM score and the human scores. The human grading reliability of the test was 0.78 and the LLM reliability was 0,78 too. This is a good result, as generally the LLM score is somewhat lower in comparison. This is also comparable to the internal consistency value of multiple graders of exams as is often found in research.
• Further, the scores correlate quite well, and are also in the same magnitude. The true manual score line (y=x) is almost exactly followed. The regression line is a bit steeper, indicating that students have a chance of getting a bit higher score if they already score high and vice versa. This is also a bit of a chance plot in this case, as other scientific studies have indicated that it is mostly the other way around: the line is less steep than y=x. This is due to regression to the mean (for high scores, the chance is larger to get lower scores when regrading, for low scores, the chances are higher to get higher scores).
• Finally, the plot shows two yellow areas based on the pass-fail score, in which students would be misclassified as compared to the manual scores. This is calculated to be about 13% of the students for this case. This would imply that teachers should manually grade this percentage of the total number of students around the pass-fail grade to make sure these grades have been checked with sufficient human-in-the-loop in the process.

In paragraph Error! Reference source not found. we shortly describe some finding in the current literature regarding grading of open ended question by LLMs.

#9.3.2Question level

At the individual question level, the scatterplots show more variations. Below we show some typical encountered plots in which the mean, median and mode scores are plotted.

What the plots show is the following.

• The correlations between manual and LLM scores are positive and consistent.
• The pattern of regression to the mean (grade compression) is well visible (less steep curve than y=x).
• The chance of getting exactly the same human score as LLM score is limited.
• More capable LLMs show higher consistency (no plot shown).
• Further, the average LLM score for different questions show varied deviations from the manual scores. Sometimes the LLM scores on average higher, sometimes lower.
• When scores are off, a process must be in place in which the poorly correlating questions must all be graded manually.

We have observed that both the quality of the question and the quality of the grading scheme are of prime importance for the quality of the scoring of the LLM as is also found in the current research. And that is very logical: both must be unambiguous, preferably must be able to be expressed in countable facts, concept, arguments reasonings etc. In Figure 9.3 we present one such a question.

In contrast, questions asking for broad, creative personal perspectives or free interpretations of answers likely lend themselves less for grading using LLMs. We note however that we have not yet studied such exams.

The conclusion is that LLMs can be used for grading open ended question for exams but with caution. Generative AI can only serve as an assistive technology. Human oversight remains very important and low correlated manual-LLM questions, edge cases as well as students around the pass-fail border need careful human attention and likely manual grading.

#9.3.3Student acceptance of LLM grading

An important aspect of grading with LLMs for which little research is available and the VU Education Lab has no experience with, is the acceptance of grades by students. Research on the acceptance of AI generated diagnosis for patients with eye problems or health related issues, showed different levels of trust (Shah et al., 2022; Wewetzer et al., 2026). Acceptance will likely depend on the way in which students are introduced and informed about the grading process and the guard rails in place. But also, we expect that differences will occur based on the presentation of results to the students. Research in this area is needed.