What teachers need to know about students’ AI literacy

For whom: teachers who want to know what their students do with generative AI, and what those students do not yet understand about it

Length: 90 minutes (a 60-minute version is at the end) · Group size: 8 to 24. This session runs on the list the group makes itself; above 24 that list becomes unreadable and the talk about it shallow.

Prior knowledge: none required. Participants who have used a chatbot themselves have an easier time of it, but the session works without.

Language: the slides are English; the conversation need not be. Ask in the first minute.

What participants can do afterwards

• Name and distinguish the key concepts around AI.
• Explain what AI literacy means for students, and why it concerns their own subject.
• Recognise and take apart the common misconceptions and concerns about generative AI.
• Hold that conversation with students without it turning into a telling-off.

The shape: one exercise, not nine statements

This Short comes with nine line-up statements. Do not use them all. Two is the maximum, alongside the opening exercise in which participants work out for themselves what AI literacy is.

This is not a matter of taste. The first time this Short was given, the Gemini Notebook exercise never made it to the end: the statements had eaten the time. Nine is also too many on its own terms - after the third the room recognises the pattern and everyone stands in the same place. The line-up is a sharpening device for the list, not the session itself.

What remains is a rising sequence: the group makes its own list, two statements put that list under strain, the Taskforce advice lays a framework beside it, and the Companion tests the whole thing. Each part needs the one before it, which is why you cannot drop one out of the middle.

What to arrange beforehand

• The notebook is ready and you have tested it yourself. This is the most important preparation for this session. Put the Companion into Gemini Notebook, ask the two questions from the exercise yourself, and see whether the answer gives a pair something to work with for ten minutes. Do not do this for the first time in the room.
• A shared link to that notebook, or an instruction on paper for participants to fill one themselves. Allow five minutes of sign-in trouble even when you have prepared well.
• Laptops, one per pair. That is not stingy but better: the exercise is a conversation, not a form to fill in.
• An empty board or a flip chart, plus sticky notes or paper strips. The list from the opening exercise stays up for the whole session; you point back at it three more times.
• Room to move for the line-up: a long wall, or an imaginary line across the room.
• The Taskforce advice to hand, cut into three parts (what, how, why). Printed works better than a slide, because the group has to lay it beside its own list.
• A back door in case the notebook will not cooperate. See the section on the Companion.

Timeline

1. 0 to 5 · Welcome and choice of language. Ask straight away whether this will be in Dutch or English. A room allowed to speak Dutch says more, and the slides can stay English.
2. 5 to 15 · What do you see students doing. Do not survey who uses AI themselves, but who has seen what among students. That is the question of this Short, and it produces examples you point back at for the rest of the hour. Write two or three of them down; you need them at the statements.
3. 15 to 35 · What is AI literacy. The opening exercise, in pairs. This is the heart of the session.
4. 35 to 45 · Two statements. Line-up. No more than two.
5. 45 to 60 · The Taskforce advice beside it. What, how and why, laid next to what the group had already worked out.
6. 60 to 80 · The Companion, with Gemini Notebook. Protect this block. If something overruns, cut the statements, not this.
7. 80 to 88 · What we know by now. The findings on what students can and cannot do.
8. 88 to 90 · One commitment. Everyone names the person they will talk to this week about AI literacy in their own curriculum. A name, not an intention.

The opening exercise: what is AI literacy

In pairs, twenty minutes, in three moves:

1. Discuss what you think AI literacy is, and what students should know and be able to do. No looking up a definition; their own words are the material here.
2. Make a list of it and put it on the board. Sticky notes work best: individually movable, and you can regroup them later.
3. Split the list in two: what students can already do, and what they cannot.

The third move does the work. The first two produce a list everyone agrees with - critical thinking, checking sources, knowing how it works - and the conversation only starts at the split, because that is where it turns out nobody knows. Most teachers have never asked what their students do with AI; they have inferred it from submitted work. That difference is exactly what this Short is about.

Two things that sharpen the exercise:

• Insist on the split. A pair that writes "partly" beside everything has not done the exercise. Push: where did you see that?
• Leave the list up. It is your touchstone for the Taskforce advice and for the Companion. Wipe it halfway and the rest of the session becomes loose parts.

A related activity for the lecture hall rather than the staff room: Ask students about their AI experiences.

The two statements

Line-up: you read a statement, people stand on a line between "strongly agree" and "strongly disagree", and you ask three people at different points why they stand there. Five minutes per statement, and the question "who moves after that argument" is worth more than a fourth explanation.

The standard pair, and the only pair that makes a point on its own:

1. "I am a digital native and know how to use generative AI." About themselves. The room always spreads out here.
2. "Students are 'digital natives' and know how to use generative AI." The same statement, about someone else. Here the group shifts as a whole, and that is the point: we rate our own skill cautiously and our students' generously. Do not say so yourself - ask whether anyone noticed what changed between the two rounds.

Swap the first one out when the group has a clearer problem:

• "I find it difficult to start a conversation with students about generative AI." For a group that is mainly at a loss for what to do. Connects straight to the fourth objective.
• "Students do not comply with a ban on generative AI in my course." For a group that wants to talk about rules. Lively, but it pulls the conversation towards enforcement; only use it if that is where you want to go.

The other statements on the slides - that AI literacy can be taught in one lecture, that students know how the technology and the algorithms work, that students have no interest in how an LLM is made, that the degree programme has consensus on what students may and may not do - are all usable, but not here. Keep them for a longer session or for the Short on AI literacy for teachers. If you do want a full round of statements as an activity in its own right, it is written up as Generative AI contested statements line-up.

The Taskforce advice beside it

The advice of the UvA-VU Taskforce divides AI literacy into three categories. Hand them out in three parts and have the group lay them beside their own list on the board.

• What - knowledge and understanding. Explaining how the technology behind AI works, understanding that models are built by collecting and processing vast amounts of data, naming basic concepts from programming and machine learning in broad terms, and knowing the AI applications that count in one's own field.
• How - understanding and application. Recognising kinds of application and their risks (bias, energy use, intellectual property), judging when use is worthwhile so that you do not keep handing over your own thinking, going on practising complex writing rather than reaching for a summary too soon, choosing methodically which source fits which question, and writing targeted prompts.
• Why - ethics and judgement. The academic core, interwoven with integrity: formulating ethical arguments for and against, judging AI output against the quality criteria of one's own discipline while recognising that it offers one perspective whose source cannot be traced, copyright and attribution, disciplinary reflection on where AI does and does not belong, and privacy.

The question for the group is not whether the advice is right, but what it adds to their own list - and what their list has that the advice lacks. That last one almost always yields the same answer: the advice pays little attention to environmental and societal impact. Let that stand as a gap rather than a fault; it is why the Shorts series devotes two separate sessions to it.

For anyone who wants to go further here: What uses more: the footprint of a prompt and Data privacy and AI infrastructure.

The Companion, with Gemini Notebook

The AI Literacy Companion is the course book for VU students. This exercise lets teachers explore the book and watch an AI tool at work, in one move. In pairs, at one laptop:

1. Ask Gemini Notebook what the ten key concepts from the Companion are.
2. Ask how the Companion relates to the Taskforce's recommendations.
3. Discuss the answers with each other.
4. Ask the same two questions again.
5. Discuss again.

Step four is the whole exercise. The answers differ, sometimes considerably, and most participants have never seen this because they never asked the same thing twice. That is the lesson: not what the ten key concepts are, but that "the" ten key concepts do not exist. Do not say so in advance; let them notice, and then ask what this means for a student using it for an assignment.

This is also the block that most often falls off the end. The first time round it never got there. Three things keep it standing:

• Set the notebook up beforehand and share the link. Participants who have to create a notebook and upload the book lose ten minutes to everything except the exercise.
• Start on time. Twenty minutes is tight; fifteen is not enough for two rounds and a conversation. That is why this plan holds at most two statements.
• Have a back door. If the notebook will not work, have the pairs put the questions to Copilot or ChatGPT with the Companion alongside, or have them pull the ten key concepts out of the teacher edition themselves and compare with what another pair found. The second round is lost, but the conversation survives.

What we know by now

The closing block. Lay these findings beside the list on the board; the group has almost always written a few of them down already, and that recognition is what the session ends on.

• Staying vague about what is and is not allowed does make students use AI less - but it also leaves them very uncertain.
• Students overestimate their own digital skills and their ability to write prompts.
• They still make the most obvious mistakes: failing to check references, and having no feel for the text they hand in.
• Teachers overestimate their students' digital skills just as badly.
• Students are embarrassed to disclose what they did with AI, even when you ask politely.
• Students believe they learn more when they use AI.

The last two together explain most of what teachers see in submitted work. To improve the disclosure question rather than repeat it, go on with The transparency statement and Agree the ground rules for AI in your course.

What goes wrong

• The Companion exercise drops off the agenda. The mistake that was actually made, and the most expensive one: it is the only part in which participants do something with an AI tool themselves.
• The statements become the session. They are enjoyable and they always overrun. Set a timer.
• The conversation turns into complaining about students. Predictable after the first statement. Turn it back with the list on the board: we have just established that we do not know what they can do, so what do we actually know here.
• Someone asks about the rules. Is it allowed in my course or not. The question belongs here but is not answered here; park it and point to the programme and to the Short on assessment.
• The group wants to know how an LLM works. Fair question, wrong Short. That one is separate in the series.

Variants

In 60 minutes. One statement instead of two, and rather than handing out the Taskforce advice, read it aloud in five minutes with the question of what is missing from it. Keep the opening exercise and the Companion block at full length; those are the two parts where the group does something itself.

Online. The line-up becomes a 1-to-5 poll, and that costs you the visible shifting. Replace it with the same poll twice and the results side by side. The pairs work in breakouts and that goes fine; do ask them to share their screen as soon as Gemini Notebook baulks.

For a programme meeting. Drop the Companion block and use the twenty minutes for the question of where in the curriculum each item on the list belongs. It stops being a Short and becomes a working session, which for a programme committee is the more useful form.

The VU CTL Shorts series

This session is the first of fifteen Shorts by the VU Centre for Teaching & Learning, in four groups. Only this one has been written up as a lesson plan so far.

Core concepts

• AI literacy among students: what teachers need to know - this session.
• GenAI & AI literacy for teachers: an introduction - the same question one row back: what do you need to be able to do yourself in order to make students AI literate. The obvious next step after this Short.

Knowledge and skills

• Prompting for beginners - building a prompt and seeing what each addition changes.
• Powerful prompting - beyond the first question: roles, steps, examples, and having the model improve its own prompt.
• Can you use GenAI to retrieve and process information? - where the model replaces a source and where it invents one.
• LLMs and technical applications - what happens under the bonnet, for those who want to go past the chat window.

Designing and strengthening education

• Impact of GenAI on learning - what happens to learning when part of the thinking is handed over.
• Impact of GenAI on academic integrity - what plagiarism and one's own work still mean.
• Impact of GenAI on assessment - which forms of assessment hold and which do not.
• Using GenAI for educational design - objectives, assignments and materials with AI alongside.
• Can you use GenAI for feedback and grading? - where it helps, and where it takes over a judgement that has to stay yours.
• Bring your teaching to life: AI roleplay - a conversation partner playing a patient, a client or an opponent.
• Powerful teaching with Socratic bots, quiz bots and debate bots - bots that ask questions instead of giving answers.

Broad and societal impact

• Environmental impact of GenAI - energy and water, per question.
• Societal impact of GenAI - labour, power and access.

Material and origin

• Based on the slides Short 1 - What teachers need to know about students' (lack of) AI literacy, VU Centre for Teaching & Learning, 22 slides.
• Download the slides (PowerPoint, 357 kB). The slides are in English and carry no speaker notes; this plan is the reconstruction of what happens between them.
• The number of statements has been cut here from nine to two. That is a deliberate departure from the slides, on the evidence of how the first run went.
• For activities to run with students rather than with colleagues: the AI literacy exercises. Shorter advice sits with the didactic tips on generative AI.