How can a chatbot support students in lectures and with assigned exercises? Researchers at the Technical University of Munich (TUM) have developed the chatbot Iris, which offers informatics students personalized assistance with programming assignments.

A study has now confirmed the chatbot's success: Iris improves the understanding of programming concepts and represents a valuable complement to human tutors. The study is published in the journal Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1.

A packed auditorium with over 1,000 students. This is not a rare sight in introductory informatics lectures. To meet the needs of each individual student under these conditions, Stephan Krusche, professor of Software Engineering, and his team have been building the Artemis learning platform since 2016. It resembles well-known learning platforms, but offers more possibilities.

For example, students not only receive their assignments through the platform, but also get immediate feedback when they make mistakes or show an incomplete understanding of concepts.

Chatbot Iris: Excellent tutor

In October of last year, Stephan Krusche's team at the TUM Centre for Educational Technologies added the chatbot to the platform. Iris is based on the GPT large language model of OpenAI. It is intended to help students with assigned problems, with the support geared to their individual needs.

To adapt the system to the learning context, the chatbot operates as a so-called excellent tutor. That means that, rather than immediately suggesting the right solution or specific steps, Iris offers subtle hints or asks leading questions. This approach encourages students to think for themselves. To boost learning efficiency, the chatbot only answers questions directly relevant to the learning content.

"To test the success of Iris, we surveyed over 200 students from three introductory informatics lectures," says Patrick Bassner, a doctoral candidate at the Chair of Software Engineering.

"The goal was to arrive at conclusions on how students rate the effectiveness of the support from Iris, whether students are more comfortable asking a chatbot questions than a human tutor and whether students subjectively trust Iris."

Iris promotes learning and students' independence

The responses show that the students value Iris as a valuable support, but also that a large share of those surveyed still rely on human tutors. The chatbot understands questions well and students feel comfortable and safe when using it. Iris gives them the feeling that the chats will remain private and that they are not being judged when asking questions.

Especially in large lectures, this saves students from having to overcome their inhibitions to ask questions in front of the class. When studying for exams, the use of Iris is becoming commonplace. However, students feel confident enough to solve problems on exams without the support of the chatbot.

"The integration of Iris into Artemis has shown that AI-controlled tutors can play a valuable role in university teaching," says Stephan Krusche. "They offer personalized support that benefits learning and boosts students' ability to help themselves."

The researchers plan to further optimize the chatbot Iris and want to see it being used in other subject areas.

More information: Patrick Bassner et al, Iris: An AI-Driven Virtual Tutor for Computer Science Education, Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 (2024). DOI: 10.1145/3649217.3653543