NC State University researchers are applying data mining techniques to analyze the effectiveness of — and make real-time improvements to — educational games. They’re creating software tools that use real-time data to assess how well students develop collaborative problem-solving skills (CPS). These algorithm-based tools can enable educational games to be more responsive and effective for learning.
The software uses constraint-based pattern mining algorithms to assess and predict learning outcomes — techniques that have yet to be applied widely in an educational context.
“One of the attractive things about this framework is that it can evolve as more students play the game and the software has more data to draw on,” says Wookhee Min, co-author of a paper on the work and a senior research scientist at NC State. “In theory, that should allow us to fine-tune in-game interventions in order to improve learning outcomes even more.”
Currently, methods exist to help students develop collaborative problem-solving skills, but testing students to measure their performance can be disruptive.
The solution was to develop software that measured the cognitive aspects of CPS without intruding on game-playing. Based on a data-driven framework, the software can identify behavioral patterns.
In a study, 61 middle school students played EcoJourneys, a game-based learning environment designed to improve collaborative problem-solving emphasizing natural sciences. In teams, students try to figure out the cause of an illness spreading among the tilapia fish on a local farm. The game includes three quests, which are phases that support inquiry and collaboration. Students explore the game’s environment during the Talk & Investigate phase, interacting with non-player characters and viewing videos. In the Deduce and Explain phases, students apply their collaborative problem-solving skills.
In the study, researchers analyzed the raw data of student trace logs. Using the software and statistical modeling, they were able to find pedagogically relevant behavioral patterns and understand more about the effectiveness of how the game teaches CPS.
The software can also modify the game in real time to adapt to student learning.
“For example, if students make a specific series of choices at one stage in the game, that may suggest that they are not grasping some key CPS concepts,” says Halim Acosta, first author of the paper and a Ph.D. student at NC State. “We could modify the game so that if students make that series of choices, the game changes in a way that emphasizes or reinforces those concepts.”
The research was presented at the Fifteenth International Learning Analytics & Knowledge Conference (LAK25), held March 3-7 in Dublin, Ireland. Additional co-authors from NC State include research scientist Seung Lee, senior research scientist Bradford Mott, and James Lester, the Goodnight Distinguished University Professor in Artificial Intelligence and Machine Learning at NC State and director of the university’s Center for Educational Informatics. The work was done with support from the National Science Foundation under grants 2112635, 1561486 and 1561655.
This article is based on a news release from NC State University.
