For years, educators have sought to glean lessons about learners and the learning process from the data trails students leave behind with every click. digital textbook, learning management system or other online learning tools. It is an approach known as “.learning analysis”
Recently, learning analytics proponents have explored how the emergence of ChatGPT and other generative AI tools brings new possibilities and raises new ethical questions for learning analytics practices. .
One possible application is using new AI tools to help educators and researchers understand all the student data they collect. Many learning analytics systems feature dashboards that provide metrics and visualizations about learners to teachers and administrators based on the use of digital classroom tools. The idea is that data can be used to intervene if students are showing signs of becoming unmotivated or falling off track. However, many educators are not used to sorting through large amounts of this type of data and may have difficulty navigating these analytical dashboards.
“AI-powered chatbots will become a kind of intermediary or translator,” said Zachary Pardos, an associate professor of education at the University of California, Berkeley. Special issue of Journal of Learning Analytics We plan to specialize in generative AI in the field. “Chatbots can be infused with 10 years of learning science literature,” he said, adding that they can help analyze and explain in plain language what you see on your dashboard. I added.
Advocates of learning analytics are also using new AI tools to help analyze online discussion boards from courses.
“For example, if you’re looking at a discussion forum and want to mark a post as ‘on topic’ or ‘off topic,'” Pardos says. Previously, it took much more time and effort for human researchers to track posts. Rubrics can be used to tag such posts and to train old-fashioned computer systems to categorize material. But now, Pardos says, large-scale language models make it easy to mark discussion posts as on-topic or off-topic “with minimal rapid engineering.” In other words, by giving ChatGPT a few simple instructions, the chatbot can categorize vast amounts of student work and turn it into numbers that educators can quickly analyze.
Learning analytics research results are also being used to train new generative AI-powered tutoring systems. “Traditional learning analytics models can track a student’s knowledge proficiency level based on their digital interactions, and this data can be vectorized and input into an LLM-based AI tutor to improve the AI tutor’s ability to interact with students. It can improve relevance and performance,” says Mutlu Kukurova, professor of learning and artificial intelligence at University College London.
Another big application is evaluation, Berkeley professor Pardos says. Specifically, new AI tools can improve the way educators measure and score student progress through course materials. New AI tools are expected to replace many multiple-choice questions in online textbooks with fill-in-the-blank or essay questions.
“The accuracy with which LLMs can score open-ended responses appears to be about the same as humans,” he says. “Thus, more learning environments now accommodate more open-ended questions that allow students to exercise more creativity and different kinds of thinking than if there was a single definitive answer required. You can see that it is possible.”
Concerns about prejudice
However, these new AI tools bring new challenges.
One problem is algorithmic bias. Issues like this were already a concern before the rise of ChatGPT. Researchers worried that if the system predicted that a student was at risk based on large amounts of data about past students, it could end up perpetuating historical inequalities. In response, there were calls for greater transparency into the learning algorithms and data used.
Some experts are concerned that new generative AI models are like the editors of the Journal of Learning Analytics. phone Many AI experts believe that ChatGPT and other new tools are also culturally sensitive in ways that are difficult to track and address, due to “a significant lack of transparency in explaining how the output is produced.” and are concerned that it reflects racial bias.
Additionally, large language models are known to occasionally “hallucinate” and give factually incorrect information in some situations, making them reliable enough to be used for tasks such as assisting with student assessment. There are concerns as to whether this will be possible.
Shane Dawson, professor of learning analytics at the University of South Australia, says new AI tools are making the question of who builds the more powerful algorithms and systems more widespread when learning analytics becomes more widespread in schools and universities. It has become a pressing issue.
“There is a transfer of ownership and power happening at every level of the education system,” he said. in recent stories. “In a classroom, a K-12 teacher will be sitting there teaching a child to read and handing them an iPad. [AI-powered] When you turn on the app, it makes recommendations for that student. Who has the power this time? Who has ownership in that classroom? These are questions that we need to address as a field of learning analytics. ”
