Divergent Study Results Highlight Unpredictable Impact of AI in Academic Settings
Dual studies from the University of Maryland reveal conflicting outcomes for artificial intelligence in education. While one experiment showed potential for math gains in middle schoolers, another found that college students using an AI assistant experienced lower grades and reduced engagement, according to reporting by Hechinger Report.
Context from Industry Pulse
The contrasting findings highlight a critical challenge in educational technology: the gap between a tool’s intended design and its actual usage by students. While platforms like Goblins and ASSISTments show that teacher-controlled, interactive tools can support younger learners, the university study suggests that older students may use AI to bypass deep engagement. This phenomenon is complicated by the five-percent problem, where most educational software fails to achieve consistent adoption. Furthermore, the decline in office hour attendance and peer collaboration at the university level indicates that poorly integrated AI might inadvertently discourage the very social and instructional interactions that are essential for academic success, particularly for vulnerable student populations.
Research led by Jing Liu at the University of Maryland indicates that the effectiveness of educational artificial intelligence depends heavily on its implementation and the age of the user. As Hechinger Report notes, an analysis of the Goblins math platform involving 6,000 middle school students in Arizona suggested that consistent use could lead to notable academic progress. Students who completed at least two problems per week showed math scores equivalent to three additional months of learning, though researchers noted these gains might also reflect the higher motivation of students who chose to use the tool.
This middle school platform offered specific features that appeared to drive engagement, particularly for students with disabilities and English learners. The Hechinger Report describes how the software allowed students to interact with a cartoon avatar using voice and text in multiple languages. This personalized approach helped overcome the common industry challenge where only five percent of students typically engage with educational technology as directed. By allowing teachers to remain in control of the assignments rather than automating the entire process, the tool maintained a connection to classroom instruction.
In contrast, a randomized controlled trial at the University of Maryland involving 2,300 students across various subjects yielded negative results. Students in courses equipped with an AI chatbot saw their average grades drop by 0.11 points on a four-point scale. These students were also less likely to participate in class discussions, attend office hours, or meet with their peers. The Hechinger Report points out that these detrimental effects were most significant among first-generation college students, raising concerns about the technology potentially widening equity gaps.
Researchers suggest the differing outcomes may stem from how students perceive the technology. While the middle school tool acted as a guided tutor, college students often treated the university assistant as a shortcut for finding answers rather than a learning aid. According to Hechinger Report, the mere presence of the bot on the course website may have inadvertently signaled that using AI was a substitute for traditional study habits and instructor interaction.
The Industry Pulse Bottom Line
AI is not a universal educational solution; its success relies on preventing students from using it as a shortcut while ensuring it complements rather than replaces instructor engagement and student collaboration.