AI in Education: Homework Gains, Exam Losses, and What Builders Should Know

A new study tracking nearly 27,000 Chinese students aged 12-18 found that AI use boosted homework scores by 18% and cut assignment time by 30%, but those same students scored 20% lower on monthly exams compared to peers who didn't use AI. For builders of educational AI tools, this signals a critical design tension: optimizing for task completion can mask learning losses.

The Study: What the Numbers Show

Researchers from Stockholm University and the University of Hong Kong tracked students in grades 7-12 over six months. About 80% reported using AI models like DeepSeek and ByteDance's Doubao, while the remaining 20% formed a control group. The AI users saw homework scores rise 18% across all subjects and assignment time drop from 64 to 45 minutes on average, a 30% reduction. But their monthly exam scores were 20% lower than non-users, upending the usual correlation between homework and exam performance.

Anecdotal evidence from Brown University and MIT reinforces these concerns. A Brown professor noticed a take-home midterm average in the high 90s with half the class getting perfect scores; when the final was moved in-person, the average collapsed to 48%, compared to a historical low of 65%. An MIT study found that students who used AI to write essays showed lower brain activity and struggled to quote their own work, and the reduced brain activity persisted when they later wrote without AI help.

Why This Matters for AI Product Builders

The study challenges the assumption that efficiency gains equal learning gains. AI tools that help students finish homework faster and score higher on assignments may actually be undermining retention and critical thinking. The worst learning losses appeared in social science subjects, followed by STEM and language courses. Notably, AI users who spent about the same time on homework as non-users had smaller losses, suggesting that time-on-task and cognitive engagement still matter.

For builders, this means that metrics like homework completion time and accuracy are insufficient proxies for learning. Products that optimize for these surface-level signals may inadvertently encourage overreliance on AI, creating a gap between what students can produce with assistance and what they can demonstrate independently.

Practical Implications for EdTech Design

Educational AI products should include measurement hooks that track retention and transfer, not just task completion. For example, periodic low-stakes quizzes without AI access could serve as a check on whether students are actually learning. Policies may need to distinguish between AI-assisted practice and assessment settings to preserve academic integrity. The study also adds to concerns about grade inflation: recent research found that A grades in college courses vulnerable to AI cheating have surged 30% since ChatGPT's release.

Builders should consider designing cognitive scaffolds that encourage students to engage with material before turning to AI, or that require explanation of AI-generated answers. Products that treat AI as a tutor rather than an answer engine may better support long-term learning.

Caveats and What We Don't Know

This study shows correlation, not causation. The results come from a specific population in China using particular AI models, and may not generalize to other age groups, tools, or educational systems. The Brown and MIT anecdotes are observational and not controlled experiments. Effects likely vary by subject, implementation, and how students use the AI. Builders should avoid overgeneralizing but take the pattern seriously: AI can boost short-term productivity while eroding the learning it's meant to support.

The takeaway for builders is clear: AI in education can't be evaluated on homework metrics alone. Products that optimize for speed and surface accuracy without measuring deeper learning may be doing more harm than good. The next wave of educational AI needs to build in cognitive scaffolds and assessment alignment from the start.