Does AI Tutoring Actually Help Students Learn?

Does AI Tutoring Actually Help Students Learn? One Paper a Day, Evidence Verdict. Today is different from our usual episode. Instead of reviewing one study, we take one question people are actually asking and answer it from the weight of the evidence: does using AI as a tutor really help students learn? This is independent educational commentary about research, not advice for a specific student, teacher, or product. Every study here describes groups of people in particular settings, not a promise about your child or your classroom. I will state that boundary once, here, and then trust you to carry it. ## 1. The question and the verdict Here is the verdict first, so you have it even if you stop after this minute. A well-designed AI tutor, one with guardrails that make it coach rather than answer, can help students learn as much as, and sometimes more than, an ordinary class. But the very same technology, handed over as a chatbot that simply gives answers, can make students do worse once it is taken away, while leaving them feeling more capable, not less. The design of the tutor, not the mere presence of artificial intelligence, decides whether learning goes up or down. The biggest caveat, in the same breath: almost all of the strong evidence measures learning immediately, under supervision. Whether these gains last for months, transfer to new problems, or survive a student alone at home with a general chatbot, is largely unknown. So the honest headline is not "AI tutors work" and not "AI makes students lazy." It is "it depends, and it depends on things you can actually check." Who should care? Parents deciding whether to trust a tutoring app, teachers deciding how to let students use these tools, and anyone building or buying education technology. If you were hoping for a simple yes or no, the useful answer is more valuable than that, because it tells you what to look for. ## 2. Why this question matters now AI tutors are being deployed to millions of students right now, sold on a genuinely old and powerful promise: personal tutoring for everyone. The marketing often rests on a single viral study or a glossy demo. That is exactly the situation where a careful look at the whole body of evidence earns its keep, because individual studies point in different directions, and the differences are not random. They track the design of the tutor and the way it is used. There is also a deeper reason. Learning and performing are not the same thing. A tool can make you look better today while teaching you less for tomorrow. That gap sits at the center of this whole question, and it is where a lot of intuition goes wrong. ## 3. The field so far To judge whether today's AI tutors are a breakthrough, you have to know the bar. And the bar is old. Start with the aspiration. In 1984, the educational psychologist Benjamin Bloom described what became known as the two-sigma problem. Students tutored one-to-one, he reported, performed about two standard deviations above students in an ordinary class. Two standard deviations would move an average student near the top of the room. That number has echoed through education for forty years. But treat it as an aspiration, not a settled fact: it came from small studies and has never robustly replicated. It tells us tutoring is powerful. It does not tell us to expect a doubling from any tool that calls itself a tutor. Now the reality check. In 2011, Kurt VanLehn reviewed decades of experiments comparing human tutors, computer tutors, and no tutoring. His finding reset expectations twice over. Human tutoring was not the mythical two sigma; it landed closer to an effect size of about zero point seven nine. And the older intelligent tutoring systems, software built long before large language models, reached about zero point seven six. In other words, well-built computer tutors were already roughly as effective as human tutors, and both delivered something closer to eight tenths of a standard deviation than to two. That is the crucial context. The question "can a computer tutor?" was answered years ago, and…

Continue with the full plain-text transcript.