New AI tutor achieves 0.71-1.30 SD effect size in Dartmouth course [pdf]
Points and comments are a snapshot, not live.
AI tutor Phosphor raised exam performance by 0.71-1.30 SD in a Dartmouth statistics course.
Phosphor embeds LLM-graded quizzes (MCQ and constructed-response) into digital lessons. Deployed across three sections of Introductory Statistics at Dartmouth, 90.2% of 151 students adopted the optional platform, vs. 10-15% baseline reading compliance. Full dosage (24 lessons, 3 module reviews) correlated with 1.30 SD higher final exam scores (0.71 SD after controlling for midterm scores). MCQ-only quizzes showed negligible learning benefit; constructed-response questions were critical. The RAG chat assistant saw minimal use (72 queries total).
What commenters are saying
Commenters highlighted the 90% adoption rate as the standout result, noting that even imperfect AI tools can outperform traditional textbooks that few students read. Several pointed out selection bias as a limitation, though the paper attempts to control for it. Others discussed the challenge of applying this to subjective subjects like literature, where LLM grading may be unreliable. A subthread debated the value of spaced repetition and the practicality of capturing handwritten notes for AI analysis.