What happens when an LLM never sees material beyond fifth grade?
Points and comments are a snapshot, not live.
LLMs trained only on K-5 material reveal post-training cannot exceed the pretraining data ceiling.
LittleLearner project trains 0.6B-5B models on an 88B-token corpus filtered to U.S. K-5 curriculum (Common Core). Results show scaling, SFT+GRPO post-training, and in-context learning amplify in-scope abilities but do not improve out-of-scope performance. The pretraining filter sets an effective capability ceiling that no intervention overcomes.
What commenters are saying
Commenters debate whether examples given actually reflect K-5 material: one user notes the model's explanation of quantum entanglement is wrong but confidently stated, while another argues why the sky is blue appears in children's science books. Some discuss that LLMs lack metacognition to say "I don't know," as training data rarely contains such phrases. A small camp suggests this inability to refuse undermines trust.