After 600 experiments, here's what I discovered about JEPA world models

273 points · 45 comments on HN · read original →

Points and comments are a snapshot, not live.

JEPA world models' planning bottlenecks are not prediction errors but goal scoring and subgoal chaining.

After 600 experiments on LeWM, a JEPA world model for Push-T, the author found that swapping the learned predictor for perfect physics barely improved long-horizon success (78% vs 77%). The real gains came from four zero-training planning changes: chaining 25-step subgoals raised success from 22% to 80%; a memory graph with per-jump costs matched expert-subgoal performance at 85%; transformed demonstration segments recovered success from 32% to 85% when goal-near states were deleted; and ignoring the pusher position in the goal cost quadrupled success (9/50 to 39/50) on a harder test. The author argues that JEPA's embedding space already predicts as well as the simulator, so further improvements depend on planning structure, not model accuracy.

What commenters are saying

Commenters praised the clear writing and animations but split on novelty. Some argued the techniques (hierarchical planning, subgoals, action chunking) are well-known from MPC and classical robotics, calling the post a personal learning journal rather than a discovery. Others found it a useful hands-on exploration. Several noted the website's background animation caused issues on OLED screens and mobile Firefox. A control-theory commenter likened JEPA to Model Predictive Control with a latent-space model. One commenter pointed out that MCTS also solves Push-T without training.