90% of US working hours are spent on tasks not covered well by AI benchmarks
Points and comments are a snapshot, not live.
90% of US working hours go to tasks poorly covered by AI benchmarks.
Mercor mapped 29 AI benchmarks against O*NET's 923 occupations and 18,797 tasks. Only 208 occupations had a task with at least 5 benchmark prompts. Benchmarks overrepresent large-wage, AI-exposed occupations and higher-importance tasks, leaving long-tail middle-income knowledge work undercovered. Even after controlling for AI exposure, tasks from larger wage bills are significantly more likely to be benchmarked.
The findings suggest it's hard to know how good AI is at the work Americans actually do. Mercor's O*NET-weighted scoring on their APEX benchmarks for four occupations changed scores little vs. simple averages, but less than 1/3 of relevant O*NET tasks had enough coverage for weighting.
What commenters are saying
Commenters broadly agree that AI benchmarks miss most real work, but they split on causes and implications. Many note that software engineers overestimate AI's reach: LLMs are uniquely suited to coding but not to office politics, stakeholder management, and the non-coding work of getting features to production. One commenter estimates 50% of their job is office politics, 30% stakeholder management, and 10% actual development. Another argues AI's productivity gains come mainly from reducing labor inefficiency tied to human capital problems, not from technology advances. Some worry AI will make office politics worse as workers try to protect their roles.