Dayjob: Tax – evaluating AI agents on 50 professional tax assignments
Points and comments are a snapshot, not live.
AI agents pass only 10.8% of professional tax assignments in new benchmark.
Surge AI's DAYJOB: Tax benchmark evaluates AI agents on 50 professional tax assignments across eight areas. The top model, Claude Sonnet 5.5, passes only 10.8% of tasks at a 95% rubric threshold. Tasks require agents to navigate messy records, identify issues clients missed, and exercise professional judgment. The benchmark includes 1,789 workspace files (1,487 PDFs, 117 Excel workbooks). Example failures include an agent missing a key deadline for a home-sale tax exclusion, and another assuming a couple was unmarried despite records suggesting otherwise. Existing benchmarks TaxBench, TaxCalcBench, and TaxLab test narrower skills.
What commenters are saying
Commenters share personal experiences with accountant errors costing thousands. Several note the filer ultimately bears liability for tax return mistakes, regardless of preparer. Some advocate doing taxes oneself after repeated bad experiences with professionals. One CPA defends the industry, arguing audits stem from data matching or statistical selection, not preparer negligence. Privacy concerns about sharing financial data with LLM companies are raised, with some suggesting local inference or using AI only for reviewing known routines.