HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88
Points and comments are a snapshot, not live.
HackerRank's open-source ATS returns wildly inconsistent resume scores across repeated runs.
The author tested HackerRank's open-source ATS (interviewstreet/hiring-agent) with the same resume and found scores ranging from 66 to 99 over 100 runs at temperature 0.1. The tool uses an LLM (gemma3:4b by default) to score applicants on open source contributions (35 points), personal projects (30), work experience (25), and technical skills (10), plus 20 bonus points. Work experience consistently scored 25/25 regardless of actual experience. Projects showed high variance due to LLM judgment inconsistency. Even at temperature 0, scores remained non-deterministic. The author warns that such tools filter resumes arbitrarily rather than by quality.
What commenters are saying
Commenters largely condemned the tool's scoring rubric, noting that weighting open source and personal projects at 65% unfairly filters out experienced engineers who don't code in their free time. Some argued that high application volume makes any automated screening inevitable, comparing it to random filtering. Others pointed out that LLMs' stochastic nature makes them unsuitable for deterministic hiring decisions, with one commenter clarifying that temperature 0 does not guarantee deterministic outputs. A minority suggested that even a 35% pass rate improves over manual review of hundreds of applicants, but this was met with skepticism about validity.