A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis

614 points · 259 comments on HN · read original →

Points and comments are a snapshot, not live.

A 10-year-old CNN beat GPT-6 Astra and Claude Fable 5.1 at diagnosing skin cancer from images.

Dermatologist Magnus Lynch tested GPT-6 Astra, Claude Fable 5.1, and a fine-tuned Inception v3 (2015) on 200 biopsy-proven skin lesions. The CNN got 62% correct first-choice diagnoses; the LLMs got 42%. The CNN correctly identified 13 of 30 melanomas, versus 8 and 11 for the LLMs. The LLMs often defaulted to specific wrong answers (e.g., GPT-6 Astra called 75 lesions basal cell carcinoma when only 25 were). The CNN is small enough to run in a phone browser via WebGPU. The author notes caveats: small sample size, hard cases (all biopsied), and the CNN is not state-of-the-art.

What commenters are saying

Many commenters found the result unsurprising, comparing it to using a calculator for math versus an LLM. Several noted LLMs are general-purpose tools not optimized for narrow classification tasks. A few criticized the comparison as unfair, arguing a fine-tuned specialized model should beat general models, and that an LLM with proper tool use could leverage the CNN or train its own. Others appreciated the reality check against hype around LLMs being universal problem-solvers. One commenter observed that many people reflexively assume LLMs excel at everything and need reminders like this.