AI chatbots did not “defeat” doctors at diagnosing illness
A recent New York Times article titled AI Chatbots Defeated Doctors at Diagnosing Illness covers a study recently published in JAMA Network Open. The main point of the study was to determine whether access to an LLM could help doctors diagnose more effectively. The outcome? Physicians using an LLM did not perform any better than physicians working alone. But this isn’t the part of the study that generated the most buzz. Instead, the news has focused on a secondary discovery of the study, which was that an LLM alone outperformed physicians at the diagnosis step. What’s going on here?
What’s going on is that even for the cases where the LLM “did the diagnosis” it was, in fact, not doing the diagnosis by itself. What do I mean? The LLM was fed case reports, carefully crafted by doctors to include all of the information needed to make the diagnosis, and the LLM just had to do the very final step of outputting what that diagnosis was, without doing any of the diagnostic work that went into creating the case report in this first place. This aspect of the study has been swept under the rug, but it is necessary to properly contextualize what this study implies about LLM capabilities.
The study itself states that the cases provided to the LLM were “based on actual patients” and “included information available on initial…
