In ChatGPT vs. Actual Doctors, an 'Unexpected' Result

Scriptwriters, lawyers, and telemarketers are all warily eyeing ChatGPT, an artificial intelligence chatbot that may soon start horning in on their jobs. Doctors might want to keep tabs on how the AI tool could affect their industry as well, especially after new research found that ChatGPT had a better "bedside manner" with patients than some human doctors did, reports the Guardian. In the study published Friday in the JAMA Internal Medicine journal, researchers found that the AI tool impressed patients more with its empathy and the quality of its response when answering public questions in writing than real doctors did.

The researchers looked to the Reddit AskDocs forum for their original data, which consisted of 195 publicly posed medical questions that received answers from actual physicians. The scientists then fed those same questions into ChatGPT, which spit out its own answers. A three-person panel of licensed health-care professionals was then asked to judge which answers were of the highest quality and exhibited the most empathy. In terms of quality, the judges deemed the answers from ChatGPT to be "good" or "very good" responses 79% of the time, while doctors notched only a 22%. In terms of empathy, ChatGPT dominated again: Its answers were rated "empathetic" or "very empathetic" 45% of the time, while doctors' replies came in at a measly 5%.

"ChatGPT provides a better answer," the University of California San Diego's John Ayers, one of the study's co-authors, tells MedPage Today. "It clearly shows that ChatGPT wins in a landslide compared to physicians, and I wouldn't say we expected that at all." Some warn that the study's findings don't necessarily mean that doctors should soon expect to be replaced by robots, as AI bots are still having trouble getting their facts straight. However, the research does point to a possible path for AI medical assistants down the road, perhaps in helping doctors communicate with patients, including by answering messages—a key contributor to doctor burnout. "The opportunities for improving health care with AI are massive," says Ayers, per the Guardian. (More discoveries stories.)