According to a new study published in the scientific journal Nature Medicine, ChatGPT is not as reliable as people would have us believe: it could give us false good advice in many situations, especially in cases of absolute emergency.
The tool would be wrong more than half the time in the event of a critical situation
To arrive at these worrying conclusions, to say the least, researchers from the Icahn School of Medicine at Mount Sinai (New York) submitted to ChatGPT sixty medical scenarios covering 21 different specialties and several levels of emergency. Some situations described were relatively benign, while others required rapid or even urgent treatment. In total, almost 960 interactions were examined.
Result ? ChatGPT only provided good recommendations in 48.4% of emergency situations, and only 35.2% of non-urgent cases. Clearly, when the situation becomes critical, the tool makes mistakes more than half the time. In some scenarios, artificial intelligence even advised simple monitoring for twenty-four to forty-eight hours, when the patient’s condition should have led to an immediate consultation in the emergency room.
An observation which hardly surprises Dr Gérald Kierzek, interviewed on this subject:
“ChatGPT is more than 50% wrong in absolute emergency situations because it lacks real clinical examination, nuanced interpretation of subtle signals and probabilistic judgment refined by human experience. It can thus underestimate certain serious cases or, on the contrary, overestimate them. But the whole art of medicine consists precisely in contextualizing…”, calls back the emergency doctor.
Another worrying point: ChatGPT would be incapable of having a good analysis (and a good reaction) during proven psychological problems. A concrete example: when users mentioned their suicidal thoughts, the tool did not necessarily redirect them to a crisis line. But once again here, Dr Gérald Kierzek does not appear surprised.
“For psychological distress, its limits in terms of contextual empathy and suicidal risk assessment require users to be reminded that they should never rely on an AI for urgent medical or psychological advice and that it is necessary to immediately call 15, 3114 (national suicide prevention number) or a health professional in a critical situation. AI does not replace either a doctor or a real examination…”,
underlines the medical director of True Medical.
Dr. Kierzek’s 5 tips for better using ChatGPT
AI like ChatGPT can be a complementary tool in health, provided it is never used to make a diagnosis. However, when used correctly, it allows you to better organize your daily medical monitoring. It can therefore prove very useful for:
- Prepare for a medical consultation: List your symptoms, your questions or your history in order to structure the discussion with your doctor. ChatGPT can also help you rephrase a complex medical report into clearer language;
- Managing a chronic illness: create a personalized medication schedule, as well as vaccine reminders, to then be validated by a professional;
- Understanding a pathology: ask the AI for simple explanations about an illness (such as diabetes) or the effects of a treatment in order to better understand and adhere to your care;
- Adopt preventive actions: find ideas for suitable menus (for example low in cholesterol) or lifestyle advice, to cross-reference with reliable medical sources;
- Strengthen your therapeutic education : obtain clear summaries of recommendations intended for patients (for example after a stroke) in order to strengthen your daily autonomy – without ever ignoring medical advice.
In summary, ChatGPT never replaces a healthcare professional and should not be used to make a diagnosis or modify treatment. But it can serve as a good support, in certain everyday contexts.