AI Health Chatbots Fall Short on Real-World Medical Advice

UK researchers tested GPT-4o, Llama 3, and Command R+ against a control group and found AI health chatbots identified health problems only about one-third of the time and suggested a correct course of action roughly 45% of the time—no better than internet searches. They attribute the gap to miscommunication between humans and AI and warn that chatbots can give wrong diagnoses or miss urgent care cues. With about 1 in 6 US adults asking chatbots about health monthly, the study argues AI isn’t ready to replace clinicians.