AI Chatbots Under Scrutiny as Study Warns of Risky Medical Advice

A growing number of people are turning to artificial intelligence for answers about their health, but a new study suggests that this trend could carry serious risks. Researchers have found that widely used AI chatbots, including ChatGPT, Gemini, Meta AI, Grok, and DeepSeek, are not always reliable when it comes to medical advice.

The findings, published in the BMJ Open, highlight that nearly half of the responses generated by these AI systems contained incorrect or misleading medical information. The study evaluated how these tools responded to a range of health related questions, raising concerns about their increasing use in sensitive areas like diagnosis and treatment guidance.

According to the research, about 20 percent of the answers were classified as highly inaccurate. These responses were not just slightly off but potentially dangerous if followed without verification. The issue becomes more concerning given that many users treat these chatbots as informal doctors, relying on them for advice on symptoms, medications, and even serious conditions.

The study involved researchers from the United States, Canada, and the United Kingdom, who tested the chatbots using structured and open ended questions across five medical categories. Interestingly, the AI models performed relatively better when answering closed ended questions on well established topics such as vaccines and cancer. However, their accuracy dropped significantly when faced with more complex or open ended queries, including nutrition and emerging treatments like stem cell therapy.

Another major concern raised in the report is the confident tone used by these AI systems. Despite lacking medical licenses or clinical judgment, chatbots often present their answers with authority. In many cases, they fail to provide complete or verifiable medical references to support their claims, which can mislead users into trusting incorrect information.

Researchers also noted that the chatbots rarely refused to answer medical questions. Out of all the queries tested, only two instances of refusal were recorded, both from Meta AI. This willingness to respond, even when uncertain, increases the risk of misinformation spreading among users who may not cross check the advice.

The timing of the study is significant, as major tech companies are actively expanding their presence in healthcare. For example, OpenAI recently introduced a health focused version of ChatGPT that allows users to share personal health data for more tailored responses. Similarly, Anthropic has launched Claude for Healthcare, aimed at helping users securely integrate medical records with AI driven insights.

Experts warn that without proper regulation and public awareness, the use of AI chatbots in healthcare could do more harm than good. While these tools can offer general information, they are not a substitute for professional medical advice. The study emphasizes the need for clearer guidelines and more responsible deployment of AI in health communication.

As AI continues to evolve, the gap between convenience and accuracy remains a critical challenge, especially when human lives may be at stake.