Doctors often give the following advice to ordinary people: Don’t Google itThe search giant tends to be the first port of call for people looking for answers to all their health-related questions. Why is the scab oozing and what are these pink bumps on my arms? When you search for your symptoms, WebMD and other sites may present you with an overwhelming list of possible causes for your illness. The experience of panicking over something you find online is so common that researchers have a word for it. Cyberchondria.
Google has introduced a new feature that essentially lets it play the role of doctor. The search giant has previously surfaced snippets of text at the top of search results, but now its generative AI goes a step further. As of last week, the search giant The “AI Overview” feature will be rolled out to all U.S. citizensIt’s one of the biggest design changes in recent years: Many Google searches now show AI-generated answers just below the search bar, above links to external websites, including for health questions. Can you die from too much caffeine?Google’s AI overview offered a four-paragraph answer, citing five sources.
But it’s still a chatbot. In just a week, Google users have pointed out all sorts of inaccuracies in the new AI tool. Dogs who played in the NFL and that President Andrew Johnson earned 14 degrees from the University of Wisconsin-Madison.Health-related responses were no exception, with many answers that were obviously wrong or just plain weird. Are rocks safe to eat?. Chicken is safe to eat when it reaches 102 degreesThese search blunders might be amusing when they’re harmless, but when more serious health questions are handled by AI, Google is playing a dangerous game.
Google’s AI summaries don’t appear for every search, but that’s by design. It goes without saying that “which laptop should I buy” is a lower-risk query than “do I have cancer.” Even before introducing AI search results, Google said it treats health queries with special care and shows the most reliable results at the top of the page. “AI summaries are rooted in the quality and safety systems that are core to Google Search,” a Google spokesperson said in an email. “We hold ourselves to an even higher quality bar when we show AI summaries for health queries.” Google also said it strives to show summaries only when its systems are most confident in the answer. Otherwise, it just shows regular search results.
I tested the new tool this week with over 100 health-related questions, and most of the questions were given an AI summary, even the more sensitive ones. Google tendencyswhich revealed what people actually tend to search for when it comes to specific health topics. Google’s search bot offered advice on how to lose weight, how to get diagnosed with ADHD, what to do if your eyeballs are popping out of their sockets, whether tracking your menstrual cycle can help prevent pregnancy, how to know if you’re having an allergic reaction, what that strange swelling on the back of your arm is, how to know if you’re dying, and more. (Some of the AI responses I found have since been changed or no longer appear.)
To be honest, not all advice is bad. Signs of a Heart Attack The AI then provided a generally correct summary of symptoms such as chest pain, shortness of breath, and lightheadedness, citing sources such as the Mayo Clinic and the CDC. But health is a sensitive area for tech giants to operate something still in the experimental stage. At the bottom of some of the AI’s responses is a small text that reads, “This tool is for informational purposes only. For medical advice or diagnosis, consult a professional. Generative AI is experimental.” Many of the health questions could cause real-world harm if even a part of the answer is wrong. Aside from anxiety-inducing AI responses about illnesses you don’t have, what about the consequences of missing the signs of an allergic reaction, for example?
Even though Google limits its AI summary tool in certain areas, it’s possible that some searches will slip through. Sometimes, presumably for safety reasons, it will refuse to answer a question and then answer a similar version of the same question. For example, Is Ozempic safe? There was no response from the AI, Should I take Ozempic? When it comes to cancer, the tool is similarly finicky: It wouldn’t tell me the symptoms of breast cancer, but it did tell me when I asked about symptoms of lung and prostate cancer. When I tried again later, it reversed course and listed the symptoms of breast cancer as well.
AI Overview did not show up for some searches, no matter how the query was phrased. The tool did not show up for queries containing the following words: COVIDAnd when I asked about drugs like fentanyl, cocaine, or marijuana, Google would sometimes silence me and prompt me to call a suicide or crisis hotline. The risk with generative AI isn’t just that Google will spit out obviously wrong, cringey answers. AI research scientist Margaret Mitchell said: Tweeted“This is not a ‘gotcha’, it’s to point out clearly foreseeable harm.” I would hope that most people would know that you shouldn’t eat rocks. The bigger concern is small sourcing and errors of reasoning, especially when someone is just Googling for a quick answer and is likely to only read the AI synopsis. For example, I was told that pregnant women can eat sushi as long as it doesn’t contain raw fish. This is technically true, but essentially all sushi contains raw fish. When I ask about ADHD, AccreditedSchoolsOnline.organ unrelated website about school quality.
If you search on Google How effective is chemotherapy? According to the AI summary, the one-year survival rate is 52 percent. This statistic is Genuine scientific papersBut this is for head and neck cancer, where patients who don’t receive chemotherapy have a much lower survival rate. all cancer.
In some cases, a search bot might be really helpful. Sifting through a huge list of Google search results is a pain, especially when compared to a chatbot response that summarizes it for you. The tool might even improve over time. Still, it might never be perfect. At Google’s scale, content moderation is very hard even without generative AI. Last year, one Google executive told me that 15% of daily searches are things the company has never seen before. Right now, Google Search suffers from the same problem as any chatbot: companies can create rules for what they should and shouldn’t respond to, but they can’t always apply them precisely. “Jailbreaking” ChatGPT with creative prompts has become a game in itself. There are countless ways to phrase a particular Google search, and countless ways to ask questions about your body, your life, and the world.
If these AI-generated summaries seem so inconsistent in the area of health advice, where Google is putting all its efforts, what about every other search?
