Google manually removes incorrect AI answers in Search

Recently, Google’s new artificial intelligence (AI) information summary feature called AI Overview is making social networks stir because it offers strange suggestions, such as putting glue on pizza or eating stone. This messy launch caused Google to hastily turn off manual AI Overview for certain searches to avoid spreading jokes online.

This is a confusing situation because Google has been testing AI Overview for a year – the feature was introduced in beta mode in May 2023 as the Search Content Creation Experience. CEO Sundar Pichai said the company served more than a billion queries during that time.

However, Pichai also revealed Google has reduced the cost of providing answers using AI by 80% over the same period, “thanks to hardware, engineering and technical breakthroughs”. It seems like this optimization happened too early, before the technology was really ready.

An anonymous AI founder told Techlade: “A company once known for being a pioneer and providing high-quality products is now famous for poor quality outputs, became an internet meme.”

Google still affirms that the AI ​​Overview product primarily provides “high quality information” to users. “Many of the examples we see are unpopular queries, and we also see examples that are edited or unreproducible,” Google spokeswoman Meghann Farnsworth said in an email to Techlade. Farnsworth also confirmed that the company is “acting quickly” to remove AI Overview on certain queries “where appropriate under our content policies and using these examples to develop improvements More broadly to our system, a number of improvements have already begun to roll out.”

Gary Marcus, an AI expert and professor emeritus of neuroscience at New York University, told Techlade that many AI companies are “selling the dream” that the technology will go from 80% to 100% accurate. %. Marcus said getting to the initial 80% is relatively simple because it involves approximating a large amount of human data, but the final 20% is extremely difficult. In fact, Marcus says this last 20% may be the most difficult.

“You really have to do a little bit of reasoning to decide: does this make sense? Is this source legitimate? You have to do the same things that a human fact checker might do, What could really require artificial general intelligence (AGI),” Marcus said. Marcus and Meta’s AI Director Yann LeCun both agree that the large language models that power current AI systems like Google’s Gemini and OpenAI’s GPT-4 won’t be what makes AGI possible.

This is a difficult situation for Google. Bing is already ahead of Google in AI with Satya Nadella’s famous “we make them dance” quote, OpenAI is reportedly developing its own search engine, a new AI search startup is already worth $1 billion, and Younger generations of users who only want the best experience are turning to TikTok. The company is clearly feeling competitive pressure, and pressure is what makes AI releases messy. Marcus points out that in 2022, Meta released an AI system called Galactica that had to be taken down shortly after launch for a number of reasons, such as the system advising people to eat glass. Sound familiar?

Google’s Summary function, also known as AI Overviews, is in its very early stages compared to what Google initially revealed. According to the plan, AI Overviews will handle complex questions, self-organize search results pages and even integrate video search in Google Lens. However, its current feature is causing Google trouble.

An expert in the field, Gary Marcus, points out that Google’s underlying AI models cannot themselves check the accuracy of their output. This is a major weakness and is seriously affecting Google’s reputation.