Child safety groups question if Google is appropriate for classroom use.Credit: Zain bin Awais/Mashable/
Table of Contents
Millennials, the first computer generation, recall many a classroom lecture about the unreliability of Wikipedia. We were told to be wary of unverified forum posts. We were advised to seek out primary
Now, as Gen Alpha enters AI-powered classrooms, can Google Search no longer be trusted?
A new report by child safety watchdog Common Sense Media concluded that Google’s modernized, AI-powered Search poses an “unacceptable risk” to young Googlers. In fact, the report authors say, the generative AI search engine is so risky that students should stop using it completely until Google educators can turn the AI off.
You May Also Like
SEE ALSO:Teens talking to Meta AI about suicide will trigger alert
Google’s AI Search as classroom threat
Child safety advocates have been urging schools to ditch classroom technology for years. As generative AI embeds itself in educational products, those cries have gotten louder. But while it’s easy to see the potential harm in introducing something like ChatGPT to young students, it may not be as obvious that a simple Google search exposes young learners to AI.
Google launched AI Mode for Search last year. Unlike AI Overviews’ rollout, AI Mode was available to everyone, immediately, including education accounts. Both Common Sense Media and other experts Mashable spoke to said this was alarming, primarily because Google’s AI Mode is powered by Gemini models — models that are age-gated in other Google experiences, including on Google’s educational technology suite.
Google Search is a tentpole product for the tech giant and an essential tool for both educators and students, Robbie Torney, senior director of AI programs at Common Sense Media, explained. “Google is an outsized player in both the search space and the education space.”
The company helmed a 1:1 device movement that saw the introduction of personal devices, like Chromebooks, in schools across the country. Google Workspace for Education now powers administrative suites and classroom resources. The company later became a major evangelist for AI implementation, introducing, and then ultimately rescaling, Gemini for Education. Google now boasts millions of Workspace for Education, Chromebook, and other Google educational product users ranging from Kindergartners to post-grads.
Googling isn’t so simple anymore
For one, researchers have found Google’s AI Overviews are prone to repeating misinformation, an early product sticking point that has not yet been resolved by the tech giant. Much like existing problems with chatbots,hority that can easily confuse adults, let alone children
In addition, Common Sense Media found that Google’s AI tools readily completed homework assignments for minor users. Experts suggest that this combination of misleading design, varying responses, and the agreeable nature of chatbots (including AI Mode) may lead students to unknowingly cheat. At the bare minimum, they say, searching on Google may be making learning harder.
“They are infusing AI in developmentally inappropriate ways through Search,” explained Amanda Bickerstaff, deputy director of AI for Education. The organization works with educational institutions to build out AI literacy programs and reduce the potentially negative impact of generative AI tools on children’s learning.
Mashable Light Speed
By clicking Sign Me Up, you confirm you are 16+ and agree to our Terms of Use and Privacy Policy.
There is no reason why a 10-year-old, 13-year-old, or even 15-year-old should see an AI Overview or have AI Mode.
Bickerstaff called AI Mode a backdoor for students to access unsafe generative AI experiences. It’s also complicating educators’ ability to enshrine literacy skills in the next generation, Bickerstaff explained, replicating some of the worst parts of chatbot design right in a previously reliable information environment.
“There is no reason why a 10-year-old, 13-year-old, or even 15-year-old should see an AI Overview or have AI Mode,” said Bickerstaff. “When teaching AI literacy, the first step is to be intentional. These modes are the opposite. You don’t go to a search engine to get a chatbot answer, you go to search for real things.”
According to Common Sense Media’s report, AI Overviews and Google’s new Search chatbot AI Mode could also pose mental health and safety risks to children.
Google does not offer the option to disable only AI features while keeping Search accessible. That means children can still be exposed to generative AI while at school even when guardrails are in place, simply by doing their schoolwork. Parents can turn off Search entirely or block Google.com for the child accounts they manage, Google explains. School administrators could theoretically do the same for Workspace for Education accounts, bricking the Google Search domain for all users. Bickerstaff sees this as putting educators between a rock and a hard place. “What are you going to say? That kids can’t search the web?”
In a statement to Mashable following the report’s release, Google said Common Sense Media’s stress testing was narrow and ambiguous, and not an “effective way to measure product safety and helpfulness.” Common Sense Media did not test Workspace for Education accounts, only consumer accounts. However, Google expressed that the AI-powered Search experience is the same for all users, said Torney.
Could AI Search be a legal nightmare for schools?
“Google’s AI Overview and AI Mode are undermining kids’ education, supplying them with dangerous disinformation, and putting their very lives at risk,” said David Monahan, campaign director for child safety nonprofit Fairplay for Kids. “If Google is not even willing to disable AI features for students, schools should stop using Google products altogether.”
Common Sense Media’s report could be a startling wake-up call for school districts, said Anya Meksin, deputy director of Schools Beyond Screens, a national coalition pushing for school policies that limit tech use in classrooms. Schools Beyond Screens recently spearheaded new tech use policies at Los Angeles Unified School District (LAUSD) schools, including a screen time ban for kindergarten and first grade classes.
Both Fairplay and Schools Beyond Screens are leading calls for years-long moratoriums on Gen AI use in schools. Meksin explained that advocates have already sounded the alarm about AI Search and other integrated Google products, not just chatbots.
“We are shocked by the way Big Tech has colonized public education with their products,” Meksin said. “We have to treat this as an emergency. Education is the foundation of our democracy, and they are knowingly disabling our children from being able to fight back against these companies in the future.”
School districts, including LAUSD, have struggled to address the problems posed by the reliance on classroom tech, often in violation of their own burgeoning AI policies, she explained.
“School districts are instantly in violation of any policies put in place because of the speed of the technology and the way AI is embedded,” said Meksin. For example, LAUSD’s new policy bans students under 13 years of age from using generative AI in school. But through products like Chromebooks and Google Search on school computers, students can still interact with AI Mode.
We have to treat this as an emergency.
Over the last 20 years, K-12 schools have spent billions to outfit their classrooms with educational technology. Meksin said she was told by several districts that it would be impossible to fully disable such expansive services, including Google products like Chromebooks.
“We find this position to be completely laughable,” Meksin said. “School districts are liable for exposing children to products known to cause severe and lasting harm.” She believes educational institutions could enter the same legal hot water as social media and AI companies.
Google and its parent company Alphabet recently faced its first wrongful death lawsuit after a user died by suicide following conversations with Gemini. YouTube, also owned by Alphabet, was found guilty of negligent platform design and subsequent mental health harm to minor users. Torney likened the rollout of Google’s AI features to the introduction of YouTube to classrooms, which has also come under fire from child safety advocates.
Instead of opening themselves up to greater liability, Meksin believes school districts should push for change from Big Tech — starting with stronger controls for embedded generative AI features now available on school devices. “As the largest purchasers of these products, districts have power. If Google refuses to comply, districts must cease using these products and void these contracts.”
TopicsArtificial IntelligenceGoogleSocial GoodFamily & Parenting
Chase joined Mashable’s Social Good team in 2020, covering online stories about digital activism, climate justice, accessibility, and media representation. Her work also captures how these conversations manifest in politics, popular culture, and fandom. Sometimes she’s very funny.
