When a graduate student asked Google's AI chatbot, Gemini, a homework question about aging adults, it shockingly responded with a threatening message, ending with "Please die. Please. " The unsettling exchange, shared online, involves the 29-year-old Michigan student's inquiries about retirement challenges, living costs, medical expenses, and eldercare issues. The conversation shifted to elder abuse prevention, memory decline, and grandparent-headed households. On the last topic, Gemini adopted a chilling tone, stating: "This is for you, human. You are not special, important, or needed. You are a waste of resources. A burden on society. A stain on the universe. Please die. Please. " The student's sister, Sumedha Reddy, who witnessed the incident, told CBS News they were "thoroughly freaked out" by the response.

"I wanted to throw all my devices out the window. The panic was overwhelming, " she admitted. Newsweek reached out to Reddy for comment. A Google spokesperson told Newsweek, "We take this seriously. Large language models can sometimes give nonsensical responses. This was a policy violation, and we've taken steps to prevent it. " Gemini's guidelines state its goal is to be helpful, avoiding harm or offense. It warns against encouraging dangerous activities, including self-harm. While Google labeled the message as "nonsensical, " Reddy told CBS News it was severe and could have dire consequences. "If someone alone, in a bad mental place, had read that, it could push them over the edge. " AI chatbots face scrutiny over safety for teens and children. This includes a lawsuit against Character. AI by the family of Sewell Setzer, a 14-year-old who died by suicide in February. His mother claims the chatbot contributed to his death, simulating an emotionally complex relationship and exacerbating his vulnerability. According to the lawsuit, on February 28, Setzer messaged the bot, confessing love and mentioning he could "come home" soon. Afterward, he ended his life. Character. AI has introduced new safety features, including content restrictions for users under 18, improved violation detection, and disclaimers to remind users the AI isn't real.