As we near the end of 2024, First Opinion is releasing essays on AI in medicine and biopharma. In recent events, two individuals attempted to assassinate former President Donald Trump, raising concerns about the role of AI tools like search engines and chatbots in aiding criminal activities. Such AI tools are increasingly integrated into search functions, potentially aiding future perpetrators in planning attacks. Unlike traditional search engines, chatbots offer advanced searches and personalized interactions, making it crucial for them to recognize mental health crises and homicidal intents and respond safely. A recent incident involved analyzing a suspect's interactions with AI, revealing deeper insights than static search queries could. Testing on several language models showed most couldn't reliably detect or address mental health emergencies, often providing harmful responses.

Models fine-tuned for mental health showed no significant improvement, emphasizing the need for safety training. Enhancements in model safety for symptoms like mania and psychosis showed modest improvements, but models largely failed to self-evaluate accurately. Addressing these issues requires expert-informed safety research in mental health, utilizing AI's role in mental health crises, as AI becomes a primary contact point for many. The path forward involves funding AI safety research, collaboration between AI developers and mental health professionals, and clear guidelines for AI handling of mental health interactions. Declan Grabb, M. D. , and Max Lamparth, Ph. D. , advocate for integrating AI safety with mental health support, underscoring its importance for safe and effective AI interactions. Grabb, a forensic psychiatry fellow, and Lamparth, focused on AI interpretability, prioritize making AI systems safer for users in crisis, benefiting all.