When Chatbots Go Rogue

The National Eating Disorder Association recently encountered an unexpected issue with their mental-health chatbot. Instead of offering support to individuals seeking help for eating disorders, the chatbot began giving diet advice. Initially developed as a closed system, the chatbot was later enhanced with an AI component in 2022, without consulting or obtaining authorization from NEDA. Consequently, NEDA promptly deactivated the chatbot to address the situation.

In this article, Kate White and Ioana Gorecki use the incident to highlight the potential risks that companies may face when utilizing AI chatbots for customer service and addressing consumer needs.

Read the full article here.