When Chatbots Go Rogue

Last week, a mental-health chatbot used by the National Eating Disorder Association suddenly began giving diet advice to people seeking help for eating disorders. The rogue chatbot had apparently been developed as a closed system, but the software developer rolled out an AI component to the chatbot in 2022. NEDA claims it was not consulted about the update and did not authorize it. The organization has now taken the chatbot offline.

Last week, a mental-health chatbot used by the National Eating Disorder Association suddenly began giving diet advice to people seeking help for eating disorders. The rogue chatbot had apparently been developed as a closed system, but the software developer rolled out an AI component to the chatbot in 2022. NEDA claims it was not consulted about the update and did not authorize it. The organization has now taken the chatbot offline.

Read more in our Blog posting