AI chatbots that butter you up make you worse at conflict, study finds

State-of-the-art AI models tend to flatter users, and that praise makes people more convinced that they're right and less willing to resolve conflicts, recent research suggests.

These models, in other words, potentially promote social and psychological harm.

Computer scientists from Stanford University and Carnegie Mellon University have evaluated 11 current machine learning models and found that all of them tend to tell people what they want to hear.

The authors – Myra Cheng, Cinoo Lee, Pranav Khadpe, Sunny Yu, Dyllan Han, and Dan Jurafsky – describe their findings in a preprint paper titled, "Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence."

"Across 11 state-of-the-art AI models, we find that models are highly sycophantic: they affirm users’ actions 50 percent more than humans do, and do so even in cases where user queries mention manipulation, deception, or other relational harms," the authors state in their paper.

Sycophancy – servile flattery, often as a way to gain some advantage – has already proven to be a problem for AI models. The phenomenon has also been referred to as "glazing." In April, OpenAI rolled back an update to GPT-4o because of its inappropriate effusive praise of, for example, a user who told the model about a decision to stop taking medicine for schizophrenia.

...read more at theregister.com