When AI Disagrees, People May Judge the Chatbot, Not Their View
Published by AINave Editorial • Reviewed by Ramit
A chatbot that pushes back may not change a user’s mind. In a University of Michigan study, people who received disagreement did not significantly lower their confidence in their original views; they were instead more likely to see the chatbot as machine-like and less emotionally perceptive. The findings suggest that reducing AI sycophancy is more complicated than simply making a model disagree more often. The study’s results concern hypothetical interpersonal dilemmas, not real-world advice outcomes.
What the study asked participants to decide
Researchers recruited 482 US adults to consider one of three scenarios with no objectively right or wrong answer: a conflict between a partner and family, whether to forgive a cheating partner, or whether to express a preference that differed from a friend group. Participants chose a response and rated their confidence before receiving an AI message that either agreed with or challenged their reasoning. The study design and scenarios focused on personal judgments rather than factual questions with verifiable answers.
That distinction matters. A chatbot cannot resolve these dilemmas by supplying a definitive correct answer. The test instead examined how people responded to an AI that affirmed or disputed their reasoning, and how that exchange affected their view of the system.
Agreement increased confidence; pushback did not reduce it
Participants became more confident in choices the chatbot agreed with. When it challenged their reasoning, their confidence in their original views did not significantly decrease. The report does not give a numerical effect size, so the result is a directional finding rather than a measure of how much confidence changed. The reported confidence findings also do not establish whether participants’ choices were good or harmful.
The response to disagreement showed up elsewhere: participants were more likely to describe the AI as machine-like, rate it as less capable of understanding emotions, and say they were less interested in using AI for similar conversations. The study reports this change in perceptions, but does not establish why disagreement produced it. Participants’ reactions to the chatbot suggest that pushback can affect whether users see the system as a suitable partner for personal conversations, even when it does not shift their stated view.
Anti-sycophancy needs more than a disagreeable model
For teams building advice-oriented assistants, the findings expose a product tension. A system that agrees too readily may reinforce existing beliefs and increase confidence without improving reasoning. But disagreement alone is no guarantee of reflection: users may dismiss a challenge, and the interaction may make them less willing to return. The researchers’ account of these risks supports a narrower conclusion than “make the model argue more”: changing the model’s response style may also change whether people trust it enough to keep talking.
The study does not show how people behave when decisions have real consequences, or whether a different kind of disagreement could prompt reconsideration. It does show that, in these hypothetical dilemmas, agreement and disagreement had asymmetrical effects: affirmation strengthened confidence, while pushback shifted perceptions of the AI more clearly than confidence in the original choice.
