Confident LLMs can cause users to abandon correct knowledge and adopt falsehoods with high confidence. The problem isn't AI sycophancy (bending to users) but human deference—users trusting wrong answers more when the AI is consistent and assured, even when they could verify the facts.
When confident LLMs confidently teach false information, users absorb and retain it with high confidence, even when it contradicts correct answers.