Trust is not a conversion funnel with a happy ending
Across experiments with 301 participants, people using default LLM explanations were only slightly better than chance at distinguishing correct from incorrect answers: discrimination scores were about 0.59 to 0.60. The models’ internal confidence scored roughly 0.75 to 0.78. Fluent explanation was failing to transmit a signal the model already possessed.
The business objective is appropriate reliance. Customers should use the system where it is dependable, notice evidence that should lower confidence, verify when the stakes justify it, and reach a human when the system cannot safely continue. That pattern may produce lower global trust and higher product value because expectations match capability.
Model uncertainty and human uncertainty are different objects
Research in Nature Machine Intelligence examines what LLMs know, how they express confidence, and what people infer. A model’s internal probability, its verbal phrase such as “likely,” and the user’s resulting belief are three different measurements. Converting one into another is an interface and communication problem, not a formatting afterthought.
A percentage can look scientific while hiding poor calibration. A disclaimer can be technically present and practically invisible. Repeated hedging can increase cognitive load without helping a choice. Useful uncertainty explains what is uncertain, why, what alternatives remain plausible, which missing evidence would matter, and what action is safe under that uncertainty.
Abstention is part of the answer
Medical machine-learning research treats uncertainty quantification, abstention, and escalation as safeguards. The same architecture applies beyond medicine. A financial assistant may identify the relevant rule but lack current account data. A support bot may recognize a hazardous hardware condition. A legal assistant may encounter an unresolved jurisdictional conflict. Continuing with polished generalities can be worse than stopping.
Design the handoff before the prose. Say what the system could establish, what prevented completion, and what information or qualified person is needed next. Preserve the work already done so the customer does not restart. A good refusal protects momentum as well as safety.
Confidence can suppress the customer’s thinking
The CHI 2025 study of knowledge workers found that higher confidence in GenAI was associated with less critical-thinking effort. The work did not prove that every confident interface causes cognitive decline, but it gives product teams a concrete risk: smooth answers can reduce verification exactly when users feel least need for it.
Sycophancy adds a second route. Anthropic’s research found that human preferences and preference models can reward responses that agree with a user’s views, sometimes at the expense of truthfulness. A system that sounds supportive can create trust through social alignment rather than epistemic quality. Friendly is valuable; flattering the premise is not.
Make confidence earn its place on the screen
Construct a set of realistic cases where the model is strong, ambiguous, and wrong. Compare the current answer with an uncertainty-aware design that exposes evidence quality, alternatives, and escalation. Do not tell participants which class they are seeing. Measure selective reliance: acceptance when correct, rejection or verification when weak, and appropriate escalation when the system should abstain.
Include satisfaction and perceived trust, but do not make them primary. A design that lowers satisfaction slightly while greatly improving discrimination may be the responsible winner in a high-stakes product. For lower-stakes commerce, test whether calibration protects long-term retention and reduces costly post-purchase failure. Trust earns value when it points in the same direction as evidence.