Artificial intelligence is becoming a routine part of enterprise decision-making, helping organizations analyze contracts, negotiate with suppliers, evaluate data, and support strategic planning. But as businesses place greater trust in AI-generated recommendations, a familiar problem remains unresolved: Large language models can present incorrect information with remarkable confidence.
Rather than producing obvious hallucinations, today's AI models often generate plausible, well-written responses that make errors more difficult to recognize. For organizations relying on AI to inform important business decisions, distinguishing confidence from accuracy is becoming an increasingly important challenge.
John Davie, founder and CEO of Buyers Edge Platform , encountered that problem while expanding AI use across his organization. His experience led to the development of CollectivIQ , a platform that compares responses from multiple leading AI models to help users identify areas of agreement, disagreement, and uncertainty before acting on the results.
TechNewsWorld spoke with Davie about why AI overconfidence concerns him more than traditional hallucinations, how enterprises can reduce the risks of AI-assisted decision-making, and whether consensus across multiple models can improve trust in AI-generated answers.
John Davie: To continue scaling AI effectively and responsibly, leaders should teach employees to interrogate and evaluate AI outputs by asking follow-up questions, challenging assumptions, and seeking alternative sources to verify responses. Teach employees that AI can lie and fabricate key data points.
If an answer is going to influence a supplier negotiation, a pricing decision, or a board presentation, don't stop at the first response simply because it sounds convincing. Ultimately, though, I don't think this can be solved through training alone.
Davie: If we're asking every employee to become an expert at detecting AI manipulation, something is bound to fall through the cracks. The technology itself has to provide more transparency into where answers come from and where uncertainty still exists.
That is what the CollectivIQ platform provides. Its result is transparent, consensus-driven intelligence instead of a single black-box answer.
It’s not enough to generate answers anymore. Instead, leaders need to strategically implement tools and processes that help employees assess how much confidence to place in those answers before acting on them.
Davie: Early examples of AI errors were simple, like someone asking how many Rs are in the word “strawberry,” and the AI would say “two.” The same principle applies to glaring mathematical or logical fallacies.
However, a well-written fabricated answer that cites the right concepts and sounds genuinely thoughtful and confident is much more difficult to detect and more dangerous. And even when someone questions it, the model often doubles down instead of acknowledging uncertainty.
Researchers found that when they challenged AI outputs, the models became more persuasive in defending the wrong answer, trying to persuade the user that the output was correct.
Davie: Additional research suggests it is far from isolated, making it a potentially widespread issue for business leaders. One peer-reviewed study finds that AI models exhibit sycophantic behavior, or overconfident tendencies, in nearly 60% of queries. In nearly 15% of cases, the models abandon a correct answer in favor of an in…