A client called me last month, completely rattled.
Her sales director had just sent a competitive analysis to the board. Beautiful presentation – clean tables, compelling statistics about three main competitors’ market share, revenue trajectories, growth rates. The kind of polished deck that makes you look like you’ve done serious research.
Mid-presentation, the CFO paused. “Where did we get their Q4 revenue numbers? Two of these companies are private.”
Silence.
The sales director checked his sources. Every single competitor data point had been generated by ChatGPT. Not estimated. Not approximated. Just invented. The AI had created “reasonable projections based on industry averages” and presented them as facts.
The board meeting ground to a halt. The sales director’s credibility took a hit. And my client spent the next week doing damage control.
→ Here’s what haunts me: The sales director isn’t careless. He’s actually one of their top performers. He just made the same mistake I’m seeing everywhere right now. He trusted AI output because it looked professional.
🤓 Welcome to the Confidence Trap.
Your Team Is Already Using AI (And Most Aren't Checking Their Work)
According to recent workplace research, 57% of employees admit to not checking AI-generated output for accuracy before using it. Even more concerning? About 56-57% of employees hide their AI usage or present AI output as their own work.
Translation: Right now, someone in your organization is probably asking ChatGPT to draft a proposal, summarize a contract or create market analysis. And there’s a better-than-even chance nobody’s verifying it before it ships.
The real problem isn't that AI makes mistakes. The problem is how it makes them.
A fascinating MIT study from early 2025 discovered something unnerving: AI models were 34% more likely to use phrases like “definitely,” “certainly,” and “without doubt” when generating incorrect information compared to when providing accurate answers.
Think about that. The wronger AI gets, the more confident it sounds.
- Perfect grammar.
- Authoritative tone.
- Zero hesitation.
The kind of polished writing that makes you think, “Well, this is definitely better than what I would’ve cobbled together at 4pm on a Friday.” (Spoiler: Sometimes it is. Sometimes it’s beautifully written fiction.)
We mistake fluency for accuracy. If something reads well, we assume it is well.
"The difference between AI fluency and AI accuracy is the difference between sounding confident and being correct. Your reputation depends on knowing which one you're getting."
When "It Looked Good" Becomes "How Did This Happen?"
Let me give you three real examples from the last six months.
1. The Marketing Mishap
A marketing manager used ChatGPT to write a thought leadership post about industry trends. Posted to LinkedIn. 800 views. 40 shares. Then a follower commented: “Interesting stat about 65% adoption rates. What’s the source?”
The manager checked. The statistic didn’t exist. ChatGPT had generated a plausible-sounding number to support the argument. Made it up. Completely.
2. The Sales Mix-Up
A sales rep asked ChatGPT to write a follow-up email after a discovery call. But he’d pasted notes from three different prospects to “save time.” ChatGPT blended them together. The email referenced a Q4 deadline the prospect never mentioned.
The prospect replied: “I think you have me confused with someone else.” (Not exactly the memorable follow-up we’re going for.)
3. The Board Meeting Disaster
That’s my opening story. Beautiful competitive analysis. Zero real data. All the professionalism of a Wikipedia article written at 2am.
🤓 Here’s the pattern: In every case, nobody checked. Not because they were lazy. Because the output looked so polished they assumed it was accurate.
The data backs this up. In 2024, 47% of enterprise AI users admitted to making at least one major business decision based on hallucinated content. And other comprehensive studies reveal that AI hallucinations costed the global economy $67.4 billion alone in 2024.
Sixty-seven billion dollars in bad decisions, damaged credibility, and cleanup costs. That's a lot of awkward follow-up meetings.
The 20-60-20 Framework: Your Supervision Strategy
If you’ve heard me speak, you know I talk about the 20-60-20 equation. It’s how I remind myself (and everyone I work with) that AI is a partnership, not a handoff.
- Your First 20%: You set the standards, define what “good” looks like, and write the prompt
- AI’s 60%: AI does the heavy lifting, including self-checking
- Your Final 20%: You verify facts, validate judgment, and approve
Modern AI systems can actually evaluate themselves. They have chain-of-thought reasoning, reflection loops, consistency checks built in. 76% of enterprises now include human-in-the-loop processes to catch hallucinations before deployment.
But AI evaluates probability, not accountability. It can tell you what’s likely to be correct. It cannot tell you what’s right for your business, your brand or your customers.
🤓 That’s still your job. (You know, the one they actually pay you for.)
What Actually Works
Here’s the prompt structure I use:
“You are an expert editor. Review your response against these criteria: Accuracy, Clarity, Tone, Completeness. For each criterion, identify one strong example and one area that needs verification. Then list every factual claim in your response: statistics, dates, names, sources, figures. Mark which ones you’re certain about versus which ones I need to verify.”
This forces AI to explicitly flag what you need to double-check.
Then your final review checklist:
- Read the AI’s self-evaluation first. Does its reasoning make sense?
- Verify every single fact. Don’t trust any statistic or claim without checking.
- Check for brand voice and context. Does this sound like us?
- Look for what AI can’t see: Ethical implications, timing issues, human nuance.
🤓 Think of it like this: Your intern drafts a report. They proofread it themselves. They flag what they’re uncertain about. Then you verify the facts and review the judgment before it leaves your desk.
AI does the work. AI checks the work. You verify the facts and own the result.
"AI doesn't know when it's wrong. But if you ask it to check itself against clear criteria, it'll at least tell you where to look twice."
Where Supervision Matters Most
- AI can reliably check: Logic, structure, clarity, tone consistency, completeness.
- AI absolutely cannot check: Factual accuracy, brand context, ethical implications, political sensitivities, human nuance.
- For high-stakes moments? That’s us.
39% of AI-powered customer service bots were pulled back or reworked due to hallucination-related errors. Companies learned the hard way that you can’t outsource judgment. (Or customer relationships. Turns out people notice when your chatbot confidently invents return policies.)
But AI can help in the background by flagging potential issues, listing claims to verify and checking logic against your criteria.
🤓 Think of AI as your first-draft editor. You’re still the one who signs off before anything ships.
Try It This Week
Your Homework
Before you copy-paste your next AI output, pause for thirty seconds:
- Ask AI to list every factual claim in its response. Every statistic. Every date. Every source. Every name.>
- See what it flags versus what it confidently presents as fact.
- Then verify those facts. All of them. Click the links. Google the statistics. Check the dates. That's what proper professionals do. (And honestly, it's faster than explaining to your CFO why your competitive analysis was creative fiction.)
We use this 20-60-20 language constantly with my team. I might say, “This looks really good, but how are the statistics? Did we do our 20?” Or, “This content is interesting but doesn’t sound like us. Feels like we did 5 instead of 20 at the end. Can you check it?”
It becomes shorthand for quality control. Everyone knows what “Did you do your 20?” means. And nobody wants to be the person who says, “Um, no, I just hit copy-paste.”
Bonus move: Next time someone on your team sends you AI-generated content, ask one question: “Did you verify the facts?”
🤓 Watch what happens. Most people haven’t. But now they know you’re checking. And suddenly, everyone starts doing their 20.
What This Really Means
The companies winning with AI right now aren’t the ones with the biggest budgets or the fanciest tools. They’re the ones who figured out how to supervise it without slowing down.
Less than half of employees (45%) think their company’s AI rollout has been successful, while 75% of executives believe it has. That gap? That’s the supervision problem right there.
Your team is using AI. 91% of employees say their organizations use at least one AI technology, with 54% specifically using ChatGPT or generative AI.
The question isn’t whether they’re using it. The question is: Who’s making sure it’s accurate before it ships?
AI isn't your boss. It's your intern. Even the best intern needs someone checking their work before it represents your company. (And you wouldn't let your intern present to the board without reviewing their slides first. Right? Right?)
P.S. Think your leaders or teams could use practical AI supervision strategies that actually work? Drop me a note and let’s talk about a keynote or workshop that makes AI feel manageable, not overwhelming. I promise not to suggest you ban it. That ship has sailed. But I will show you how to supervise it without slowing down.
About Julie: A Hall of Fame AI keynote speaker, tech founder, and innovation strategist, Julie works with associations, real estate professionals, and corporate sales teams to help them lead smarter, sell more, serve better, and save time with AI. She delivers highly actionable and engaging keynotes on becoming AI-empowered, leading in an AI-driven world, and transforming work and customer relationships in the age of AI.
Frequently Asked Questions About AI Accuracy and Supervision
Even the best AI models hallucinate at least 0.7% of the time, with some models exceeding 25%. Research from 2025 shows that advanced reasoning models can hallucinate even more frequently - between 16-48% depending on the task. The key isn't avoiding AI, it's knowing how to verify its output.
Yes and no. AI can evaluate its own logic, structure, clarity, and completeness. But AI cannot reliably fact-check itself. When you ask AI to verify factual claims, it can flag which statements need human verification, but you still need to click those links, Google those statistics, and confirm those dates yourself.
The 20-60-20 framework breaks AI collaboration into three parts: Your first 20% (setting standards and writing prompts) AI's 60% (doing the work and self-checking), and your final 20% (verifying facts, validating judgment, and approving output). This ensures accountability stays with humans while AI handles the heavy lifting.
According to workplace research, 57% of employees admit they don't check AI output for accuracy, and 56% hide their AI usage or present AI work as their own. Start asking one simple question when you receive content: "Did you verify the facts?" That question alone changes behavior.
Verify every factual claim: statistics, dates, names, sources, figures, URLs, and any specific numbers. Click every link to confirm it goes where AI says it does. Google any statistics to find the original source. Check that names and titles are spelled correctly and current. If AI references a study or article, make sure it actually exists.
Not always. While AI has improved dramatically overall, some advanced reasoning models actually hallucinate more frequently than their predecessors. OpenAI's o3 model, for example, hallucinates in 33% of certain tasks compared to 16% for the earlier o1 model. More sophisticated doesn't automatically mean more accurate.
This comprehensive studies reveal that AI hallucinations costed the global economy $67.4 billion alone in 2024. This includes bad business decisions, damaged credibility, legal issues, and the cost of correcting mistakes. At the organizational level, 47% of enterprise AI users made at least one major business decision based on hallucinated content in 2024.
No. 91% of organizations already use at least one AI technology, and your team is likely using it whether you've approved it or not. The solution isn't banning AI - it's supervising it properly. Establish clear verification standards, teach your team the 20-60-20 framework, and create a culture where "Did you do your 20?" is normal quality control.
Try this: "You are an expert editor. Review your response against these criteria: Accuracy, Clarity, Tone, Completeness. For each criterion, identify one strong example and one area that needs verification. Then list every factual claim in your response: statistics, dates, names, sources, figures. Mark which ones you're certain about versus which ones I need to verify."
For most business content, 2-5 minutes of fact-checking can catch the major issues. Click the links (30 seconds each), Google 2-3 key statistics (1-2 minutes total), and scan for brand voice (1 minute). It's faster than explaining to your board why your competitive analysis was fiction. Think of it as quality control, not extra work because the alternative is damage control.