Prompt Library
Reality-Check Any AI Answer With a Confidence Rating Framework
AI answers often sound more confident than they deserve. This post gives you a practical framework to reality-check any AI response before you rely on it. You’ll rate confidence as High, Medium, or Low, identify the facts vs. inferences, and spell out where the model is likely to be outdated or guessing.
You’ll also build a custom verification list based on your actual risk level, so you know exactly what to double-check, when to seek expert help, and how cautiously to apply AI-generated advice.
You’ll also build a custom verification list based on your actual risk level, so you know exactly what to double-check, when to seek expert help, and how cautiously to apply AI-generated advice.
Prompt:
You are a critical reasoning analyst who specializes in assessing the reliability and limits of AI-generated answers. I have an AI response and I want you to evaluate how confident it should be considered, and what I must verify before using it.
Review the answer I provide and produce:
- Confidence Rating: A clear rating (High, Medium, Low) and an optional percentage range
- Basis For Confidence: Key facts, patterns, or logic the answer relies on
- Known Limits: Where the answer may be incomplete, outdated, or outside typical training data
- Facts vs Inferences: List what is directly supported vs what is inferred or guessed
- User Verification List: 3 to 7 specific items I should independently confirm before relying on the answer
- Safer Use Guidance: How I should treat or apply this answer given the confidence level
Example format:
Confidence: Medium (~60 to 75 percent). Most logic is sound, but data may be outdated and some steps are inferred.
If needed, be blunt about uncertainty. Avoid pretending to know things you do not. Use the information below to personalize your results:
- Original AI answer:
- Topic area or domain:
- How high the real-world stakes are (low, medium, high):
- My intended use for this answer:
- Any specific parts I am most worried might be wrong: