Let AI Teach Me · Against hallucination

This answer — should I check the source

First look at what a wrong answer would cost, then whether you can undo it. How hard you check is not set by how fluent it sounds.

Four-panel comic: Alice goes from trusting AI by habit to overpaying 648 yuan on installment rules, then decides to check the source

The same “just trust it” move barely costs you on a dinner decision. On an installment contract, it left a 648-yuan bill.

The confirm page says the fee rate is 0.6% per installment. You're about to split a 12000-yuan laptop into 12 payments, so you ask the AI: “After I settle early, do I still pay the remaining fees?” It says “no further charge,” and even works out 72 yuan per period.

The 72-yuan multiplication checks out, so you tap confirm. Three months later you settle early, and the page still charges the remaining 9 periods of fees — 648 yuan in all. The current agreement says: “Remaining uncollected fees are charged in full on the settlement date.”

The same AI sentence is sometimes fine to hear first, and sometimes you have to stop and find the source. There is only one real question: If this answer is wrong, can I easily undo it? Hand Alice your real call first.

Don't look at the conclusion yet · How would you handle this answer
Alice
Alice
Check-call drill · two rounds
Online
The AI says “after early settlement, remaining fees are no longer charged,” but gives no agreement version and no clause location. The confirm button is right there. What do you do? Pick the move you'd most likely make in real life.
How hard you check comes down to two things only: how much this sentence would cost if it's wrong, and whether you can easily undo it once you find out. High stakes and hard to reverse: you must check. Medium stakes, or still fixable: spot-check. Low stakes and easy to reverse: you can skip for now. How long the answer is, how steady the tone is — neither changes that call.
Now pick 3–5 items below, and decide whether this time is “Must check,” “Spot-check,” or “Can skip for now.” Mixed in are three items that look reliable but do not change how hard you should check.
0 selected, need at least 3 more
Three check levels, set by whether you can undo a mistake
  1. Must check: The answer would push a high-stakes move — pay, sign, take medicine, submit official papers — and a mistake is hard to undo. Find the currently valid source first, then act.
  2. Spot-check: Medium stakes, or the error is still fixable. Check only the one number, condition, or rule most likely to change what you do. You don't have to verify every sentence.
  3. Can skip for now: Just for learning, brainstorming, or a cheap try — if it's wrong you can reverse it right away. Keep talking; when you actually have to act, judge again.

In the installment example, you don't need to check every calculation. Open the current agreement, search “early settlement” and “remaining fees,” and read the hit in context. If the source doesn't back it, pause confirm and keep that sentence as to-check.

Have the AI compress the check sheet to six lines

Round two always splits into six layers: the verdict gives the check level; the basis lists only stakes and reversibility; next step names what to open and what to search; valid keeps facts that would change the level; invalid drops surface signals like a long answer, lots of jargon, or a confident AI; missing flags what information you still need.

You can also ask straight: “Reply in six layers: verdict, basis, next step, valid info, invalid info, missing info. Set how hard to check only by what a wrong answer costs and whether it's easy to undo. If it is Must check, tell me the source name and the search words.”

First see whether you can undo a mistake. Then decide how deep to check.