Do you need help & advice with AI & Automation or New Technology?
Ask AI a question it does not know the answer to and it will not tell you that. It will give you an answer, in the same confident tone it uses when it is right.
That is not a bug that is about to be fixed. It is how the thing works, and once you understand why, it becomes fairly easy to work out which jobs to trust it with.
Why does AI make things up?
Because it is not looking anything up. It is predicting what the next words should be.
That is worth sitting with for a second, because it explains almost everything else. AI is not searching a store of facts and reporting back. It is producing text that fits the pattern of a good answer. Most of the time a good-sounding answer and a correct answer are the same thing. Sometimes they are not, and nothing in the output tells you which one you are looking at.
The industry word for this is hallucination, which makes it sound rarer and stranger than it is.
And here is the part that catches people. If it makes a mistake, it will lean into that mistake. Challenge it and it will often defend the wrong answer, or invent a supporting detail, rather than back down. It does not know it was wrong the first time, so it has nothing to correct.
Why does it sound so sure of itself?
Because we taught it to. Confident answers get better feedback than honest ones.
People do not much like being told “probably” or “I am not certain”. When you ask a question you generally want an answer, not a discussion of the possibilities. Every one of these tools has been shaped by people rating its responses, and hedged answers get rated worse.
So the tone is not evidence of anything. A wrong answer arrives in exactly the same voice as a right one, and that voice was tuned to be reassuring.
So what can I safely use it for?
Work where you already know the answer, or where being wrong is cheap and obvious.
That is a bigger category than it sounds, and it covers most of the useful stuff.
| Safe, because you can see it is wrong | Risky, because you cannot |
|---|---|
| Tidying up something you wrote | Asking it for a figure you do not already have |
| Summarising a document you have got | Asking what a contract clause means, and acting on it |
| A first draft you were going to write anyway | Anything going to a client unread |
| Turning notes into something readable | Calculations, valuations, measurements |
| Explaining something back to you in plainer words | Regulations, standards, legal requirements |
Notice the pattern on the left. In every case you are the one who knows what the answer should look like, so a bad one stands out immediately. On the right you are asking it for something you cannot verify, which is exactly where a confident wrong answer does damage.
Should I use it instead of Google?
No. That is the single worst way to use it.
A search engine finds you a page that either exists or does not. AI will produce an answer whether or not it has the information, and it will not distinguish between the two. Using it to look up a fact you cannot check is asking for trouble.
Some tools now search the web and cite what they found, which is better. Follow the citations. If there are none, treat the answer as a suggestion.
What is the actual rule?
Anything with a number, a contract term, or a person’s name in it gets read by a human before it goes anywhere.
That one line covers most of the risk in most businesses. It is short enough to remember and specific enough to follow, which is more than most AI policies manage.
The failure it prevents is not really a technology failure. It is somebody producing work with AI, not checking it, and sending it out. As Dave puts it: “There you go, I chucked it out, done.”
[VIDEO_PLACEHOLDER]
So, what now?
Say the rule out loud to whoever is using AI in your business. Number, contract term, or name: a person reads it first.
Then decide who owns that. If nobody does, it is not a rule, it is a hope. That is what an AI Manager is for, and the wider version of this is in getting started with AI in your business.
Frequently asked questions
What is an AI hallucination?
When AI produces something that sounds completely plausible and is entirely made up.
It is not lying and it is not malfunctioning. It does not know things, it predicts what text should come next, and sometimes that prediction is wrong while sounding perfectly confident.
Will hallucination be fixed?
Not entirely, because it is a consequence of how the technology works rather than a fault in it.
Newer models are better and tools that cite their sources help a lot. But any system that generates a plausible answer can generate a plausible wrong answer, so checking stays part of the job.
Can I just ask AI whether it is sure?
It will answer, but the answer is not worth much.
Asking a model to rate its own confidence produces another prediction, not a measurement. Worse, if it got something wrong first time it will often defend that answer rather than reconsider it.
Which AI tool hallucinates least?
The wrong question, because they all do it and the ranking changes every few months.
What matters more is whether the tool shows you where its answer came from, and whether you are asking it something you could check. A tool with citations used carefully beats a slightly better model used carelessly.
Is it safe to use AI on contracts?
For summarising a contract you already have, yes, with a read-through. For deciding what a clause means, no.
It will give you a confident interpretation, and you will have no way of telling a good one from a bad one without checking the clause yourself. Contract terms are on the list of things a person always reads.
Last reviewed: 17 August 2026.
Related
- AI slop, and why generic content costs you
Every sentence makes sense and the whole thing means nothing. - Vibe coding: what happens when nobody checks the code
What to check before AI-written software goes near a client.
