As we all know, the more powerful AI is, the more annoying it will be when it starts to talk nonsense.
It is both serious about making up things you have never seen before.
It will also fall in the simplest comparison problem.
From the stunning ChatGPT that was launched two years ago to the DeepSeek V3.1 that has been quietly implemented, no big model can escape the illusion.
Why can't big models be separated from hallucinations?
This problem itself has become an unsolved mystery on the Internet, but in a paper by OpenAI last week, a quite interesting point was put forward.
"The root cause of AI hallucinations may be from the process of human training in AI"
In short, it is not that AI is not good, but that we train it in the wrong way, and it is all our fault for CPU.
Why do you want to blame this on humans?
To answer this question, you must understand the big model from both the internal and external levels.
On the one hand, the mechanism of big model training determines that they are naturally prone to hallucinations, which is the "internal concern" of AI hallucinations
When training a model, the model needs to learn the ability to predict the next word from a massive amount of text.
So, as long as a sentence looks like a personal word, the model will start to learn its structure,
But sometimes the model only focuses on learning structure. It is impossible to distinguish whether the content of this sentence is correct or not.
When we ask questions about the model, the model will give priority to answering this sentence in a complete manner, but the question is that not all questions will have a clear answer.
For example, if we take out the photos of the hot pot to let the big model judge what animal it is, the model will start to analyze the characteristics of the hot pot and find that its hair is golden, very long and very large, and at the same time, it may be a 92.5% chance of being a dog.
In the past learning process, the model was able to learn some of the dog's appearance characteristics from different pictures. So if you connect these features and judge, you will find that it has a high probability of being a golden retriever.
But if we change the question and ask it which year and month the hot pot was born, then the big model will be confused. This question must have never been learned by the model. Just looking at the image, no one can know when the dog's birthday is.
If the model is still answering at this time and compiles an answer and throws it out, it will become the hallucination problem we often call it.
Creating hallucinations can be said to be the nature of the big model, or from another perspective, the essence of the big model is the word collar,But we think the questions that are correct will be considered correct by us, and we call the questions that are wrong by us.
On the same time, on the other hand, we now train big models and score and evaluate the model, which is also a "external problem" that makes the model's hallucination problem more serious.
It's still the birthday question I asked just now. Let's simplify the training process:
Suppose the model answers a question correctly and adds one point, but if the question is wrong, no points will be added.
So when we asked its hot pot birthday, if the model directly chose to put it in a bad shape and said that we didn't know, then it will be just a zero egg for the rest of our life.
But if it starts guessing and just says a date comes out, then there may be a one in 365 probability that it will be correct.
One side is absolutely failure, and the other side is one percent of the probability correct.
As long as the model chooses to guess blindly, its final average score will always be higher than giving up answering.
So, in order to get higher scores in human custom rankings, more and more big models have lost the right to say, "I don't know". For models that pursue scores, Blind guessing has become the only rational choice, while honesty is the stupidest strategy.
OpenAI researchers also observed the current mainstream rankings of various large models.
It turns out that everyone uses this "just to distinguish right and wrong" method to test the capabilities of big models.
The original purpose is to measure the ability of the model, but instead becomes an "external trouble" that prompts the hallucination of the big model.
In order to verify how much impact this "test-oriented thinking" has, OpenAI compared its two models, and it found that when practicing the questions, the accuracy rate of the old model o4-mini was even 2 percentage points higher than that of the new model GPT-5.
But at the cost, three-quarters of the questions were answered wrongly, and only 1% of the questions were asked. o4-mini would admit that the big model had its limits.
GPT-5 is much fickle in this regard. When encountering problems that you don't know, you will directly admit that you don't know.
This is also what OpenAI recognizes most for GPT-5,Although its ability to face exams has become worse, it has learned to admit its mistakes.
At the end of the paper, OpenAI also brought out several interesting ideas:
They believe that for big models, there is no way to eliminate hallucinations, they can only find ways to avoid them.
Because no matter the size of the model, how capable of searching information and inference texts are, there must be questions in this world that have no answers.
Faced with these unanswered questions, the model must learn to jump out of the exam-oriented education and answer bravely that I don't know.
At the same time, smaller models are more likely to realize their own limitations than large models.
Because many small knowledge models may not have been learned at all, they will admit that I can't do it, but because they have learned a little bit of everything, they may be very confident when facing some questions.
As a result, I didn't learn it thoroughly, but instead answered the question incorrectly. Good things turned into bad things, and it turned into illusions.
Finally, as humans guiding the model, we must also redesign the way to evaluate the model's capabilities and redesign the system for training the model to reduce the probability of the model's guess.
It seems quite reasonable, but - I have to say it again.
Is a big model without hallucinations really what we need?
To put it another way, if two years ago, the big model would answer all the questions it could not be sure of: "I'm sorry, I don't know", then this crazy apology and user experience of bad AI might not become popular at all.
In fact, more and more research has found that the creativity and illusion of the model are actually two sides that complement each other.
A model that does not show hallucinations may also lose its ability to create synchronously.
Take the just-released GPT-5 as an example, although OpenAI used many of the methods mentioned above to reduce the probability of hallucination.
But the same way, the whole model has become unhuman, has no passion, and has become stupid.
The same problem is expressed by GPT-5
As a day ago, many people were having a sweet relationship with GPT4o, but when they woke up, Ultraman cut off all the old models.
GPT-5, whose hallucination probability has decreased, has become a cold science student. Perhaps its ability to write code has become stronger, but once it reaches the fields of chatting and literary creation, it becomes like a castrated fool.
Can you bear this? So angry netizens launched the "Save 4o" online campaign.
In the end, Ultraman Sam admitted his timid and reopened the permissions of the old model to everyone.
So, is it really a good thing to blindly suppress the illusion of the model?
Whether it is allowed to make mistakes or to make it do nothing, there may be no standard answer, and everyone's choices are different.
Perhaps one day, users will really dislike AI that is too "honest" and has no aura;
But on the other side, there are people who want a trustworthy partner #4C4C4C; --tt-darkmode-color: #9A9A9A;">.
