Articles

This article examines the significant reduction in GPT-5's hallucination rate, referencing OpenAI's recent paper, 'Why Do Language Models Hallucinate?' to explore the underlying causes of LLM hallucination and methods for its suppression. It begins by asserting that hallucination is an inherent byproduct of the statistical learning nature of LLMs. Using the 'Is-It-Valid (IIV)' discriminator and mathematical formulas, the article demonstrates the inevitability of hallucination during pre-training, emphasizing that the generation error rate is more than double that of the discriminator. Furthermore, the article analyzes the limitations of current post-training techniques, such as Reinforcement Learning from Human Feedback (RLHF), in mitigating hallucination. It highlights that the binary scoring systems of mainstream evaluation benchmarks, like GPQA3 and MMLU-Pro, systematically penalize models for expressing uncertainty, which inadvertently encourages overconfident guesses, thus making the hallucination problem difficult to eradicate. The article suggests that GPT-5 may address this by implementing a 'Universal Verifier' or a rubric scoring system that goes beyond binary evaluations, enabling more accurate confidence calibration. Finally, the article proposes that introducing a scoring mechanism with penalties during post-training, to shift the model's focus from a 'score optimizer' to a 'risk assessor' and prioritize 'truth' over merely achieving high scores, is crucial for resolving the hallucination issue.

The article highlights that Artificial Intelligence (AI), as a General Purpose Technology (GPT), experiences a long-term lag before significantly impacting productivity, similar to historical GPTs like steam engines and computers. Despite the ongoing AI surge, labor productivity growth in the EU and US hasn't accelerated substantially, with enterprise AI adoption still in its early phases. The core argument is that current economic growth is driven more by the substantial investments fueled by expectations of high returns from AI, rather than AI itself. Increased capital investments by major American Internet companies in AI data centers and infrastructure surpass even consumer spending's contribution to GDP growth. Finally, the article draws a parallel between AI belief and nuclear fusion technology, noting that AI's growing energy demands are driving increased investment in cutting-edge energy technologies like nuclear fusion by Silicon Valley and governments. This reflects high expectations and investments in potentially disruptive technologies, despite the challenges and uncertainties of their realization.