Why Gemini 3 Pro "Failed" Humanity's Last Exam: The 99% Myth Exposed

For the last three years, we have been spoiled.

Every time a new AI model dropped, the headlines were the same.

"GPT-4 scores 90% on Bar Exam."
"Gemini scores 99% on MMLU."

We got used to the idea that AI was essentially perfect. We believed the "99% Myth"—the idea that AI had effectively solved human knowledge.

Then came Humanity's Last Exam (HLE), and the illusion shattered.

If you are looking for the raw numbers, check our Live Leaderboard 2026: Gemini 3 Pro vs. DeepSeek vs. GPT-5 to see exactly where each model ranks.

But if you want to know why the world's most expensive AI couldn't ace this test, read on.

The Death of MMLU: Why We Needed a Harder Test

To understand the failure, you have to understand the old yardstick.

For years, the industry standard was MMLU (Massive Multitask Language Understanding). It tested general knowledge across 57 subjects, from math to history.

But by late 2025, MMLU was broken.

Models weren't just "smart"; they were memorizing the test questions. Because MMLU data is all over the internet, AI models had already "seen" the answers during training.

This is a phenomenon known as "Benchmark Saturation."

It is the main reason Why we moved away from MMLU as the primary metric for intelligence. We needed a test that an AI couldn't cheat on.

What is "Humanity's Last Exam"?

Enter the Center for AI Safety (CAIS).

They designed HLE to be the "AGI Litmus Test."

Unlike MMLU, which asks trivia questions like "What represents the atomic mass of Oxygen?", HLE asks abstract, multi-step reasoning questions that require:

• Visual interpretation.
• Novel problem solving.
• Cross-domain synthesis.

These are questions that cannot be answered by a search engine.

If you memorize the entire internet, you will still fail HLE. You have to think.

Did Gemini 3 Pro Actually "Fail"?

So, did Google's flagship model fail?

It depends on your definition of failure.

• The Hype: Google marketing suggests Gemini 3 Pro is superhuman.
• The Reality: On HLE, it scored 74.2%.

In the academic world, a 74% is a C grade.

For a tool that claims to be smarter than a PhD, a "C" is a shock to the system.

It exposes a critical weakness in modern LLMs: They are great at retrieval, but mediocre at reasoning.

When Gemini 3 Pro faced questions where it couldn't rely on its massive training data, it started to hallucinate. It made logical leaps that a human expert wouldn't make.

The "AGI Threshold"

Why does this specific score matter?

The creators of Humanity's Last Exam have set a theoretical "AGI Threshold."

• 90% - 100%: Superhuman / Expert Level (True AGI).
• 70% - 80%: Competent Graduate Student.
• < 50%: Undergraduate Level.

Gemini 3 Pro is currently stuck in the "Graduate Student" phase.

It is brilliant, yes. But it is not yet the autonomous scientist that can cure cancer or invent cold fusion.

To do those things, it needs to score 95%+ on HLE.

Right now, no model on Earth is even close.

Analyze Traffic & Dominate Your Niche. Try SimilarWeb for Free.

We may earn a commission if you buy through this link. (This does not increase the price for you)

Frequently Asked Questions (FAQ)

Sources & References

• [Internal] Live Leaderboard 2026: Gemini 3 Pro vs. DeepSeek vs. GPT-5 - The real-time tracker of all model scores.
• [External] Center for AI Safety (CAIS) - Official release paper on the HLE dataset.
• [External] arXiv.org - Research papers on "LLM Benchmark Saturation" and Data Contamination.

Back to Top