Google Admits Search Console Can't Make Sense of AI Search. So What Should You Watch Instead?

The other day I scrolled past a conversation on Reddit that made me sit up straight.

An SEO practitioner asked Google: in the age of AI search, how are you actually supposed to track rankings?

John Mueller, Google's Search Advocate, replied personally. His point: trying to track rankings for AI features the traditional way is very hard to make "useful." To this day, Search Console still records the entire AI Overview as a single block.

What does that mean?

Put plainly: Google itself has admitted that its own ranking reports can no longer make sense of its own AI search.

Reading that, I froze for a second. Because for the past two years, so many people around me have still been using Search Console data to tell clients how they're "performing in AI."

This deserves to be unpacked.

What Is "Block Flattening"?

First, the bottom line: the AI position data you see in Search Console right now is basically unusable.

Why? You need to know one term: block flattening (the whole AI Overview block recorded as a single ranking unit).

Google doesn't record AI Overviews by looking at where exactly your link sits within the summary. Instead, it treats the whole summary box as one unit and gives it a single overall ranking. That box usually appears above the organic results — so the whole box gets recorded as position 1. And every link inside it, logically enough, gets recorded as position 1 too.

Sounds flattering, doesn't it?

Hold on. Here's an example.

Imagine a choir taking the stage. The lead singer stands front and center; a third-row member is almost entirely hidden behind the curtain. When the judges score, they don't rate individuals — they mark the whole choir as "first to perform." So every member gets "first to perform" on their record.

In AI Overviews terms: the eye-catching card under the first sentence is position 1; an ordinary link tucked inside "show more" is also position 1.

Yet in the report, those two links look identical. In the real world, one gets seen and the other is skipped entirely.

The report says you're on the mountaintop; the reality is you may never have made it past the gate.

Average Position Was Never Trustworthy to Begin With

Some will say: fine, then I'll just look at average position.

I'd advise against it. That metric was already broken before AI came along.

What is average position? Add up your rankings across all searches and divide.

The problem: averages are only reliable when the numbers cluster in the middle. Search rankings refuse to cluster. They shift with where the searcher is, with whether it's mobile or desktop, and with whatever Google feels like showing you at that moment. For the same query, a phone shows three cards and a desktop shows five, some of them hidden behind a button; search again a few minutes later and the wording has completely changed.

Let's do the math.

Your page ranks #1 on a search that triggers an AI Overview, and #19 on one that doesn't. Fifty-fifty. Search Console averages them: 10.

On the report: 10. Nice and steady, comfortably mid-pack.

In reality? Either #1 or #19 — never once #10.

A "tidy" average smooths both extremes away and hands you a sense of stability that never existed. Never mind that for the same keyword and the same page, switching the reporting view can produce several different average positions.

The numbers are right; the conclusion is wrong. That's what makes reports so deceptive.

The Even More Absurd Part: How Impressions Are Counted

Block flattening is just the first layer. The second layer hides in how impressions are counted.

The old Search Console rule: once a result loads on the page, that's an impression — whether or not the user ever scrolls.

So the moment an AI Overview loads, every link in it that's expanded by default racks up impressions. Even if that person reads the first sentence and swipes away, your "impression" is already safely on the books.

And the reverse?

Anything that requires a click to expand is the exception: links folded behind "show more" earn no impression until a user physically clicks them open.

The result is a deeply twisted situation.

Links nobody saw in the body are sitting on a pile of impressions in your account; the collapsed links look like they don't exist — until someone clicks.

One can't tell you whether it was read; the other can't show you whether it was seen. Tell me — what decision is this report supposed to inform?

So What's Actually Happening to Traffic in the Real World?

If the reports are distorted, look at the real world in the research instead.

Seer Interactive's research shows a clear drop in click-through rates for cited sites, with the AI Overview's own CTR also down 61%. Part of why the drop looks that steep is that Search Console counted so many automatic impressions, skewing the whole arithmetic.

Another field study puts it more bluntly: when an AI Overview appears at the top of the page, clicks on regular organic results fall by about 38%. Users get their answer from the summary and walk away.

And researchers have tallied that AI Overviews appear in 21% of searches. Question-style searches trigger them even more often.

Why more often for questions?

Because questions beg a direct answer. Google says the answer outright, and users don't need to click into any site.

The old formula "rank #1 = get visitors" is broken.

So What Should You Actually Watch Now?

A bucket of cold water first: the old method can't be fixed, and new tools won't save you either.

The AI visibility tools on the market mostly sell the story of "I can help you measure AI." But if even Google admits its own reports can't keep up with AI search, on what grounds does a third-party tool claim to measure it more accurately?

My advice comes down to three things.

Treat AI visibility as a yes/no question. Was your brand cited? If yes, it's yes; if no, it's no. Don't agonize over "average position 1.2 versus 1.5" — that number usually tells you exactly one thing: your link got stuffed into that box. Where in the box, nobody knows.

Move your attention to the numbers that can't be re-counted. Real visits, real inquiries, real conversions. However impressions are recorded and however the rules change, those numbers don't budge.

Really want to know what your page looks like in search results? Don't look at reports. Search directly yourself, or use a tool to capture the actual results page. Seeing is believing — and it's more honest than any flattened data.

Finally

The 10 blue links pattern served search for 25 years. That ruler was calibrated for that era.

Back to the Reddit conversation at the start. Mueller also mentioned that Google has already explained these rules in its help documentation — and is asking the SEO community: when the search results page is no longer a tidy list, how should position even be measured?

Even the people who set the rules are rethinking the ruler.

So before you rush to send up that weekly report reading "average position: 10" —

Ask it first: what does this number still tell us?

If it can't answer, get a new ruler.