Key takeaways
- A 2025 SE Ranking study of 10,000 keywords found only 9.2% URL overlap when the same query was run three times on the same day in AI Mode, and 21.2% of queries returned zero overlapping URLs across the runs.
- AI Mode's citations barely overlap with classic organic results (14% URL overlap) or with Google AI Overviews (10.7% URL overlap), suggesting it runs on different retrieval logic entirely.
- Platform-wide shifts can happen overnight: ChatGPT's average citations per response dropped about 27% within a day of the GPT-5.3 rollout on March 4, 2026, and never recovered.
- Google launched a dedicated Search Generative AI performance report inside Search Console on June 3, 2026, giving free hourly, daily, weekly, and monthly visibility into AI Mode impressions.
- Despite URL-level chaos, domain-level leaderboards are comparatively stable. Practitioners generally recommend daily tracking only for brand-defense queries, not everything.
Why "ranking" doesn't really mean the same thing in AI Mode
If you're coming from classic SEO, the word "ranking" carries baggage. Position 3. Position 7. A list you climb. AI Mode throws that model out. Google's conversational search surface, built on Gemini, doesn't return ten blue links for a query, it synthesizes an answer and attaches a shifting set of citations underneath it. Ask the same question twice and you might get a different answer shape, a different set of sources, and a different number of links entirely.
That's not a bug anyone is apologizing for. It's how the system works. AI Mode breaks a single prompt into sub-queries, runs them in parallel (a process generally called query fan-out), and assembles an answer from whatever those parallel searches turn up. The practical effect is that the thing you're trying to track isn't a fixed position, it's a probability distribution.
This matters because a lot of people still check their "AI Mode ranking" the way they'd check a Google position, once a week, screenshot in hand, calling it done. The data below suggests that habit gives you almost no signal.
The headline number: 9.2% overlap on the same day
The clearest demonstration of AI Mode's instability comes from an SE Ranking study covering 10,000 keywords, recapped by Search Engine Journal. Researchers ran the exact same queries through AI Mode three separate times on the same day. The overlapping URLs across those three runs matched only 9.2% of the time on average. Worse, 21.2% of queries returned zero overlapping URLs at all across the three runs, meaning the citation list was essentially rebuilt from scratch each time. Domain-level consistency (same websites cited, even if different pages) was only marginally better at 14.7%.
The same study found AI Mode results barely resemble anything else Google shows for the same query:
- Only 14% URL overlap and 21.9% domain overlap between AI Mode and the classic organic top 10
- Only 10.7% URL overlap and 16% domain overlap between AI Mode and Google AI Overviews, a Google surface running right next to it
Read that twice. AI Mode agrees with organic search roughly a sixth of the time and agrees with Google's own AI Overviews even less. These aren't two views of the same ranking engine. They're different retrieval systems that happen to live under one search box.
To be fair to the study's authors, they flag their own caveat: the fieldwork is from mid-2025, US-only, and AI Mode "is a new and growing system" whose behavior could shift under different conditions or later timing. Treat the exact percentages as directional rather than this month's live number, but the magnitude of the instability, single digits to low teens in overlap, has held up as a pattern through 2026.
It's not just query-to-query noise, it's platform-wide shifts
The same-day volatility above happens even when nothing about the AI model changes. But sometimes the entire citation behavior of a platform flips overnight because of a model update, and that's arguably the bigger risk for anyone relying on infrequent checks.
Promptwatch's data on the ChatGPT side illustrates this well, even though it's a different assistant than AI Mode, because the mechanism is identical: platform-controlled retrieval behavior is not a stable baseline you can assume holds from month to month. Around the GPT-5.3 rollout on March 4, 2026, average citations per ChatGPT search-enabled response dropped roughly 27%, from about 6.4 sources per response the week before, settling at 4.7 to 4.9 sources by late March, with no recovery a month later (ChatGPT Citation Drop After GPT-5.3). That change happened within a day of the model update, not gradually.
A similar overnight jump hit ChatGPT's search behavior again on August 8, 2026: the share of fanout queries using the site: operator jumped from 0.37% to 16.8%, roughly a 46x increase, and average searches per response nearly doubled, from about 1.08 to 1.83, and stayed there (ChatGPT Search Now Uses the site: Operator at Scale). Promptwatch's own framing of this is worth repeating here: "platform-controlled search behavior can change overnight, so continuous monitoring matters more than one-off audits."
On the Google AI Mode side, the same pattern shows up in citation-share data rather than citation counts. Promptwatch's June 2026 tracking found google.com's own share of AI Mode citations climbed from 4.24% in May to 7.31% in June, a 72% jump on top of what the report calls "last month's sixfold jump" (Google AI Mode Citation Share, June 2026). YouTube fell for the first time in that series, from 3.42% to 2.88%, while Reddit, Instagram, and Quora all ticked up slightly. The point isn't the specific percentages, it's that even the top of the citation leaderboard moves by double digits month over month.
What stays stable, and what doesn't
There's a useful distinction buried in all this data: URL-level churn is extreme, domain-level churn is much calmer.
In the SE Ranking study, specific cited pages shuffled constantly, but the domains that tend to get cited, Indeed, Wikipedia, Reddit, YouTube, NerdWallet, stayed relatively consistent across the sample. Google-owned properties (mostly Google Business Profile and Maps listings) accounted for 5.7% of all links and appeared in 9.2% of responses. So if you're tracking "does my domain show up at all," you'll see a steadier signal than if you're tracking "does this exact page show up."
A separate Semrush comparison study backs this up from a different angle: AI Mode responses average 7 unique domains per sidebar, roughly double the 3 domains typical of an AI Overview, and 92% of AI Mode responses include a linked sidebar at all. More domains per response means more surface area, but also more room for any single domain's presence to look noisy day to day even while its underlying authority with the system stays roughly flat.
How content durability varies by type
It helps to know which kinds of pages tend to hold a citation once they earn one, versus which get swapped out quickly. Promptwatch tracked citation half-life on ChatGPT over a 90-day window from July 6 to October 4, 2026, and the pattern is instructive even for AI Mode content strategy, since the underlying content-durability dynamics (fresh news decaying fast, reference material lasting) tend to generalize across LLM-powered search:
| Content type | Still cited after 30 days | Still cited after 60 days |
|---|---|---|
| Landing pages | 78% | 69% |
| Documentation | 76% | 67% |
| Comparisons | 72% | 63% |
| Product pages / reviews / how-tos | lower, reaches half-life around day 74-77 | 55% |
| News | roughly half gone by day 36 | 41% |
(Source: ChatGPT Content Half-Life)
If you're seeing a product page or news piece drop out of citations after a few weeks, that's expected behavior, not a sign something broke. Landing pages and documentation are the ones worth worrying about if they suddenly disappear.
So how often should you actually check?
Given all that noise, the honest answer is: it depends what you're checking for, and checking more often isn't automatically better.
iPullRank's tracking-cadence research found something counterintuitive: tracking 50 prompts once a day produced a more stable monthly visibility score (about 6.1 points of uncertainty) than tracking just 10 prompts five times a day (13.2 points of uncertainty). More unique prompts checked once beats fewer prompts checked repeatedly. Breadth matters more than frequency for getting a reliable read.
Omnia's practitioner guidance, which lines up with what I've seen recommended elsewhere, splits tracking cadence by query type:
| Query type | Recommended cadence | Why |
|---|---|---|
| Brand-defense queries (15-25 queries) | Daily | Alert if inclusion drops below 90%, these are your name and core product terms |
| Category-ownership queries (30-50 queries) | 3x per week | Enough to catch meaningful shifts without noise |
| Competitor monitoring | Weekly | Slower-moving, less urgent |
| Expansion / research queries | Weekly | Exploratory, not defensive |
Semrush's own guidance for AI Overview tracking echoes this: review priority keywords at least weekly to catch real gains and losses "without overreacting to normal daily fluctuations," even though their underlying tracking tool refreshes daily. And there's a dissenting view worth noting from AEO practitioner discussions on Reddit: some argue weekly checks are genuinely enough for visibility, since the source set under an answer changes over weeks rather than days, and that daily checks are mainly useful for catching technical failures like a tracker silently breaking, not real visibility movement.
My take: both camps are right about different things. If you're watching one keyword, daily checks mostly show you noise. If you're watching a wide, well-chosen set of prompts and tracking the trend line rather than any single day's snapshot, daily data earns its keep, especially for brand terms where a sudden inclusion drop is worth knowing about fast.
Free tracking: Google Search Console's new AI report
You don't need a paid tool to get started. Google launched a dedicated Search Generative AI performance report inside Search Console on June 3, 2026, giving site owners impressions, pages, countries, and device data specifically for generative AI features, AI Overviews, AI Mode, and generative Discover, with hourly, daily, weekly, and monthly granularity. It rolled out to a subset of websites first for testing, so not everyone has it yet, but it's the clearest free signal available and a good starting point before investing in a dedicated tracker.
Where dedicated tracking tools come in
Once you've confirmed AI Mode traffic matters for your site, GSC's data gets you impressions and countries but not the competitive picture, which domains are winning citation slots, how your inclusion rate trends day over day, or what content gaps are costing you visibility.
Promptwatch tracks AI Mode alongside Google AI Overviews, ChatGPT, Perplexity, Claude, Gemini, Grok, and more, with prompt-level volume and difficulty scoring, citation trend data broken into 22 content types, and crawler logs showing exactly when AI systems hit your pages. Rather than stopping at "were we mentioned," it layers on content gap analysis and automated content generation so the gaps it finds actually get fixed, which matters a lot given how often the data above shows citation sets rebuilding themselves.
For teams that want a wider lineup to evaluate, the GEO software directory at bestgeosoftware.com catalogs dedicated trackers across price points and feature sets, and the AI rank tracking directory at ai-rank-tools.com is worth a look if you're specifically comparing cadence and refresh frequency across tools.
A quick comparison of tracking approaches
| Approach | Cost | Refresh cadence | Best for |
|---|---|---|---|
| Google Search Console AI report | Free | Hourly to monthly | Baseline impressions and country data |
| Manual spot-checks | Free (time cost) | Whenever you remember | Not recommended given the volatility shown above |
| Dedicated AI visibility platform (e.g. Promptwatch) | Paid, tiers vary | Daily, with historical trend storage | Brand-defense monitoring, competitive citation share, content gap fixes |
| Agency-managed AEO/GEO program | Paid, custom | Continuous | Teams without in-house bandwidth to act on the data |
Practical takeaways
A few things worth doing differently given what the data shows:
- Stop judging AI Mode visibility off a single screenshot. The 9.2% same-day overlap figure alone should kill that habit.
- Track domain presence as your primary metric, and treat exact-URL citation as a secondary, noisier signal.
- Reserve daily checks for brand-defense queries where a sudden drop actually matters. Everything else can run two or three times a week without losing much signal.
- Watch for platform-wide shifts, not just your own content changes. If your citation count or inclusion rate moves sharply across many unrelated queries at once, that's probably a model update, not something you did.
- Expect product pages, reviews, and how-tos to decay out of citation sets faster than landing pages or documentation. Plan content refresh cycles accordingly.
If you're setting up a monitoring program for the first time, 1001 SEO Media works with brands on exactly this kind of GEO and AI visibility strategy, pairing tools like Promptwatch with content and technical SEO work so the tracking data actually turns into fixed citations rather than a dashboard nobody acts on.