A User Types "Yes, Go On" Inside AI Mode. Google Logs It as a Brand New Query on Your Site.

A User Types "Yes, Go On" Inside AI Mode. Google Logs It as a Brand New Query on Your Site.

On August 7, 2026, Search Engine Watch published an observation that looks trivial and is not. Consultant Anastasia Kourou posted a screenshot on LinkedIn of her Search Console performance report showing query strings like "Yes", "Yes, go on." and "Yes, pricing" — strings nobody has ever typed into a search box. Google's John Mueller replied to confirm it and pointed at the documentation. The wording leaves no room: when a user asks a follow-up question inside AI Mode, they are performing a new query, and all impression, position and click data from that response is counted against the new query.

This surfaced now because AI surface coverage just crossed a threshold. Similarweb's 2026 Generative AI Landscape puts AI Overviews on roughly 43% of all Google queries; Semrush Sensor reads about 48%. A year ago the same measurements sat between 13% and 20%. Meanwhile Google only shipped the Search Generative AI performance report into Search Console on June 3, 2026, with data starting May 18, 2026, no historical backfill, and dimensions limited to impressions, pages, countries, devices and dates — no query strings. So one account now carries two measurement standards at once: the new report isolates AI impressions, the old report quietly blends AI Mode conversation fragments into your keyword list, and no field tells you which row is which.

The entire AI visibility tracking category grew over the past 12 to 18 months on one premise: Google would not hand over first-party data. Otterly.AI starts at $29/month for 15 prompts and runs $189/month at Standard. Peec AI's Pro plan is €199/month for 100 prompts. Semrush's AI Toolkit is $99/month per domain standalone. Ahrefs Brand Radar needs an existing plan plus a $398–$699/month add-on. Profound starts around $499/month. The band stretches from free to over $4,500 a month, and the category is now splitting in two: absorbed into large SEO suites as one more tab, or fanning out into a long tail of thirty-dollar subscriptions.

For a small business the real question is not whether to buy another tool. It is that the free, first-party report you already check every week changed what it measures. What follows: how contaminated your data is, what to do this week, and how to separate the two standards without paying a subscription.

The Details and the Full Numbers

Google's own sentence is that a follow-up inside AI Mode is effectively a new query, and every impression, position and click in the new response is attributed to it. Translated into operating terms, your Search Console now blends three different things:

• Traditional search queries — typed by a person, with clear intent.
• AI Overviews impressions — your page cited in a summary block, usually attached to longer, more conversational strings.
• AI Mode conversation fragments — "Yes", "Yes, pricing", "go on", each carrying its own impressions, position and clicks.

The statistical consequences differ. Conversation fragments drag site-wide average position down, because they sit deep in a response while still counting an impression. They also inflate the impression denominator, so click-through rate falls without anything changing on your site. If your monthly report compares August 2026 against August 2025, last year's dataset had AI Overviews on 13% to 20% of queries and this year's has them on 43% to 48%. Those two numbers do not measure the same object.

The June report does not patch the hole. It launched to a limited set of UK properties first — a response to the UK competition regulator's disclosure requirements — and has rolled out in waves since, so many site owners still cannot see it. Even when visible, the Search Console documentation confirms it exposes impression-level dimensions only, and merges AI Overviews, AI Mode and Discover's generative AI surfaces into a single view with no way to split them. In the same week, Search Engine Roundtable logged heavy ranking volatility from August 1 through August 3 with no confirmed update from Google.

What Three Kinds of Readers Should Do Now

Brand owners and small business operators

• Treat August 2026 as a new baseline. Stop comparing average position and CTR against 2025.
• Require your agency to label every report with whether impressions include AI surfaces. A report that cannot answer that has no meaningful growth number in it.
• Move your scorecard from impression growth to branded-query clicks plus form and LINE enquiries. Neither is contaminated by conversation fragments.

Marketers and SEO practitioners

• Build two saved views with the query filter: one excluding strings of eight or more words and strings starting with yes/no/ok/sure, one keeping only those. The first is traditional search; the second is a proxy for AI surfaces.
• Mine the long conversational strings that carry real intent and turn them into H3 questions with two-sentence answers on the page.
• Rebuild rank reporting. Average position lost its comparative basis the moment AI Mode entered the same table; use per-query position distribution instead.

Developers and contract studios

• Pull query-level data daily through the Search Console API into your own store. It bypasses the 16-month UI retention limit and gives you a snapshot nobody can revise.
• Add a classification column at the data layer — traditional, long conversational, fragment — so a client dashboard can switch standards with one toggle.
• Audit server logs for Google-Extended, GoogleOther and the renamed Google-GeminiNotebook user agents to confirm the pages you want fetched are being fetched.

Tool Comparison: What You Are Actually Buying

Note the dividing line. Only the first row measures how many times your site was actually shown; everything below is a sampled estimate from asking an AI a set of prompts. The two cannot validate each other and should not share a chart.

What the Sales Page Leaves Out

• Reading "Yes, pricing" as a demand signal is over-reading. Search Console applies an anonymisation threshold to low-frequency queries, so most rare strings never appear. The fragments you can see are the subset that cleared the threshold, and stripped of conversational context you cannot tell which product was being discussed.
• Sampled visibility scores have no denominator. These tools run a few hundred prompts of their own choosing and compute how often you appear. That ratio bears no mathematical relationship to the impressions or clicks your site earns. A vendor's month-over-month growth line describes their prompt set, not your market.
• A change in counting will be sold as performance. Plenty of agency reports will soon show "AI impressions up 300%". Much of that is the generative AI report switching on for that property, or AI Mode fragments beginning to register. That is a new ruler, not growth.

The No-Subscription Route

Before buying anything, clean up the free first-party data. This route fits in a weekend:

• Step one: build two views. The query filter supports custom regular expressions. Save one view excluding conversational long strings and yes/no fragments, and one keeping only them. Every monthly report uses the first.
• Step two: back up daily. A small scheduled script against the Search Console API writes query-level rows into a database or sheet each day. Free, and it solves the 16-month retention problem.
• Step three: connect Looker Studio. The official free connector turns the two views into one side-by-side dashboard, so a client can see the difference for themselves.
• Step four: sample by hand. Ask your 20 most commercially important questions in ChatGPT, Gemini and Perplexity every fortnight, logging whether you were mentioned. That is what the $189 plan does, with labour substituted for licence.
• Step five: fix the markup. Add FAQPage and Organization structured data to main service pages so AI surfaces can lift a clean question-and-answer block.

• ☐ Saved view excluding AI conversation fragments created in Search Console
• ☐ Monthly report labelled with whether AI surface impressions are included
• ☐ Stopped comparing average position against 2025
• ☐ Daily API backup or export running
• ☐ Long conversational queries rewritten as on-page Q&A blocks
• ☐ Server logs checked for AI crawler access

Frequently Asked Questions

My average position in Search Console suddenly dropped. Have I been demoted?

Rule out the measurement change before you conclude anything about rankings. AI Mode follow-ups are counted as new queries, they typically sit deep in a response while still registering an impression, and they pull the site-wide average down. Apply the query filter to exclude conversational long strings and yes/no fragments, then look again. If average position is stable after exclusion, this is a composition change, not a ranking problem.

I cannot see the generative AI performance report. Did I configure something wrong?

Almost certainly not. It launched to a limited set of UK properties on June 3, 2026 and has been rolling out in waves since; many properties still do not have it. It also does not backfill history — data starts May 18, 2026. Until it appears, use query segmentation in the general performance report as your proxy.

Do I still need to buy an AI visibility tracker?

If your average deal size is small and your customers are local, not yet. Manual sampling of 20 core questions plus Search Console segmentation already answers "am I mentioned" and "which pages get cited". Revisit paid plans when you need cross-language, cross-market comparison or a board-level report.

Do these fragment queries hurt my SEO rankings?

Not directly. This is a counting behaviour, not a ranking signal. The real risk is to your decisions: if you cut pages based on a contaminated click-through rate, that is where the actual loss happens.

Is structured data still worth doing?

Yes, but the purpose shifted. Its main value now is not winning a rich result style; it is making it easy for an AI surface to extract a citable question-and-answer block. FAQPage markup and clear H3 questions remain the cheapest citable material you can produce.

My Take

The consensus advice is that AI search has arrived, so buy an AI visibility tool. I expect the opposite: within 12 months this category collapses into one tab inside a larger suite and standalone subscription prices halve. Not because the tools are bad, but because the condition that created them is disappearing. The category exists only because Google withheld first-party data, and the generative AI performance report is Google releasing that data one column at a time. Once first-party impressions are a free field, a paid sampled estimate loses the argument. The survivors will pivot from measurement to content supply.

For a contract studio like ScriptWalker, the opportunity is not reselling licences — it is data cleaning and report reconstruction. Package it as a fixed-fee build plus a monthly retainer: a client-specific daily snapshot database on the Search Console API, a query classification layer, and a one-page Looker Studio dashboard showing how much of their "search traffic" is traditional search and how much is an AI surface. No SaaS can do this for a client, because the classification logic has to understand that client's product vocabulary. That is a structural advantage for service firms and one of the few gaps subscriptions have not eaten yet.

Sources

• Google Search Console Help: Search performance report, AI Mode section (primary)
• Google Search Console Help: Generative AI performance report (primary)
• Google Search Central Blog: Introducing Search Generative AI performance reports in Search Console, June 3, 2026 (primary)
• John Mueller's confirmation on Anastasia Kourou's LinkedIn post (primary)
• Search Engine Watch: Google is mixing AI Mode replies into Search Console query data, August 7, 2026
• Search Engine Roundtable: Almost Half Of Google Searches Have AI Overviews & Climbing
• Similarweb: 2026 Generative AI Landscape; Semrush Sensor
• Search Engine Roundtable: August 2026 Google Webmaster Report

Want Someone to Pull Your Reports Apart?

If you check Search Console every month but have no idea how much of it comes from AI surfaces, we can build you a segmented dashboard and a daily backup pipeline. Get in touch:

• Email: [email protected]
• Phone: 0916-224-047
• LINE: @ufv9089p