Can you just ask ChatGPT whether it recommends your company?

Asking ChatGPT is free and worth doing once, and a single answer tells you almost nothing about whether it recommends your company, because the answer is sampled rather than looked up.

SparkToro, working with Gumshoe.ai, recruited roughly 600 volunteers across five survey rounds and ran individual brand-recommendation prompts about 100 times each through ChatGPT, Claude and Google AI. On Google AI Mode the chance of getting the same brand list twice in 100 tries came out under 1%, and it took roughly 124 queries before the same two brands appeared together twice, according to the SparkToro session published on 8 May 2026. Average visibility for the top three brands ran about 64% on ChatGPT, 73% on Claude and 68% on Google AI, with a standard deviation near 20 points. Gumshoe.ai sells AI tracking software, so a finding that presence needs many runs to measure is one that suits its business, and the design remains the best-powered version of this test we have read.

The published churn in cited sources points the same way. Ahrefs analysed more than 43,000 keywords, each with at least sixteen recorded AI Overviews over a month, and found that about 45.5% of cited sources were entirely new between consecutive captures of the same query, while the average cosine similarity between consecutive answers stayed at 0.95 (published 11 November 2025). The meaning of the answer is stable; the names and links inside it are not.

So treat your own check as a sample of one from a distribution with a wide spread. Being named once does not mean you are usually named, and being absent once does not mean you are usually absent. Both readings feel like facts and neither is one.

How many times would you have to ask before the answer means something?

Roughly 100 runs of the same prompt produces an answer you can quote with an interval around it, and roughly 400 produces a tight one. Those two numbers are arithmetic rather than opinion.

A share estimated from repeated yes-or-no observations carries a confidence interval that narrows with the square root of the sample. Per the normal approximation to a binomial distribution, at 95% confidence about 100 observations gives an interval of plus or minus 10 percentage points and about 400 gives plus or minus 5. A brand named in 4 of 10 runs could therefore sit anywhere between roughly 10% and 70%, which is not a number worth putting in a report.

Julius Schulte, Malte Bleeker and Philipp Kaufmann argue the same point from the measurement side, writing that "Answers can vary across runs, prompts, and time, making one-off observations unreliable" and that visibility should be characterised as a distribution rather than a single-point outcome (arXiv 2604.07585, submitted 8 April 2026).

Two practical consequences follow for a team doing this without a tool. First, pick fewer prompts and run each of them more times, rather than many prompts once each: ten prompts at ten runs will tell you more than a hundred prompts at one run, because the second design cannot distinguish any result from chance. Second, hold the prompt set still between measurements. A rewritten prompt produces a new baseline rather than a comparison, which is the most common way a self-check turns into a story about improvement that never happened.

Our own practice is built around the same constraint. The SIGNALS methodology runs the baseline twice, a week apart, before any fix is made, specifically so the second run measures the noise in your own category before anyone claims a change.

Why does your own ChatGPT account give you a flattering answer?

Your own ChatGPT account is the worst place to run this test, because the answer it gives you is conditioned on what the product already knows about you and your interests.

Several things make a logged-in reading unrepresentative. Memory and custom instructions carry your company and your interests between conversations. Earlier turns in the same thread steer the reply, so asking "who are the best vendors in our category" after a conversation about your own product is a different question from the one a buyer asks. And personalisation means the answer a stranger receives for the same words is not necessarily the one you see; our guide on whether ChatGPT recommends different vendors to different people covers how far that goes.

There is a deeper version of the problem that signing out does not fix. Mahe Chen and Nick Koudas, studying 158,099 generative engine answers carrying 468,037 citations, found that ChatGPT writes brands it already knows into its own searches, prefers sites it knows, and names brands with no supporting page at all, and that changing only a brand name reordered its product rankings over identical facts (arXiv 2610.15033, submitted 10 October 2026). Familiarity is part of the mechanism rather than a bias to be controlled away.

What to do instead is cheap. Run the check in a logged-out or temporary session, with memory off, one question per session, and never after a conversation about your own company. Then have somebody outside the company run the same prompts and compare. Where the two disagree, your own reading was the unreliable one.

What does a self-check actually tell you?

A self-check tells you reliably whether an engine can produce your name at all, and tells you nothing reliable about how often it does. Reading it for what it is good at makes it genuinely useful.

Four things come out of a careful self-check and hold up. Whether you appear at all across ten clean runs, which is a yes-or-no question that does not need statistics. Which competitors appear, which is usually a longer list than the sales team expects. What the engine cites when it answers, which tells you whose pages are doing the work in your category. And whether the facts in the answer about your company are right, which is a different problem with a different fix.

Three things do not come out of it. A share of voice, because that is a rate and a rate needs a sample. A trend, because two unstable readings cannot establish a direction. And a cause, because the answer does not say whether you were absent from the retrieved set or present in it and passed over.

The asymmetry is the useful part and it is rarely stated. Absence is cheap to establish and presence is expensive. Ten runs with zero mentions is strong evidence of a real problem, because the probability of missing a brand that normally appears half the time ten times in a row is about one in a thousand. Ten runs with four mentions is not evidence of anything in particular. So spend your free checks looking for zeros, and spend money on measurement only once the zeros are gone.

Worth separating from all of it: the question of whether your pages can be read at all. An engine that never fetched your site cannot name you whatever else is true, and our guide on checking whether AI can read your website covers that first.

How do you run a free self-check properly?

Run a free self-check with ten buyer questions, ten runs of each, in clean sessions, across four engines, and record the cited sources rather than only the brand names.

Step	What to do	Why it matters
Pick the questions	Ten questions in the words buyers used on real sales calls, not keywords	Prompt wording changes the answer more than anything you control
Clean the session	Logged out or temporary chat, memory off, one question per session	Your account history flatters you
Repeat	Ten runs of each question, same wording every time	A single run cannot be distinguished from chance
Cover the engines	ChatGPT, Perplexity, Google AI Overviews and Claude, scored separately	A blended number hides a zero on one engine
Record sources	Log the cited URLs, not just who was named	Shows whose pages win your category and why
Wait and repeat	Run the whole set again a week later before changing anything	The second run measures your own noise floor

Score each engine on its own. Our deep category study shows why: across 27 buyer queries in three pharmaceutical skid lanes, one fabricator was named in the top three on Google, Perplexity or Claude in twelve of the fourteen queries in its lane while scoring zero on ChatGPT in eleven of those same fourteen. Averaged together, that firm looks mid-table. Read per engine, it has one fixable problem on one engine across an entire lane of queries.

Budget about two hours for the first pass and an hour for the repeat, and write the results into a sheet with one row per run. The work is boring and it is the cheapest honest measurement available to anyone, which is why so few teams do it before buying a tool.

Why can't you just look in Search Console instead?

Search Console now reports impressions in Google's own AI features and still cannot answer this question, because what it shows is narrow and what you want sits mostly outside Google.

Google's generative AI performance report covers AI Overviews and AI Mode, and reports "how many times links to your site were shown to a user in a generative AI feature on Google Search" (Search Console Help, read 9 November 2026). Impressions in Google's own surfaces, in other words, with no clicks and no position, and nothing about ChatGPT, Perplexity or Claude. The data is also absent from the programmable interface: the Search Analytics API's type parameter still accepts only discover, googleNews, news, image, video and web (read 9 November 2026), so anything you build on the API is a web search picture.

Reconstructing ChatGPT's citations from outside turns out to be hard in a measured way. Jiang, Zhang, Xing, Chang and Xiao studied production ChatGPT web search across four settings from ChatGPT 5.2 to 5.5 and found that Google Search and OpenAI's own Responses API, combined, recover only about one quarter of the URLs ChatGPT cites, and that both proxies match better at hostname and entity level than at exact URL (arXiv 2610.15382, submitted 11 October 2026). They also report that the Thinking setting frequently runs multiple retrieval rounds while Instant settings stay mostly single-round.

Two things follow. No report built from Google data can tell you what ChatGPT does, and the honest way to describe any external measurement of AI visibility is as a sample of engine behaviour rather than a record of it. If you want to see the prompts that do reach your site, Bing Webmaster Tools is the most useful free view, and our guide on what people type into AI search shows what that looks like on this site.

When is a single check enough?

A single check is enough when you are testing for absence rather than measuring presence, and that covers more real situations than it sounds like.

Three cases justify stopping after one pass. You asked the obvious buying question in your category and got a list with no sign of you and no sign of anyone like you, which says the engine is answering from a different kind of source than vendor sites. You asked about your own company by name and the facts came back wrong, which is a specific, checkable problem with a specific fix. Or you asked and the engine could not fetch your site, which is a technical fault to hand to a developer rather than a visibility question at all.

One case does not justify it: anything you plan to report as a number. A share, a position, a percentage of prompts or a month-on-month comparison all need repeated runs, and presenting a single reading as any of those is how an internal report becomes a story nobody can defend when a competitor's name shows up next week.

Where a self-check leaves you is usually with a decent list of symptoms and no diagnosis. The gap between "we are not named" and "here is why" is where the work sits, and it needs the cited sources behind each answer rather than the answers themselves. A free visibility assessment runs your buying questions across ChatGPT, Perplexity, Google AI Overviews and Claude, scores each engine separately, and shows which pages win each phrasing today, which is the part a manual check cannot reach in an afternoon.

What else do people ask about checking AI visibility yourself?

Marketers ask these five questions most often before paying anybody to measure AI visibility for them.