Why do two AI visibility tools show different numbers for the same brand?

Two AI visibility tools show different numbers for the same brand because each one picks its own prompts, its own engines, its own sampling frequency and its own formula, and all four choices move the result. None of that is vendor error. There is no official denominator in AI search, so every tool invents one.

Start with the formula, because it produces the largest gaps and the least discussion. One tool may report the proportion of tracked answers your brand appears in at all. Another may report your mentions as a share of every brand mention in those same answers. A third may count only answers where a vendor recommendation was actually made. Those are three different questions and they will not agree even on identical data.

The prompt set does the next most damage. A tool that tracks 40 prompts you chose will report a different picture from one tracking 400 generated from a category model, and neither sample is the whole market. Where the vendor asks you to nominate competitors, the denominator becomes a pool you assembled, which flatters or punishes you depending on who you listed.

Then come the engines and the sampling. Engines overlap far less than people assume, so a tool covering three engines and a tool covering four are reading largely separate pools. And because every engine gives different answers to the same prompt on different runs, a tool that runs each prompt once reports a number with an interval around it wide enough to swallow most month-on-month movement. The rest of this page puts measurements on each of those four.

Is one of the tools simply wrong?

Neither tool is wrong in the sense buyers mean when they ask. Both can be internally correct, computed exactly as documented, and still disagree by a factor of two, because they are answering different questions about the same underlying answers.

Similarweb's own documentation is the clearest published example, and it is worth reading because the vendor shows both figures side by side rather than picking the flattering one. In its May 2026 explainer on AI share of voice, the same campaign data covering 5,018 tracked answers and 11,295 brand mentions produces a brand visibility figure of 26.72%, meaning the brand appeared in 1,341 of the 5,018 answers, and a brand mention share of 11.87%, meaning it accounted for roughly one in nine of all brand mentions. The company appears often and holds a small slice of the conversation. Both statements are true.

What follows is that "which tool is accurate" is the wrong question, and asking it tends to end with a buyer picking the tool that reports the higher number. The better question is which definition matches the decision you are making. Presence rate tells you whether you are in the conversation. Mention share tells you how much of it you own against competitors. A brand entering a category wants the first. A brand defending a position wants the second.

Where a tool becomes genuinely unusable is when it will not tell you which it computed. A score published without its formula cannot be reconciled with anything, cannot be compared across months if the vendor changes it, and cannot be explained to a finance director. Our page on reporting AI visibility to executives covers which of these numbers survives that conversation.

How much do the engines themselves disagree about what to cite?

The engines disagree with each other almost completely, which is why engine coverage changes a tool's output so much. Two tools can run the same prompt set and report different brands simply because one included Copilot and the other did not.

Tannenbaum put 15 fixed commercial prompts to ChatGPT, Microsoft Copilot, Google and Perplexity on 6 June 2026, collecting 589 citation observations across 528 unique URLs and 356 domains. The mean pairwise Jaccard similarity of the cited URLs was 0.0079, the median was zero, and 84.9% of engine pairs shared no cited URL at all. Hold that at arm's length for the right reasons: 15 prompts on a single day is a small, brittle sample, the prompts were commercial rather than random, and the author's company sells engine-measured visibility, which is the alternative the paper argues for. The direction survives those objections even if the precision does not.

A second study points the same way from inside the pipeline. Chen and Koudas, analysing 158,099 answers and 468,037 citations across two 90-day collections, found that citations track each engine's own retrieval ranking and rarely match what Google or Bing return for the same question, Gemini included. So the engines are not drawing from one shared index with different presentation layers. They are drawing from different pools.

Our own category study found the same effect at a finer grain. When we ran 27 separate buyer queries across all four engines for pharmaceutical process skids, the shortlist changed with the phrasing of the question as well as with the engine: the same lane held queries an independent fabricator wins outright and queries owned entirely by global brands. A tool that samples a different wording of your question is sampling a different market.

How much does a single engine disagree with itself?

A single engine disagrees with itself nearly as much as two engines disagree with each other, and this is the cause buyers find hardest to believe. Asking the same question twice does not return the same sources.

Schulte, Bleeker and Kaufmann ran four AI search engines daily across four verticals from 24 January to 20 March 2026, 45 to 46 days per campaign, with a second dataset of prompts submitted repeatedly inside a single day so run-to-run randomness could be separated from change over time. Cited sources overlapped between consecutive days by about 34% in consumer electronics, 36% in sporting goods, 38% in real estate and 42% in telecommunications. Rank-aware overlap was lower still, about 21% to 26%, so the order of sources moved as well as the membership.

The same-day result in that study is the one that settles the argument. Repeated runs inside a 24 hour window overlapped about 32% to 43%, and the authors conclude that this intra-day randomness accounts for most of the observed instability rather than genuine change over time. So when two tools sample the same engine on the same day and report different sources, the engine did that, not the vendor.

Drift sits on top of randomness. Chen and Koudas report that the repeatable part of citation variation moves over weeks, which means a score, a report and a before-and-after comparison all describe a target that is moving while you measure it. The honest handling is to report an interval rather than a point, because a point implies a precision the measurement does not have. Our page on why AI engines give a different answer every time covers the mechanism behind it.

How many runs does a visibility number need before it means anything?

A visibility number needs roughly 100 observations before it can carry a plus or minus 10 point interval, and roughly 400 for plus or minus 5. That is not a finding from any study. It is the normal approximation to a binomial proportion at 95% confidence, and it sets the floor under every claim any vendor makes.

Put that next to the instability above and the implication is uncomfortable for most reporting in this category. A tool running 40 prompts once a month on one engine has 40 observations, so per the arithmetic above its monthly share figure carries an interval of roughly plus or minus 15 points. A move from 18% to 24% in that setup is not a result. It is the same number twice.

Depth is also not free, which is why vendors differ. Running each prompt five times across four engines multiplies the API cost by twenty, and credit-based pricing makes that visible on the invoice. So a cheaper tool is often cheaper because it samples less, which is a legitimate trade rather than a defect, and the only problem is when the sampling depth is not published. Our comparison of AEO tools in 2026 records which vendors state their run count.

What to do with this practically: stop reading the score and start reading the source list. Which of your URLs appeared as a cited source, on which questions, is a list rather than an estimate, and a list is stable enough to act on even when the share figure around it is not. Our page on how many prompts you need to track AI visibility works through how to size a prompt set properly.

Which questions should you ask a vendor before comparing two tools?

Ask five questions, and ask them of both vendors before you compare any numbers. The table sets out what each one exposes and what a good answer looks like, so a trial can be run as a real comparison rather than a beauty contest between two dashboards.

What to ask	Why the number moves	A usable answer
What exactly is the formula?	Presence rate and mention share can differ by more than a factor of two on identical data	The formula written out, with the denominator named, and a commitment not to change it silently
Which prompts, and who chose them?	A prompt set you nominated measures the market you imagined; a generated one measures the vendor's model of it	The full list, exportable, with a documented process for adding and retiring prompts
How many times is each prompt run, and how often?	Run-to-run variation inside one day already runs to roughly 60% of sources changing	A stated run count per prompt per period, and an interval or error bar on the reported figure
Which engines and which surfaces?	AI Overviews and AI Mode are separate surfaces, and the engines share few cited URLs	Engines and surfaces listed separately, never summed into one headline score
Does a mention count the same as a citation?	A brand named in prose and a page cited as a source are different events with different causes	The two reported separately, with the cited URLs available per answer

One practical rule covers the comparison itself: align the prompt set, the engines, the window and the run count before you compare anything, or accept that you are comparing design choices rather than brands. Where that is too much work for a trial, compare the cited URL lists instead of the scores. Our guide to tracking AI visibility covers how to keep the measurement stable once you have chosen.

What else do people ask about AI visibility tools disagreeing?

The five questions below come up during visibility tool trials, usually in the week the two dashboards first get compared side by side.