Appearing in an AI-generated response may look like a visibility success.
Your website is mentioned, a link is provided and the response describes your work in confident detail. An AI visibility report might record the appearance and add it to your brand’s share-of-voice figures.
But what if the AI has misunderstood what your website actually does?
I recently entered the domain name AIVisibilityTesting.com into Google AI Mode. Unsurprisingly, the response mentioned the website.
The appearance itself was not particularly meaningful. Searching for an exact domain gives Google a strong indication of which website the user wants.
What mattered was how AI Mode described it.
It presented AIVisibilityTesting.com as an automated software platform capable of scanning multiple AI engines, tracking prompts and providing competitor intelligence.
None of those facilities currently exists on the website.
The experiment exposed an important limitation of counting AI mentions:
A website can be prominently visible in an AI response while being represented inaccurately.
What Google AI Mode said
The response began:
“AIVisibilityTesting.com is a dedicated tool and platform designed to help website owners and brands measure how they appear and rank within generative AI search engines.”
It then attributed several capabilities to the site:
- scanning ChatGPT, Gemini, Claude and Perplexity;
- evaluating whether brands are recommended for selected prompts;
- tracking website citations;
- measuring competitors’ visibility;
- providing a structured environment for automated testing.
It also described the site as addressing Generative Engine Optimisation and offered to provide advice about optimising content for AI engines.
This sounded like a plausible description of an AI visibility monitoring product.
It was not an accurate description of my website.
What the website actually does
AI Visibility Testing is an independent educational website documenting practical experiments into how brands, products and websites appear in AI-generated answers.
The site currently covers subjects such as:
- selecting commercially relevant prompts;
- repeating the same AI search;
- recording changing recommendations;
- measuring coverage and share of voice;
- distinguishing recommendations from supporting citations;
- interpreting what visibility results can and cannot establish;
- recognising when manual testing becomes difficult to scale.
It does not provide software into which someone can enter a brand and automatically scan several AI platforms.
It does not maintain a customer dashboard, run automated audits or provide its own competitor-monitoring service.
AI Mode had understood the general subject correctly. It had misunderstood the site’s role within that subject.
Correct subject, invented product
The response was not entirely fabricated. That is partly what made it convincing.
| AI Mode’s claim | Assessment |
|---|---|
| The site concerns visibility in AI search | Correct |
| It examines share of voice | Correct |
| It discusses prompts, recommendations and citations | Correct |
| It is associated with John Greer | Correct |
| It may be relevant to brands and website owners | Reasonable |
| It provides an automated multi-engine scanner | Incorrect |
| It offers competitor-monitoring software | Incorrect |
| Users can run automated audits through the platform | Incorrect |
| It provides a GEO optimisation service | Incorrect |
The answer combined correct information with believable but invented functionality.
It did not look obviously nonsensical. A reader unfamiliar with the website could easily accept it.
The citations may explain how the error formed
The response listed 17 references, which initially made it appear well supported.
Only two were pages from AIVisibilityTesting.com:
- the About AI Visibility Testing page;
- the AI Visibility Testing homepage.
The remaining sources concerned subjects such as:
- AI visibility software;
- brand monitoring;
- GEO services;
- prompt tracking;
- citation measurement;
- competitor analysis.
Those pages may accurately describe what AI visibility tools generally do. They do not establish that AIVisibilityTesting.com provides those capabilities.
Citation volume therefore did not equal claim support.
Most of the references helped explain the wider product category. They did not substantiate the specific claims being made about my website.
I cannot see AI Mode’s internal reasoning, but the finished response is consistent with an inference such as:
- The domain resembles the name of an AI visibility product.
- The website discusses AI visibility measurements.
- Other sources describe the capabilities of automated monitoring tools.
- Those general capabilities are attributed to this particular website.
The category was understood, but the individual entity was not.
The domain name and tagline may have contributed
The name AI Visibility Testing is deliberately descriptive. It also sounds like the name of a product or service.
The tagline, “Measure How Your Brand Appears in AI Search,” can be interpreted in two ways:
- the website teaches visitors how to conduct the measurement;
- the website performs the measurement for them.
The first interpretation is intended. AI Mode selected the second.
Because the site was only a few days old, Google may have had relatively little indexed first-party material from which to distinguish it from the wider software category.
The error remains AI Mode’s, but clearer wording on the site could reduce the opportunity for that inference.
An exact-domain appearance is not evidence of discovery
Searching for a website’s exact domain is not a meaningful test of whether it is being discovered for relevant non-branded questions.
The query effectively tells Google which website to discuss.
This experiment does not establish that AIVisibilityTesting.com is already being discovered for searches such as:
- how to test AI search visibility;
- how to measure AI share of voice;
- tools for checking AI brand mentions;
- how to monitor citations in AI answers.
The domain query revealed something different:
It tested how AI Mode currently understands and describes the website.
That understanding matters because it may influence the role assigned to the site in other responses.
If AI Mode believes AIVisibilityTesting.com is a software platform, it could include the site when users search for automated monitoring tools.
It might also overlook the site when someone specifically wants independent experiments or guidance about manual testing.
Search Console showed why the distinction matters
Shortly after the domain experiment, I examined the site’s first Google Search Console data.
The numbers were extremely small. The site had received only 13 property-level impressions and no clicks, so the data could not establish meaningful traffic or ranking performance.
The disclosed search queries were nevertheless closely aligned with the site’s subject:
| Search query | Impressions |
|---|---|
| measure ai visibility | 2 |
| is there a way to a/b test content for ai visibility? | 1 |
| test ai visibility | 1 |
| how to set up real-time prompt tracking for brand visibility? | 1 |
| how to test if ai understands my brand | 1 |
| measure your ai brand presence | 1 |
| ai brand visibility assessment | 1 |
These pages appeared at low positions, and some impressions may have resulted from my own testing. The standard Search Console Web report also cannot establish whether an impression came from traditional results, AI Mode or another Google search feature.
The queries still revealed the general subjects Google was associating with the site:
- measuring AI visibility;
- testing brand representation;
- assessing AI brand presence;
- monitoring prompts;
- testing content changes.
One query was especially relevant:
“how to test if ai understands my brand”
When I tested whether AI Mode understood my own brand, it identified the correct subject and then invented the product.
The query about real-time prompt tracking was also revealing. It represents a potentially valuable audience, but it reinforces the need to explain the site’s role accurately.
AIVisibilityTesting.com discusses prompt monitoring and when automation may become necessary. It does not currently provide real-time tracking software.
The audience may be right while the description is wrong
Appearing for searches about AI visibility tools would not necessarily be a bad outcome.
People investigating monitoring software are close to the website’s intended audience. Before selecting a tool, they may need to understand:
- which prompts are worth monitoring;
- why one search is insufficient;
- what share of voice measures;
- how citations differ from direct recommendations;
- why results change between runs;
- when manual testing becomes impractical.
An independent testing website could help those readers understand the problem before they evaluate an automated solution.
There is a legitimate journey:
A user investigates AI visibility software, discovers independent experiments, understands the measurements more clearly and then makes a better-informed tool choice.
The problem is not necessarily the audience. It is the expectation created by the inaccurate description.
Compare these two statements:
AIVisibilityTesting.com is an automated platform that scans major AI engines.
And:
AIVisibilityTesting.com is an independent educational site that documents manual AI visibility experiments and explains when automated monitoring tools become useful.
Both could interest someone researching AI visibility software. Only the second accurately explains what the visitor will find.
A visitor expecting an automated audit, monitoring dashboard and competitor report may leave when presented with articles and experiments.
The audience could be appropriate while the promised product is wrong.
A better way to assess an AI mention
Counting whether a website appeared is only the beginning.
This experiment can be assessed through a wider set of questions:
| Question | Result in this experiment |
|---|---|
| Was the website mentioned? | Yes |
| Was it linked? | Yes |
| Was the general subject relevant? | Yes |
| Was the audience potentially relevant? | Yes |
| Was the website’s role described accurately? | No |
| Were its capabilities described accurately? | No |
| Would a visitor find the product that was promised? | No |
| Did the citations support the site-specific claims? | Mostly not |
A basic visibility report might classify this as a successful appearance.
A more useful assessment would also examine:
- description accuracy;
- the role assigned to the website;
- the relevance of the query;
- commercial alignment;
- visitor expectation;
- support for the claims being made.
This result had strong branded visibility and potentially relevant audience alignment, but poor description accuracy.
It was neither a simple success nor a worthless appearance.
Automated collection still requires interpretation
This experiment does not undermine the value of automated AI visibility monitoring.
Manually checking hundreds of prompts across several platforms would be extremely time-consuming. Software can collect mentions, citations, recommendation positions and competitor appearances at a scale that would be impractical manually.
But collection and interpretation are different tasks.
A system may correctly detect that a brand appeared while missing that it was assigned the wrong role or described as providing a product that does not exist.
Automation can identify the responses worth examining. Human review determines what those appearances actually mean.
That means automated monitoring may answer:
Where and how often did the brand appear?
Someone must still consider:
Was the appearance accurate, relevant and commercially useful?
Clarifying the site creates another experiment
Rather than simply accepting the error, I can make the website’s role more explicit and test whether AI Mode’s interpretation changes.
The homepage and About page can include a statement such as:
AI Visibility Testing is an independent educational website, not an automated AI visibility monitoring tool or optimisation service. It documents practical manual experiments examining how brands, products and websites appear in AI-generated answers.
A second statement can preserve the site’s connection to monitoring software:
The site explains what can be learned through manual testing, where those methods become difficult to scale, and when businesses may benefit from specialist monitoring tools.
The aim is not to prevent the site from appearing for searches related to AI visibility software.
The aim is for it to appear in the correct role.
The planned follow-up test
The current AI Mode response and its cited sources provide a baseline.
The next steps are:
- Preserve the original response and screenshots.
- Record the capabilities incorrectly attributed to the site.
- Add explicit clarification to the homepage and About page.
- Allow Google to recrawl the updated pages.
- Repeat the same domain query.
- Compare the new description with the original.
- Test relevant non-branded queries about manual testing and monitoring tools.
The follow-up will examine whether clearer first-party wording changes AI Mode’s understanding.
Possible outcomes include:
- the site is correctly identified as an educational publisher;
- the invented software features disappear;
- the same misunderstanding continues;
- the site remains visible for tool-related searches but in the correct supporting role.
Whatever happens, it will provide evidence about whether clearer first-party information can correct an AI synthesis error.
Final conclusion
Searching for the exact domain made an appearance highly likely. That appearance was not the important result.
AI Mode found the correct website, understood its general subject and then assigned it capabilities it does not possess.
It reached a potentially relevant audience but created the wrong expectation.
That leads to a broader principle for AI visibility testing:
Do not ask only whether your website appeared. Check what the AI said, why it included you and whether the description matches what you actually provide.
A mention is not automatically valuable merely because it can be counted.
Its quality depends on the context, accuracy and commercial relevance of the appearance.
Being visible is useful. Being understood correctly is better.