AI visibility testing can be carried out manually.
In fact, I think there is considerable value in starting that way.
Running searches yourself helps you understand what you are actually measuring, how much AI-generated answers can change, the difference between a mention, recommendation and citation, and why the prompts you choose matter so much.
But manual testing has an obvious limitation:
it does not scale particularly well.
A handful of prompts on one AI platform is manageable.
Dozens of prompts across Google AI Mode, ChatGPT, Gemini, Perplexity and other AI systems, repeated over time and compared against several competitors, quickly becomes a much larger monitoring job.
That is where AI visibility software becomes useful.
This section of AI Visibility Testing explores that transition:
when manual testing is enough, when automation starts to make sense, which tools may help, and what businesses should look for before paying for them.
It also considers another possibility that can easily be overlooked:
you may not want to operate the software yourself at all.
Some businesses will want to manage their own monitoring. Agencies may need software capable of managing many clients. Other businesses may prefer to hire someone to handle the entire process.
Those are different requirements, and they should not automatically lead to the same recommendation.
What this section is — and what it isn’t
AI Visibility Testing is not an AI visibility monitoring software platform.
This is an independent website documenting practical AI visibility testing, experiments and the tools and services that can make that work easier.
The basic principle behind this section is:
Use manual testing to understand what matters. Use software when repeating that useful testing manually no longer makes practical sense.
That distinction is important.
Software can automate large amounts of repetitive monitoring.
It can check prompts, record which brands appear, compare competitors, identify citations and show changes over time.
But software cannot automatically decide whether the prompts you are monitoring represent customer situations that genuinely matter to your business.
Human judgement still comes first.
Start with the measurement, not the software
Before choosing an AI visibility platform, it is worth understanding the process it will eventually automate.
For example, suppose a potential customer asks an AI system:
Which accountancy firms in Birmingham should I consider for my manufacturing business?
A business testing that query might want to record:
- whether it appears;
- whether it is actually recommended;
- where it appears;
- which competitors appear;
- how it is described;
- which websites are cited;
- whether the answer changes when the search is repeated.
Now imagine doing the same thing across 30 or 50 customer situations, several AI platforms and repeated monitoring periods.
The need for automation becomes much clearer.
But there is little value in efficiently monitoring 100 prompts if those prompts have little connection to the situations in which genuine customers might choose your business.
If you are completely new to the subject, start with the AI Visibility Testing hub before choosing software.
If the bigger challenge is deciding what to monitor, the Customer Journeys section looks at building prompts around real customer needs.
Do you actually need AI visibility software yet?
Not necessarily.
If you are monitoring five or ten important prompts occasionally, a spreadsheet and some manual searches may still be enough.
Manual testing can be particularly useful while you are:
- learning how AI-generated recommendations behave;
- experimenting with potential prompts;
- identifying genuine competitors;
- deciding what should be measured;
- working out which parts of the customer journey matter.
Software becomes increasingly useful when you want to:
- monitor more prompts;
- cover several AI platforms;
- repeat checks consistently;
- compare several competitors;
- maintain historical results;
- identify changes over time;
- reduce the amount of repetitive manual work.
There is no universal number of prompts at which every business should switch to software.
The better question is:
Has monitoring become large or repetitive enough that automation would now save meaningful time or provide information that would otherwise be difficult to collect consistently?
There are different types of AI visibility software buyer
One of the things that became apparent as I started looking at the AI visibility software market was how similar many platforms initially appeared.
The central service is usually some variation of:
prompts → AI platforms → brand appearances → competitors → citations → historical monitoring.
The more important differences often emerge when you ask:
Who is actually going to use the software?
Our overview of the AI visibility tools and services landscape looks at this in more detail.
Broadly, I think there are three main routes.
Route 1: I want to monitor my own business
This could be a business owner, marketer, SEO specialist or internal marketing team.
The requirements might include:
- straightforward setup;
- sensible prompt allowances;
- monitoring of the AI platforms that matter;
- competitor comparisons;
- citation tracking;
- historical trends;
- access to the underlying AI responses;
- understandable reporting;
- reasonable pricing as the prompt set grows.
The biggest mistake may be assuming that the platform with the most features is automatically the best choice.
A small business monitoring one brand does not necessarily need agency white-labelling, large numbers of team accounts, sophisticated permission controls or enterprise APIs.
Our guide to choosing AI visibility software for a small business sets out the criteria I think are worth considering before subscribing.
It also starts with the question that should probably come first:
Do you need software yet at all?
Route 2: I am an agency or consultant managing clients
An agency is solving a different problem.
Monitoring 30 prompts for one business may be relatively straightforward.
Monitoring:
30 prompts × 20 clients × several AI platforms
is a very different operational task.
Now features such as these can become much more important:
- separate client projects;
- flexible prompt allocation;
- multiple team members;
- client access;
- automated reporting;
- white-label reports or dashboards;
- integrations;
- data exports;
- permissions;
- pricing that remains viable as client numbers grow.
For an agency, the question isn’t merely whether the platform monitors AI visibility effectively.
It is:
Can we build an efficient, useful and commercially viable client service around it?
The AI Visibility Software for Agencies guide looks at those requirements separately.
This matters because software could be an excellent choice for a small business while being much less suitable for an agency — or the other way around.
There does not have to be one universal winner.
Route 3: I don’t want to manage any of this myself
There is another perfectly reasonable option.
A business may decide:
I understand why monitoring AI visibility could be useful, but I don’t want another software subscription and I don’t want my team operating another dashboard.
In that situation, the solution may not be software.
It may be an agency, consultant or specialist service that manages the monitoring on the business’s behalf.
But then the buying criteria change again.
The business should be asking things such as:
- How will you choose the prompts?
- How will they relate to our customer journey?
- Which AI platforms will you monitor?
- How often will monitoring take place?
- How do you distinguish mentions, recommendations and citations?
- Can we see the evidence behind important findings?
- What software do you use?
- Do you validate unusual findings?
- What happens when you discover a visibility gap?
- What claims are you prepared to make about improving visibility?
Our guide to hiring an AI visibility agency or consultant provides a practical checklist for businesses taking this route.
The important distinction is that when you outsource the work, you are not really paying someone simply to have access to monitoring software.
You should be paying for the judgement and interpretation surrounding the data.
Which AI visibility software have I tested?
This section will grow as I spend enough time with different platforms to form a useful view.
I do not want to publish reviews simply because a tool exists or because it offers an affiliate programme.
The aim is to test software sufficiently to understand what it does, who it appears particularly suitable for, what it automates well and where human interpretation is still required.
OtterlyAI
The first dedicated AI visibility monitoring platform I tested in detail was OtterlyAI.
I used its seven-day trial, examined the suggested prompts, looked at the reporting and explored how the software could fit with the manual testing process already being documented on this site.
You can read the full OtterlyAI review here.
My broader conclusion was not that software replaces the manual thinking process.
It was that the two fit together:
Explore manually → identify meaningful customer situations → build a prompt set → automate the repetitive monitoring → investigate what changes.
As I properly test further platforms, I will add them to this section rather than trying to produce a large comparison based mainly on marketing pages.
Tools can also help with deciding what to monitor
AI visibility software is only part of the tools picture.
There are also useful resources that can help businesses explore which prompts and customer situations may be worth testing.
Again, I would treat these as research aids rather than sources of a definitive prompt list.
Query fan-out
One useful concept is query fan-out — exploring the related questions and subtopics that may sit around a broader customer query.
Our article on what query fan-out means for AI visibility testing looks at how this can help identify areas of a customer journey that may otherwise be overlooked.
The important point is not that every suggested query should be tracked.
The suggestions provide areas to investigate.
You still need to decide whether they are genuinely relevant to your customers and commercially meaningful to the business.
Building a larger prompt portfolio
During my OtterlyAI trial, I also looked at its prompt-selection framework.
That led to a broader article on how to choose better prompts for AI visibility testing.
One of the most useful lessons is that prompt selection is not necessarily a one-and-done exercise.
Some searches that initially appear promising may become less useful after testing.
Others may reveal new customer situations worth investigating.
The prompt portfolio should be capable of evolving as the monitoring itself provides more information.
What can real-world use of the software teach us?
Vendor case studies can also be useful — provided they are treated appropriately.
I looked at an OtterlyAI case study involving NOLA Marketing and SPOC to understand how an agency had approached a real AI visibility project.
The interesting part was not simply whether a visibility metric increased.
It was the process around it.
The work considered the customer’s buying journey, built a prompt set around relevant situations and used the monitoring results to identify areas where the client was absent.
You can read our analysis in What an OtterlyAI Case Study Tells Us About AI Visibility Testing That Actually Matters.
I have not independently reproduced the results in that case study, so I do not treat it as proof that following the same process guarantees the same outcome.
But it provides a useful example of how monitoring software can fit into a wider research and content process.
How future AI visibility software reviews will work
The three buyer guides in this section are intended to do more than provide general advice.
They also give us criteria against which future products can be considered.
That is deliberate.
Rather than testing a product, seeing which features it has and then deciding those must be the features that matter, we can establish the questions before deciding which product performs best.
For a small business, we can consider areas such as:
- ease of use;
- useful prompt capacity;
- AI-platform coverage;
- access to underlying responses;
- competitor monitoring;
- reporting;
- pricing;
- whether the software provides enough value to justify automating the process.
For an agency, additional questions become important:
- multi-client management;
- reporting;
- client access;
- white-labelling;
- team permissions;
- integrations;
- scalability;
- cost per client.
And when a business wants to outsource everything, the emphasis shifts away from the software and towards the quality of the service and interpretation.
This means I do not expect every future review to produce the same “best” platform.
A product may be especially suitable for one type of user and less suitable for another.
Sometimes the recommendation may be: don’t buy software yet
This is an important part of how I want this section to develop.
AI visibility software has a genuine purpose.
Repeatedly running large numbers of prompts across several AI platforms manually can become impractical surprisingly quickly.
But that does not mean every business should immediately subscribe to a monitoring service.
A business may be better starting with five or ten carefully chosen prompts, testing them manually and learning from the responses.
That initial work can help answer:
- Which prompts actually matter?
- Which competitors appear?
- How variable are the answers?
- What information is worth recording?
- Which parts of the customer journey deserve continued monitoring?
Once those questions begin producing a larger monitoring workload, automation has a much clearer value.
That is the part of the market this section is particularly interested in:
the point where useful manual testing begins to become useful automated monitoring.
A practical route through this section
If you are unsure where to begin, this is the route I would suggest.
New to AI visibility testing?
Start with the AI Visibility Testing hub and understand the manual process first.
Unsure which prompts to monitor?
Explore the Customer Journeys section.
Trying to understand the software and services market?
Read AI Visibility Tools & Services: Who Are They Actually For?.
Running your own business and considering software?
Read AI Visibility Software for Small Businesses: What Should You Look For?.
Running an agency or consultancy?
Read AI Visibility Software for Agencies: What Features Actually Matter?.
Want someone else to manage it?
Read Hiring an AI Visibility Agency or Consultant: What Should You Look For?.
Ready to look at a platform I have actually tested?
Read my OtterlyAI review.
The relationship between manual testing and software
The purpose of this section is not to argue that manual testing is better than software.
Nor is it to argue that every business needs automated monitoring.
They solve different stages of the same problem.
Manual testing is particularly valuable for:
exploration, understanding and investigation.
Software becomes particularly valuable for:
repetition, scale and ongoing measurement.
And human judgement remains necessary for deciding what the resulting information actually means.
That leads to the principle I intend to keep using as this section develops:
Human judgement decides what matters. Software makes repeated measurement practical.
As new platforms emerge and existing products change, I will continue adding reviews and practical experiments where there is something useful to learn.
But the starting point will remain the same.
Understand what you are trying to measure first. Then decide whether software — or someone operating that software on your behalf — is the right way to measure it.