Methodology
How we test what AI says
We ask AI tools the questions real customers ask, save every answer with the date and the tool, and check how often a business is mentioned, linked to and described correctly. This page explains exactly how.
Last updated
Which AI tools do we test?
We test ChatGPT, Gemini, Perplexity and Claude. For every answer we note exactly which version of the tool replied, because answers can change when a tool is updated.
How do we choose the questions?
Questions are written the way customers actually ask them, for example "Which hotel in Gozo is best for families?" rather than a company name on its own. For client work, we agree the question set with the client before testing. For research, each category and place has a fixed question set so results can be compared month to month.
We include three kinds of question:
- Category questions, such as "best accountant for a small company".
- Comparison questions, such as "Business A or Business B for property management".
- Fact questions about a named business, such as prices, location and services.
How do we record answers?
Every answer is saved in full, with the question, the tool and its version, the date and time, and whether the tool searched the internet. Nothing is summarised before it is stored. That record is the evidence behind every finding we report.
Who is the most reliable accountant for a small company?
Several firms are often recommended, including Competitor A and Your Firm, which charges a fixed fee of €90 a month1.
- 1Current published fee is different. Source: an old price list
How do we score answers?
We use five measures. Each one is reported with the answers behind it, so you can check the number yourself.
- Mention rate
- The share of answers that name the business.
- Link rate
- The share of answers that link to the business's own website or profile.
- Position
- Where the business appears when an answer lists several options.
- Accuracy
- Whether facts such as address, prices, services and opening hours match the business’s own published information.
- Share of voice
- How often the business is named compared with the competitors named alongside it.
Possible errors are flagged automatically, then checked by a person before they are reported. We only call something an error when it clearly contradicts the business's own published information.
How often do we test?
Research pages are re-tested every month and show a "last tested" date and a change log. Client question sets are tested at the start of the work and then every month.
Our tools and your client dashboard
We built our own tools to do this work. They ask the AI tools your customers use, save every answer, track which businesses are named and flag what has changed since the last check. They also look for opportunities: questions where competitors are named and you are not, sources that need fixing and content that is missing.
Every client gets their own dashboard. It shows what AI tools say about you now, everything we have changed on your behalf and what we are working on next, so you never have to wait for a report to know where things stand.
What are the limitations?
- AI answers vary between sessions, even with the same question. We ask more than once and report patterns, not single answers.
- Asking a tool directly can give slightly different answers from the everyday apps, which may remember past chats or know your location.
- AI tools are updated without notice. A result is true for the date and version recorded, not forever.
- We cannot see why a tool chose an answer. Where we name a likely source of an error, it is our best assessment, and we say so.
Corrections and right of reply
If your business appears in our research and you think something is wrong, email hello@saidso.agency with the page, the statement and the correct information. We report what AI tools said, in neutral language, and never make our own negative claims about a business. Where a correction is justified, we update the page and note it in the change log.
FAQ
Frequently asked questions
Do you use the apps, like ChatGPT on your phone?
Both, depending on the job. Research and the free AI review ask each tool directly through its official business service, with internet search turned on. Audits can also include checks in the everyday apps. We always note which was used.
Why do results differ from what I see in ChatGPT?
AI answers vary between sessions, accounts, locations and versions. That is why we ask each question more than once and report patterns, with the date and model recorded.
Can a business pay to be included in your research?
No. Research results are never for sale, and clients are not given different treatment in published research.
How do I ask for a correction?
Email hello@saidso.agency with the page, the statement and what is wrong. We check it and update the page, with a note in its change log.
Find out what AI is telling your customers.
Get a free AI visibility review from our team. We check what ChatGPT, Gemini, Perplexity and Claude say about your business, then talk you through what it means.
