Method
How we measure
Three things decide whether a number in this month's report can be compared with last month's: a frozen question panel, three runs per question, and the same way of collecting data on every engine.
The prompt panel
We build a panel of 30–60 questions with you — the questions your buyer actually asks: category questions ("best X supplier in the UK"), problem questions (how the buyer describes the pain before knowing the name of the solution), comparison, segment, price and reputation questions. You approve the panel before the measurement and from that moment it is frozen — we store its cryptographic hash so that next month we can prove the questions were identical. Changing the panel mid-engagement changes the unit of measurement, so we only do it explicitly and with a new baseline.
Three runs per prompt
Model answers are non-deterministic: the same prompt asked twice gives different answers. So we ask every question three times on every engine and count occurrences across all runs, not one. A single call is an anecdote; three runs × three engines is a number you can compare month over month. Every call is stored with the full answer, the list of cited sources, the timestamp and the cost — you get the raw data as an annex to the report and you can verify it independently of us.
Three engines through official APIs
We measure on the OpenAI API (GPT models, with web search when the model invokes it), the Perplexity Sonar API and the Gemini API with grounding, always under the providers' terms and for a fee. We do not scrape search result pages, we do not simulate a browser, we do not circumvent technical protections and we do not break CAPTCHAs. For every answer we also record whether the engine searched the web at all — without that, the citation metric is not comparable between engines.
The three metrics, and what they do not mean
| Metric | Definition | What it does not mean |
|---|---|---|
| Share of Prompts | the share of prompts in which the brand appeared in the answer | it is not a "position"; a brand can appear and be described critically — which is why we also report the context |
| Citation Share | the share of prompts in which the model cited your domain, counted only among prompts where it searched the web | it is not comparable between engines that search with different frequency; where the denominator is too small we report a fraction, not a percentage |
| Qualitative context | how the brand is described: what it is recommended for, who it is not recommended for, which reservations recur | it is not automated sentiment analysis — every flagged sentence is reviewed by a person, because our own detector has a measured precision of about 26% |
The limits of this measurement
We do not measure Google AI Overviews or Google AI Mode, because they have no official API and measuring them would require circumventing Google's technical protections.
We do not do that, and we will not do it on a client's request. We would rather have a smaller scope and real data.
- We measure nothing inside the assistants' web interfaces. Official, paid APIs only.
- We do not send prompts containing third parties' personal names to API providers — not even in the competitor benchmark. In sectors that trade under personal names we name that as a gap in the report.
- Citation Share is sometimes computed on a small denominator. We then report a fraction rather than a percentage that would look stronger than it is.
What those limits look like on real data, counted on our own measurement: what we do not measure and why (in Polish).
Want to see this for your own category?Tell us what you do and who you compare yourself against.