Prompt Set Design for GEO Measurement: How to Choose the Right Test Queries

How many prompts do you need for a valid Citation Share measurement?

Start with 40 prompts when you need a directional Citation Share baseline. That count is large enough to stop one unusually favorable answer from deciding the entire report, while remaining small enough to inspect every prompt, answer, cited source, and classification decision.

Forty prompts do not create a universal statistical guarantee. They create an operational baseline for a defined question universe. A measurement becomes more credible when its prompt set covers the demand a buyer would plausibly express, stays consistent from one reporting period to the next, and exposes the segments that are hidden by the overall percentage.

Use 60 to 80 prompts when the business needs segment-level reporting. A B2B software company may need a separate view of category discovery, competitor evaluation, implementation research, and brand validation. A local service business may need equivalent coverage across service terms, locations, comparison questions, and branded checks. Do not report a thin segment as if it carries the same weight as the whole set.

The relevant standard is repeatability, not a ceremonial threshold. The NIST AI Risk Management Framework describes an iterative cycle of Govern, Map, Measure, and Manage. Prompt measurement benefits from the same discipline: define the question universe, preserve a measurement method, inspect what the score misses, then revise the next version with a documented reason. NIST AI RMF 1.0 does not prescribe prompt counts, but it supports the need for structured measurement rather than ad hoc checks.

Run a smaller exploratory set only when you label it honestly. Ten prompts can help discover whether a brand appears at all. It cannot reliably stand in for a category measurement, especially when those ten prompts all come from one product page, one search-intent pattern, or one team member's wording.

1. Use 40 prompts for an initial directional baseline.
2. Use 60 to 80 prompts before publishing segment-level performance.
3. Add prompts only through a documented revision, not after seeing a disappointing answer.
4. Keep prior prompt versions so historical changes remain explainable.

Which query types should a GEO prompt set include?

A representative prompt set should include four query types because buyers ask AI systems for different kinds of help. Category-level prompts test broad discovery. Comparison prompts test shortlist formation. How-to prompts test problem-led research. Brand-specific prompts test whether the company is present when the buyer already knows its name.

This split prevents a common reporting failure. A company may appear regularly in brand-specific answers because the system can identify an established name, then disappear from category answers where a new buyer asks for options. Combining those outcomes without a taxonomy can make a weak discovery position look healthy.

The taxonomy is an editorial measurement framework, not a claim that every market follows the same buyer journey. Adjust the wording and allocation to the business, but keep all four buckets unless a documented audience constraint makes one irrelevant. The research paper that introduced GEO-bench uses diverse queries across multiple domains and reports that strategy effects varied by domain. That is a reason to preserve category context rather than assume one prompt style is every market.

For each type, write prompts in the language a buyer would use. Category prompts should name a market or job. Comparison prompts should name credible alternatives or a decision criterion. How-to prompts should describe the work. Brand prompts should test the company name alongside questions about fit, proof, implementation, or alternatives. Avoid prompts that contain the conclusion you want the model to reach.

A prompt set is not a keyword list with question marks added. It is a declared sample of decision-relevant questions. Store the intended query type, audience, geography where applicable, inclusion reason, and any named competitor beside every prompt.

1. Category-level: broad discovery prompts such as best payroll software for global teams.
2. Comparison: choice prompts such as Rippling versus Deel for contractor management.
3. How-to: problem-led prompts such as how to manage payroll for contractors in several countries.
4. Brand-specific: validation prompts such as is Deel suitable for contractor payroll.

What sample size should each query type receive?

Allocate the first 40 prompts across the four query types instead of filling the set with whichever prompts are easiest to write. The table gives a practical starting allocation for a single category or focused market. Treat it as a measurement design choice that must be recorded, not as an industry benchmark.

Category and comparison prompts receive the largest allocation because they usually reveal whether a company enters a new buyer's shortlist. How-to prompts deserve enough coverage to show whether educational content is being cited during problem research. Brand prompts need a smaller but protected allocation because they diagnose brand understanding without overwhelming the broader discovery signal.

Change the allocation when the evidence supports it. A company with a narrow technical implementation market may increase how-to prompts. A service company with multiple locations may split category and comparison prompts across priority geographies. A brand with a large competitor set may need more comparison coverage, provided each competitor and decision criterion is selected before the run.

Do not infer quality from an even score across all four types. The score answers a defined measurement question. The allocation states whose questions receive representation. If the business weights a segment more heavily, report both the unweighted result and the segment result, then state the weighting rule in the technical notes.

The minimum sample in a segment should be large enough to expose a pattern, not merely produce a percentage. Eight brand prompts that all repeat the same claim do not is brand validation. Eight prompts spanning fit, alternatives, proof, use cases, and constraints can.

How do you choose representative GEO test queries?

Choose representative queries from evidence of buyer demand, then turn that evidence into a balanced sample. The strongest inputs are customer language from sales calls, support questions, on-site search, search-query data, category pages, comparison pages, and recurring objections. Each source can reveal a different question form, so preserve its provenance in the prompt inventory.

Begin by defining the population. For a B2B SaaS company, the population may be questions a buyer asks while discovering, evaluating, or implementing a category solution. For a multi-location service business, it may be questions asked by people seeking a service in a priority area. The definition should be narrow enough to guide selection and broad enough to avoid one campaign deciding the result.

Next, create a longlist that is larger than the final sample. Classify every candidate by query type, audience, journey stage, geography, industry, and competitor reference. Then select prompts across those fields. This process makes omissions visible before the run, rather than after a score falls or rises.

Write prompts as natural questions and preserve the original intent. A search-quality framework makes the same point from an evaluation perspective: Google's September 2025 Search Quality Rater Guidelines include dedicated guidance on understanding a query and note that query meanings can change over time. A measurement system should therefore review wording periodically without changing prompts casually during a comparison period.

Keep prompts specific enough to test a real decision, but do not pack them with product claims, preferred features, or a desired answer. A question such as best CRM for a small sales team tests category discovery. A question that names your product, preferred feature, budget, and conclusion mainly tests whether the model repeats your framing.

1. Define the buyer population and the decision the prompt is.
2. Build a longlist from named demand sources and retain the source label.
3. Classify each candidate before selecting the final sample.
4. Use a fixed selection rule for every measurement period.
5. Review prompt wording on a scheduled cadence, then version any changes.

How do you avoid sampling bias in AI citation measurement?

Avoid sampling bias by preventing one easy-to-win slice of demand from dominating the prompt set. The usual source of bias is convenience: teams pull questions from their own pages, choose only branded wording, test only a familiar geography, or add prompts after seeing answers they dislike. Each shortcut makes the resulting Citation Share less representative of the stated market.

Set the sampling rules before running the prompts. Decide the taxonomy, allocation, priority audience, geographies, industry variants, named competitors, and exclusion rules. Freeze those decisions for the reporting period. If a prompt must be replaced because it is obsolete or ambiguous, log the old prompt, the replacement, the reason, and the date.

Use blind review where possible. One reviewer should assess whether the answer cited the brand according to a written rule. A second reviewer should spot-check disputed answers and edge cases. That reduces the temptation to count a passing mention as a citation or dismiss a competitor citation that does not fit the desired narrative.

Do not delete difficult prompts to improve the score. Retain them if they remain part of the defined question universe. A prompt may be retired only because the market, product, geography, or user language has materially changed. Version the set at that point and do not compare the revised result directly with the old one without explaining the break.

Bias also enters through engine coverage. A prompt set can remain fixed while answers differ across ChatGPT, Perplexity, Gemini, Copilot, and AI Overviews. Report results by engine before presenting an aggregate. An aggregate can guide a leadership view, but it should never hide a large engine-specific gap.

1. Do not select prompts only from owned content or branded searches.
2. Do not change prompts after viewing the current measurement results.
3. Use a written citation-counting rule and retain disputed cases.
4. Report engine-level results alongside the aggregate.
5. Version the prompt set whenever the defined question universe changes.

What should you record with every GEO prompt set?

Record enough metadata that another analyst can rerun the set and explain every result. The prompt text alone is not sufficient. A durable inventory identifies why the prompt exists, which segment it is, which engine was tested, when it ran, and how a citation decision was made.

At minimum, record a stable prompt ID, prompt text, query type, source of the prompt idea, represented audience, geography or industry modifier, named competitor where applicable, priority level, version, run date, engine, answer URL or captured output reference, cited domains, brand-citation outcome, reviewer, and review notes. Keep the raw answer reference outside public reporting when it contains unnecessary personal or confidential material.

Use the inventory to compute the Citation Share score consistently. The numerator is the number of relevant AI answers in the defined set that cite the brand. The denominator is the number of valid answers in that same set. Document exclusions, such as an unavailable engine response or a malformed answer, before calculating the percentage.

A technical record also makes trend reporting defensible. If citation share changes, the team can inspect whether the shift appeared in category discovery, comparison prompts, a particular engine, or a revised question set. Without this context, a single percentage is easy to circulate and hard to trust.

Build the first inventory before the first run. The next action is to create a 40-prompt longlist, classify it against the four-type taxonomy, select the baseline set using the stated allocation, and freeze version one before measuring any engine.

1. Prompt ID and exact prompt text.
2. Taxonomy, audience, geography, industry, and competitor fields.
3. Prompt source and inclusion reason.
4. Engine, run date, captured-answer reference, and cited domains.
5. Citation rule, reviewer, exclusions, and version history.

Sources

Source: arXiv, KDD 2024 paper

The GEO paper introduces GEO-bench as a large-scale benchmark of diverse queries across multiple domains and reports that strategy effects varied across domains. arXiv, KDD 2024 paper, 2023-11-16

Source: Google

Google's General Guidelines include sections on understanding the query and on query meanings changing over time. Google, 2025-09-11