AI Category Answer Test Protocol
Test how search-enabled AI answers a specific product-category decision, checking suitability, factual conditions and cited evidence under a fixed protocol.
Read this resource as Markdown
When to use this template
Use this when the team needs to understand whether AI answers help the intended reader choose an appropriate solution. Predefine the task and factual criteria before observing engines or brands.
Choose another approach when: You plan to keep prompting until your brand wins, infer universal visibility from one answer, or count an inaccessible or gated session as a negative answer.
Gather these inputs
- A realistic category question and the reader’s actual task or constraints.
- Verified public product and alternative facts needed to assess the answer.
- A fixed observation protocol with engine, mode, locale and account context recorded.
Copy and adapt the template
Edit the text below and copy it into your own document. Edits stay in this browser field and are not submitted or saved by this page.
Worked example
Illustrative example, not a Crany customer result. Replace every example detail with facts you can verify.
Fictional example: a founder asks a search-enabled assistant about software for coordinating a small client handoff. The protocol requires checking current availability and whether the suggested tools address the actual task. An answer describes a waitlist-only product as generally available. The reviewer records that specific factual error against the public access page rather than celebrating the brand mention. Another answer suggests a suitable manual approach and explains its limits; it may be useful even if the founder’s product is absent. If a service requests account verification before returning an answer, the row is marked gated. The reviewer does not invent a citation result, bypass the gate or change the question repeatedly to obtain a favorable screenshot.
How to use it
- Define the category task and factual suitability checks before running the question.
- Record actual engine context, answers and supporting links without selecting only favorable attempts.
- Evaluate correctness and fit, preserving gates, uncertainty and each repeat observation.
Check before using it
- A mention or citation is not assumed to be a correct recommendation.
- An authentication gate is not recorded as proof of brand absence.
- The prompt is not altered after seeing results while calling the observation comparable.
What to measure
Report observed answer suitability, factual errors and claim-supporting citations within the recorded sample. Keep failed or gated attempts visible and distinguish them from obtained answers. This protocol does not estimate global citation share or prove that an SEO edit caused a recommendation.
Research and sources
Google describes variation in AI answers and supporting links and the role of ordinary search eligibility. This original category-quality protocol is not an official platform benchmark and does not generalize one engine’s behavior to all AI products.
- Google: AI features and your website — checked 2026-10-24. Search eligibility, visible content, AI feature reporting and the absence of special AI markup requirements.
About Crany
Crany is an AI marketing co-founder in development for software founders and vibe coders, designed to strategize, execute approved marketing and improve based on results. This free resource is usable independently of the product. Join the public waitlist.