Query
Capture the exact input like a customer used, including spelling and capitalization where relevant.
Template
Build a fixed query set for search quality.
A representative test set makes ranking changes reproducible, protects strong queries and gives vendors the same practice test.
Query type
For example, classify brand, SKU, category, attribute, long-tail, typo, natural language or problem query.
Intention
Describe what the user is trying to find or achieve, regardless of the current results.
Expected results
Write down which products, categories or properties are demonstrably relevant.
Priority
Combine volume, buying intent, strategic importance and regression risk.
Baseline
Capture current results, first relevant position and known issues.
Regression status
Highlight critical queries that need to be checked again with each relevant release.
Segment
Capture store, market, device, or other context when the same query needs to be judged differently.
Owner
Make it clear who can adjust judgments, priority and expected outcomes.
Representativeness
A test set should contain both strong and difficult queries.
Include top queries, but also long-tail, typos, zero-result searches, technical codes and known regression risks. Otherwise, you only optimize for the easy part.
Query groupWhyExample type
TopLots of rangeCategory/brand
ExactPrecisionSKU/Model
Long-tailInterpretationDescriptive
ProblemRegressionTypo/no-result