Exact searches
Test brands, SKUs, model codes and concrete product names.
Query test set →Demo evaluation
Test more than the most beautiful demo query.
Use the same cases with each vendor and assess shopper experience, management, analytics, integration and operations separately.
Natural language
Use descriptive, longer and incomplete questions that reflect real customer language.
AI Search →Errors & synonyms
Test typos, alternative terms, professional language and local writing methods.
Synonyms audit →Filters & mobile
Review refinement, active filters, touch behaviour and result cards on small screens.
Mobile checklist →Management
Show how ranking, synonyms, campaigns, stores, and changes are actually managed.
Governance →Analytics
Check that zero-result searches, CTR, conversion, segmentation and data quality are practically usable.
Analytics →Integration
Ask which feeds, APIs, events, scripts and internal capacity are needed.
Integrations →Operations
Ask for latency, availability, index freshness, monitoring, releases and rollback.
Operations →TCO
After the demo, ask for full costs and not just the package amount.
TCO →Same demo assignment
A comparison only becomes usable if every supplier gets the same difficult cases.
Additionally, let suppliers show their strengths, but keep the formal assessment based on the same test set and criteria.
BlockTestWrite down
QueriesOwn test setResult quality
ManagementOwn workflowSteps/rolls
IntegrationOwn architectureDependencies
CostsEqual periodTCO