Make search improvement a repeatable process instead of loose tuning.

1. Observing

Use analytics, zero-result searches and reformulations.

2. Diagnosing

Determine whether the problem is in data, retrieval, ranking, UX or assortment.

3. Hypothesis

Describe which change which behavior should improve.

4. Evaluate

Use query set, judgment and regression tests.

5. Roll out

Use release gates and phased monitoring.

6. Learning

Document results and add new problem queries to the test set.