The world of experimentation is a numbers-obsessed one. Everywhere you look, you're surrounded by programs, case studies, and advice on conducting "more" tests.
But it seems this experimentation advice doesn't align with better business outcomes.
Here's what we found: When programs move beyond 30 tests per developer per year, the expected impact drops by a whopping 87%.
Volume at the cost of quality can harm performance and the expected impact of your experiments.
That is why you need to carefully plan how to use your developer resources. Maybe you’re not eyeing 30 tests per developer just yet, but consider this:
The highest expected impact occurs at 1-10 annual tests per engineer.
