Requirements define the things we want our applications to do, but before they are requirements, Microsoft’s agile “guru” Sam Guckenheimer said, they start as a belief.
“Everything is a hypothesis,” he said. “The hypothesis has to be turned into an experiment, substantiated or diminished with data. Then, you either do more or do something else. What we get is validated learning.”
The goal, said Guckenheimer, is to get the maximum validated learning at the least cost by having your hypothesis validated or diminished in the shortest time possible.
Or, as Tasktop CTO Mik Kersten more succinctly stated, requirements are “what you’re going to build to delight users and how you make sure you’ve got all the right parts.”
Guckenheimer said experiments change requirements thinking from how good the organization is at predicting the future to how many experiments can it run per unit of time that are substantiated by data. He called this an extension of the build-measure-learn principle.
Guckenheimer gave an example of a company with the requirement of raising customer engagement. Under the old way of thinking, the company would come up with features it thinks will achieve the goal, and spec them out for development. Today, he said, the first question to ask is what’s the first meaningful experiment that can be done to raise customer engagement.
“So maybe we run an AB test on the sign-on experience. Does that move the needle?” said Guckenheimer. “Next is an opportunity to run another experiment. Your opportunity to [achieve] the business goal is gated by the number of meaningful experiments you can run in the unit of time you have.”
Why do this? “Our ability to predict the outcome,” he replied, “is not as good as we think.”
