“These guys making all this noise are in no way stakeholders,” he said. “They’re part of the self-identified disciples of this context-driven school of testing. One of their central principles is there are no best practices. It’s almost like they’re born to hate [ISO] 29119 no matter what it says.
“Compare that against over 300,000 ISTQB-certified testers around the world. It’s a drop in the bucket. These context-driven guys, maybe they account for 1% of the community at large. The true believers in standards that’ll go out and follow 29119 to the letter, that’s another 1%. Then there’s the 98% of the rest of us, and almost everybody looked at this whole thing as silly.”
An uphill battle for a standard
A software-testing standard to get the entire sprawling community on the same page does have inherent value. Yet with the craft as contentious and divided as it’s ever been, with Continuous Deployment pressuring testers to automate and the ripple effects of the published standard still unfolding, a true consensus looks like a pipe dream.
“Most people don’t take notice of these standards,” said Gartner’s Murphy. “I have only talked with a few companies in 13 years of calls that have asked about adherence to any previous [quality and testing] standards. Bottom line: I am less worried about the standard for how you test; I am more interested in results. Things like defect containment and a focus on continuous quality, not on testing as a discrete process. Thus you are going to see a lot of pushback from the world of continuous and organizations that don’t think of testing and or quality in isolation.”
Even Reid knows the testing community won’t change quickly, and admits the standard may not gain adoption at all.
“There must be millions of testers across the globe, and in my lifetime I see little chance of the majority of this huge group ‘embracing’ and complying with the standards,” he said. “My vision would be for more testers to use the standards to either improve their testing or give themselves confidence that their testing is already good. I believe these standards will provide a benchmark that supports future innovation and advancement of testing practices.”
Bach believes many testers working for large companies don’t feel free to express their opinions, and that Reid and the standard represent the interests of large companies that may not involve testing more efficiently as it could negatively affect billable hours.
“Stuart once told me, to my face, that he didn’t believe that I really thought differently about testing than he did, but rather that I affected that position in order to market myself as a maverick,” Bach said. “[WG26] needs to honor the differences among us and see if there is even the possibility of bridging what divides us.”
Bach’s view of the testing world is that of “a period of warring paradigms,” where testers can’t even agree on what the word “testing” means or what its scope is. As for the standard, he predicts it would have a chilling effect on young testers and the future of the craft.
“Testing needs to get better,” Bach said. “The standardizers wish to sell our field into bondage to 40-year-old ideas at a time when we need to do our best work. This standard is a formula for determined ignorance.”
RBCS’s Black agreed, adding that while past standards have been a source for useful ideas, very few organizations have made a strict effort to comply.
“I sit with somewhat [amused] watching this enormous conflagration because there’s been an IEEE 829 test standard for documentation since the early 1990s,” he said. “There also was this IEEE 1044 standard for bug management out in the late 1980s and early 1990s. These things have been around forever and have had very limited impact and the 29119 standard was always doomed for the same impact, especially since most testers won’t even see it because it’s behind a paywall.”
Ultimately, Black sees the ISST and context-driven testing camp as a noisy minority who’ve managed to make a big splash on social media. He believes the context-driven school uses categorization to create a dualistic view of the testing world: it’s either this or that. The standard, he thinks, is simply their latest tool with which to polarize.