The world of software testing has always been a mishmash of ideologies and methodologies. A diverse worldwide testing community has adjusted and innovated techniques independent of any unified set of standards, sharing philosophies and ideas but not common practices.
ISO 29119 aims to change all that, and in the process has set the testing world on fire.
The software testing community is in the midst of a deep and contentious rift, with ISO 29119 at its heart. The International Organization for Standardization, in conjunction with the International Electrotechnical Commission (IEC) and Institute of Electrical and Electronics Engineers (IEEE) has put forth ISO/IEC/IEEE 29119—ISO 29119 for short—a set of standards governing testing processes, documentation, techniques and keyword-driven testing designed to create uniform testing practices within any SDLC or organization. Parts one, two and three of the standard were published in September 2013, with parts four and five on track for publishing in late 2014 or 2015.
While testers have spoken out against the standard’s creation for years, recent times have seen the debate and controversy escalate to a breaking point with the growing momentum of the Stop 29119 petition. The petition, which eclipsed its goal of 1,000 signatures in less than a month, aims to block the ISO or anyone else from making a legally defensible claim that their “standard” represents a consensus of experts in testing, and ultimately to strike the standard down.
(Related: More about the ins and outs of testing)
The petition has brought this debate to the fore in blog posts, forums, social media and testing communities, all arguing about the standard itself, the testing philosophies perceived to be represented (or not represented) within, and whether testing needs or would even be affected by a standard in the first place.
Behind the petition is an organization called the International Society for Software Testing, formed right before the first parts of ISO 29119 were published. The ISST’s mission was founded to promote the philosophy of context-driven testing and the value of tester skills, and to oppose the over-automation of testing or practices that, according to the ISST website, “are wasteful or seek to dehumanize testing.”
Its mission hits on one of the core drivers behind the movement: testers who’ve always tested as they saw fit using a variety of techniques and methodologies will naturally lash out against a group and viewpoint threatening to fundamentally change how they work.
“The petition to stop [ISO 29119] is just the idea that if this becomes broadly accepted, there are already other organizations like ISST that have published similar standards and terms,” said Gartner analyst Thomas Murphy, an IT expert with decades of experience covering software development and delivery. “I don’t know that the ISO standard would all of a sudden make these invalid, but I think of it as protectionist and [against] the idea of a special group setting standards.”
Anatomy of a testing standard
ISO 29119 didn’t happen overnight. In May 2007, the ISO formed WG26, a working group to develop new standards on software testing. In the past seven years, WG26 has held twice-annual meetings, presented at testing conferences and workshops, and gone through multiple drafts of each part of the standard as the process has run its course.
According to Stuart Reid, convener of WG26 and CTO of the U.K.-based Testing Solutions Group, ISO 29119 builds upon past ISO and IEEE standards that touched on but didn’t directly concern testing. The goal was ideally not to reinvent the wheel, but to build upon and stitch together established practices.
“Up until last year, there was no definitive international set of software testing standards,” Reid said. “There are standards that touch upon software testing, but many of these overlap and contain what appear to be contradictory requirements with conflicts in definitions, processes and procedures. There were large gaps in the standardization of software testing, such as organizational-level testing, test management and non-functional testing, where no useful standards existed at all.