Requirements management is evolving as companies strive to balance the need for sound requirements with accelerating release cycles. If you’re working in a highly regulated environment, generating long, detailed requirements documents is likely necessary. However, if two- to four-week sprints are the norm, then you may have concluded that traditional requirements management is onerous if not downright impractical.

Sound requirements are central to building “the right” software, regardless of its scope or release schedule. It’s well understood that poor requirements definition and management can cause cost overruns. At the same time, it is not practical to spend five months gathering and memorializing requirements when entire release cycles are shorter than that.

Requirements management tools have existed for some time; however, since software development continues to change, the tools must necessarily evolve to suit the needs of the market.

“The traditional requirements management stack is well understood, but it’s not just about management,” said David Ruiz, a vice president at Ravenflow, which makes tools for requirements definition. “There is a question of how to define what to manage, which is why we’re seeing the emergence of the requirements-definition space.”

According to tool providers, requirements may be considered a waste of time or absolutely necessary, depending on speed or compliance. Most companies fall somewhere in between, so the best way to manage requirements may not be obvious.

Jama Software, which sells requirements management software, has even applied a statistical model to hybrid implementations, estimating that 80% of companies fall somewhere in between pure waterfall and pure agile in terms of software development and requirements implementation.

According to John Simpson, director of outreach and marketing at Jama, those 80% describe their process as “agile” and “iterative.” However, unlike the agile purists, those companies need to maintain the kind of structure and control inherent in requirements management best practices.

“The most significant trend is going from document-centric requirements to a more collaborative, shared approach,” said Simpson. “The old way is to start with requirements, but the new way is to work in parallel.”

Of course, the best way to manage requirements in your organization will depend on a number of things, including the industry you work in, your preferred development methodologies, your release schedule and your corporate culture.

Short release cycles affect requirements
Because more organizations are adopting agile practices, the approach to requirements is necessarily evolving to better support the mindset. Regardless of how long solution providers have been in the space, they seem to agree that requirements tools and practices must become nimbler, lighter weight, and more collaborative than they have been in the past to accommodate businesses that need to build high-quality software, faster and more collaboratively.

Managing change is and always has been an issue regardless of how formally or informally requirements have been defined and memorialized. Perhaps a condition has changed or the requirements were described too ambiguously to implement properly. It may be that non-functional requirements do not align with functional requirements, or that customer expectations do not align with the end product.

Because agile development breaks software up into smaller pieces, it is easier to adapt to change and reorganize priorities. Rather than spending two years on a software project that has massive requirements, many companies are working on two- to four-week sprints, which could mean the difference between 2,000 requirements and five, said Bob Johnston, president and CTO of Critical Logic, which makes requirements-based testing software.

“If you’re dealing with comprehensive requirements, the question is how to best implement them. If you’re looking at bits of software, you run the risk of losing sight of the whole picture,” he said.

Quite often, small changes can impact something larger, which some tools reflect by showing the connection between requirements. However, in the case of component reuse, it may not be obvious that the requirements associated with components have changed. As a result, automated requirements validation and verification is necessary, said Johnston.

A more iterative approach to requirements management can be adopted by any type of organization, although the approach and degree of iteration will be influenced by internal and external forces.

“The end goal is the same for all types of organizations: They want to produce good software with fewer defects,” said Maya Agaskar, a customer solution engineer at MKS, another software tools company.

“If you’re in a highly regulated environment, you don’t have a choice when it comes to documentation because you must trace requirements to risks and test cases, or you won’t have a reliable product. That said, software development can be broken up into [several] sprints, and traditional artifact requirements management can be mapped to user stories.”

Matt Terski, president and cofounder of Serlio Software Development, said giant documents are a relic, and the agile mindset has at least rubbed off on less-agile and even non-agile organizations. The company sells software to help developers capture requirements.

Out with the words, in with the pictures
Time to market is an issue for just about every organization, but it plays out differently in companies that lack heavy, formal compliance overhead. In an agile environment, speed is a key driver, so there is an incentive to using fewer words and more visual images, especially if they result in a better collective understanding of a project.

When requirements are ambiguous or sometimes missing, software teams often encounter problems later in the development cycle. Johnston said requirements elicitation tools (which use diagrams, models and functional flow charts) have helped software teams better articulate and understand requirements. Ravenflow’s Ruiz agrees.

“Traditionally, you write down all the requirements you can think of in Word or Excel, but that type of requirements management is too much for agile organizations,” Ruiz said. “[Agile] organizations want a lighter, more business-oriented approach to requirements that provides just enough requirements and discovery, so people understand the context. A number of organizations don’t want an agile free-for-all; they want disciplined agile.”

Not all pictorial representations are created equal, however, meaning that different stakeholders may interpret them differently. Wireframes, UI mockups and prototypes may help if they more effectively illustrate processes and user experiences than functional descriptions alone. The entire idea is to accelerate agreement between business and technical stakeholders as early as possible and keep them informed.

“Agile puts more emphasis on including stakeholders throughout iterations and keeping them involved,” said Bill Shaw, requirements management segment leader at IBM Rational. “Change doesn’t happen in a vacuum, so a more open dialog is healthier for the requirements management process. We must broaden our perspective on stakeholders and include them in the requirements process to make sure the implementation is right.”

Not all specs are created equal
There is also more emphasis being placed on use cases, because the end result and user experience matter. It is entirely possible to conform perfectly to a functional specification and yet miss the mark in terms of performance, scalability or security. A classic case is software that performs its exact function correctly but fails to anticipate the need to handle x number of records in y time, so the application is unusable.

How to alleviate the issue is an interesting question. Some say don’t focus too much on functional specifications because you may lose sight of non-functional specifications. Other say use cases and well-written stories can solve the problem.

“For some customers, it’s all about the use case,” said Jama Software’s Simpson. “We’re storyboard-driven. We don’t have 10-page use cases. We have tight stories developers translate into tasks.”

Ravenflow’s software automatically generates visual representations from natural language because even though people are using visual tools, they may not be getting a complete picture.

Let’s collaborate
The collaborative nature of agile development is also affecting the evolution of requirements tools. Because so many software teams are distributed, collaborative capabilities are necessary to help move projects along in a timely fashion. Increasingly, requirements tools are providing more collaborative capabilities, so all stakeholders understand the priorities, the necessary resources and the end goal.

“Collective genius is very powerful,” said Simpson. “If everyone is accountable, then everyone is thinking about how to produce the best product. Before, people viewed requirements as the beginning of a process, and then they forgot about it until revisions were necessary. Now requirements management is a foundational layer that ties the beginning to the middle and end.”

One application life-cycle management company, Serena Software, uses crowdsourcing to determine what’s best for its customers. According to chief evangelist Kevin Parker, it’s wise to show stakeholders a road map at least internally, and let people vote on work items to clarify what should be prioritized.

“Going public on projects is important,” he said. “All demands on IT need to be posted on a public forum, so business [roles] can decide, out of all the possibilities, which are the top priorities and what resources are required. If you don’t expose information to stakeholders, you can’t collaborate because everyone is driven by their own self-interests.”

Building “the right” software
Because software development cycles are accelerating, businesses must focus on prioritizing work items. There is a major push to build “the right” software; however, requirements do not always align with the intended outcome.

“If you want to know you’re building the right thing, you have to understand what stakeholders are asking for,” said IBM’s Shaw. “You have to keep sight of what you’re going to build and be able to access information to understand requirements. Because software is being broken up into smaller pieces, it’s important to make sure people are building the right part, not just something that’s [intriguing]. To get there, you need to make it easy for stakeholders to see requirements in their own applications so they don’t have to launch and use requirements tools.”

IBM is currently focusing on integration with the goal of making requirements as visible as possible to everyone involved. In fact, Rational is involved with Open Services for Lifecycle Collaboration, a community that is defining ALM interoperability specifications based on particular scenarios. The goal is to develop services, so users can get the information they need without having to launch another application.

For example, if you’re using a modeling application, the relevant requirements would automatically pop up in context.

The ALM angle is an interesting one, because not all ALM tool and platform vendors and providers are placing equal emphasis on requirements. Historically, there has been a lot of emphasis placed on better integration between coding and testing, and more recently there has been a greater focus on release management, deployment and operations. The next logical wave, and one expected by some requirements tool providers, is a heavier emphasis on requirements, since they need to be tied in with the later stages of software development.

“Requirements is the least scientific area of application development and the one in most serious need of focus,” said Serena’s Parker. “The thing I’m hearing is that no one is codifying an absolute focus on accurate requirements before you do anything.”

Most of the people interviewed for this article agree that the hardest part of software development is defining what should be built in the first place, which is why so many roles need to have visibility into requirements. According to Critical Path’s Johnston, only 20% of bugs are caused by programming errors, while the remaining 80% are requirements problems. Because the cost of poorly defined requirements is so high, the need to get requirements right in the first place is getting the attention of CFOs and CTOs.

“If you’re outsourcing development, offshoring or working with third parties, you need accurate requirements, because otherwise the software you end up with won’t be right,” said Johnston.

Serlio’s Terski advocates getting a working product in front of stakeholders as quickly as possible, and then make sure traceability has been baked in along the way so that requirements can be traced to code and testing.

“Building the right software is a tough problem,” said Terski. “Requirements tools wrestle with it, but if the spec isn’t right, you can’t build the right thing.”

Although stories and use cases may be helpful, it’s important to represent other requirements, which means a mix of tools is typically necessary.

“There are lots of requirements, so you need to make sure which tool covers which parts of requirements,” said Terski. “We can develop a functional test case, but how it integrates with performance and security is an issue. So the question becomes how to create a single set of test cases that address all requirements. It’s a challenge.

“Too often testing is based on software vs. requirements. The best test cases are based on business requirements, so traceability is key. Now that we’re able to build high-quality code, we need better quality requirements.”

Fifty years and counting
Johnston said he’s astounded at how bad the industry is at building software. While companies have been trying to improve software development by hiring more testers, they’ve been overlooking the need to improve requirements.

Requirements definition and management need more attention than they’re getting in some organizations, or so say the vendors interviewed for the article. However, as software cycles continue to shrink, tool providers will have to overcome perceptions that requirements tools represent too much overhead when there is no time to waste. Although it is common knowledge that bugs identified early are less costly to fix than bugs caught later, the collective wisdom still stems from coding and testing rather than requirements.

Lisa Morgan is a contributing editor to SD Times.