At Agile 2011, we celebrate the 10th anniversary of the Agile Manifesto, a milestone worth celebrating. From my perspective on this occasion, there are three major phases of the Agile movement: Rogue Team, Courageous Executive, and Enlightened Enterprise. What excites me about this anniversary are the accomplishments of the last decade and what we can imagine for the future.
The first major phase of Agile, from about 2001 to 2005, is what I call the “Rouge Team” period. Individual teams received dispensation to try this “Agile” stuff and sometimes several successful Agile projects were completed. But resistance to further implementation was high as organizational antibodies attacked (though there were exceptions in a few software companies).
In this period, teams focused on iterative development (iterations, stories, stand-ups, pair-programming, backlogs, iteration planning, collocating teams), but the core technical practices (automated testing, continuous integration, test-driven development) were often bypassed. Teams tended to focus more on iteration management practices than technical practices.
This first phase of Agile led to mixed results. Many managers were pleased with project performance, better ROI, faster results and improved customer relationships. However, focusing on the iteration management aspects of Agile rather than the technical practices didn’t lead to improved quality. Those organizations that utilized both iteration management and technical practices (particularly those in the Extreme Programming community) had a better record of improving both quality and overall results.
The second, “Courageous Executive” phase of Agile, roughly from 2004 to today (the phases overlap of course), saw us moving from rogue teams to larger initiatives. Larger projects were commissioned, and courageous CIOs and VPs of engineering decided to transform their entire development organizations to Agile methods.
In this process, iteration management was extended into project management, and the emphasis on technical practices improved. Courageous organizations began to emphasize practices such as automated testing, test-driven development and continuous integration.
Some of these transformations worked well, but many dragged out too long for all the reasons large changes often fail to achieve their objectives. Many of these efforts focused on Agile practices—“doing” Agile—and failed to achieve their objectives because they ignored Agile principles—“being” Agile.
During this second phase, there was considerable discussion about Agile project management, whether or not a Scrum master was a project manager, whether project managers were necessary, and other project management topics. Maturing Agile organizations realized that project management was a critical aspect of effective teams, and they should have been focusing on an Agile style of project management.
They realized that good project managers could be effective catalysts in large-scale transformations because they could be a bridge between Agile teams and management in areas such as governance, organization, performance measures and processes. Toward the end of this second phase, the Project Management Institute created an Agile Community of Practice, and in 2011, it began an Agile project management certification program.
Overall, the success rate for Agile transformations was higher than for similar initiatives. I think the reasons for this success were threefold: One, the business benefits were demonstrable; two, Agile practices appealed to engineers (whereas previous methodologies had little appeal); and three, the Agile Manifesto stated a clear purpose and principles for the movement.
The heavyweight waterfall methodologies of the 1980s and 1990s were document-driven and bureaucratic (forms, standards, sign-offs). Their implementation was driven from the top (management) to the bottom (engineers). Agile implementations have, for the most part, either been driven by or supported by engineers.
The third phase, the “Enlightened Enterprise” phase, is just underway. Enterprise agility is at a tipping point, and as with any tipping point, it could go either way. It is being driven top-down from critical business needs and bottom-up from emerging technology and organizational capabilities, such as continuous delivery and devops.
In 2010, IBM interviewed more than 1,500 CEOs and published its findings in “Capitalizing on Complexity.” “Our interviews revealed that CEOs are now confronted with a ‘complexity gap’ that poses a bigger challenge than any factor we’ve measured in eight years of CEO research. Eight in 10 CEOs expect their environment to grow significantly more complex, and fewer than half believe they know how to deal with it successfully,” it said.
Enterprises are being challenged to be more responsive—more Agile—in the face of growing market complexity, uncertainty and volatility. CEOs are asking, “How can we create responsive organizations?” As product development and IT groups develop continuous delivery capabilities, CIOs and VPs of engineering are asking, “How can our organizations take advantage of the capability to deploy software faster and faster?”
A few organizations are driving toward enterprise agility. Some organizations are starting the journey. Some are still strategizing and thinking. The next few years will tell if we were at the tipping point or just fantasizing.
Most movements last a few years and then get subsumed into general practices, or they flame out completely. Object-oriented development swept through the 1990s, but today is just part of the fabric of software engineering. Business process reengineering—that which elevated processes over people—was the rage for a few years in the 1990s, then fizzled out. The Agile movement has grown dramatically over a 10-year period, and it’s not going away.
The real question is how we continue, how we keep Agile fresh and invigorated through continuous innovation. The answer lies in staying on the cutting edge, and not succumbing to the bureaucratization and standardization that afflict many movements.
Jim Highsmith is an executive consultant at ThoughtWorks. A cosigner of the Agile Manifesto, he is the author of “Agile Project Management: Creating Innovative Products,” “Adaptive Software Development: A Collaborative Approach to Managing Complex Systems, “and “Agile Software Development Ecosystems.”