Almost a decade has passed since much of the influential early material on agile software development was published. In that time, the agile movement has gone from being perceived as undisciplined, impractical and scary, to being front and center on the agenda of a growing number of mainstream IT executives.

Today, there’s hardly an IT organization on the planet that isn’t using, adopting or thinking about trying an agile approach. There are many things to celebrate about this progression, not least of which is that, as an industry, we are learning a lot about new and better ways to build software.

But where mainstream agile adoption is concerned, we are massively under-delivering on the potential of the Agile movement and, by association, on our potential as software professionals. We are failing to advance good agile engineering practices at anywhere near the pace that we are advancing agile planning practices.

Currently, the typical approach to agile adoption within a large mainstream IT organization starts with introducing some flavor of iterative planning and daily stand-up meetings. Though it’s never straightforward, most business and IT stakeholders will eventually grok the value of these practices, and work at them until they achieve some degree of successful implementation.

In contrast, it’s often much harder for organizations that are new to agile software development to understand the value of engineering practices like pair programming, test-driven development, refactoring and continuous integration. So simply trying (let alone implementing) these practices is often de-prioritized to a later date, which often never arrives.

There are many people in the industry who will argue that it doesn’t matter. They will say that for most organizations, iterative planning is a huge step forward and they should be happy with such an achievement. I do have some sympathy for this view, considering I have seen how hard it is for most organizations to get that far. However, this does not change the simple fact that when we do software development, we end up with software. And the better designed and simpler that software is, the easier it will be to change. If you believe in “responding to change over following a plan,” you will agree that’s pretty important.

Change happens all the time on an agile project. Some changes come from stakeholders in the form of new requirements. Some come from the discovery of new technical platform capabilities or constraints. Many more come from the fact that agile teams in their flow are constantly checking in changes to their working software, often several times a day. That’s a lot of change to be factored.
Daily stand-ups and iteration planning meetings on their own do not make changing software easy. That’s why we have agile software engineering practices. Pair programming and test-driven development help teams think through their design ideas and decisions at the moment they are writing the code. If something is hard to test, it will be hard to change, a good indicator that they need to try another approach.

If what should be a simple refactoring takes many days to complete, it tells the team that they need to simplify the architecture before it’s too late. And running thousands of automated tests with every check-in is a pretty good indicator of whether or not a newly introduced change has broken anything.

Beyond just making software easy to change (which is no small advantage), agile engineering practices can deliver other sometimes less-obvious but very important benefits. Agile teams that have the discipline to apply good engineering practices over a sustained period will develop a growing sense of confidence in themselves and the software they are building. This will positively show up in the increasing accuracy of their estimates and the speed at which they can deliver new features, which will make it possible to develop an accurate plan that is grounded in reality.

Organizations that extend agile engineering practices beyond just development will find they’re tremendously useful in breaking down barriers between the traditional silos of development and IT operations. Extending continuous integration toward continuous deployment encourages defining non-functional and operational requirements early on in the process in a way that they can actually be tested.

Implementing and validating these requirements calls for constant collaboration among those who are developing the software and those who will have to manage it when it goes live. The net result is that when it’s time for release, there are no surprises because all stakeholders, including the traditionally forgotten IT Operations team, have been intimately involved throughout the entire process.

From my experience, enterprises that attempt to adopt a half-agile approach by taking on the planning practices without embracing the engineering practices often lose out. In the best case, their agile adoption programs deliver very little long-term sustainable value, and sometimes they even fail completely. In the worst case, they eliminate their opportunity to achieve true agility.

As agile methods are transitioning into the mainstream, it’s now more important than ever to acknowledge that enterprise software development is a complex and multi-faceted activity; collaboration, planning and engineering all play critical roles. We must resist the temptation to over-simplify agile adoption or take the easy way out by choosing only those practices for which we can see a smooth path to near-term gratification.

A holistic approach, where engineering practices are not optional, may not be the easiest path to agile adoption, but it’s certainly the most realistic, longest lasting and most rewarding.

Cyndi Mitchell is managing director of ThoughtWorks Studios, which sells tools for agile software development teams.