The concepts of Agile software development are well understood, more than a decade after the original manifesto was put to paper. It calls for things such as “people over process” and “responding to change over following a plan.”

Of course, the devil is in the details, and companies are hitting a wall in trying to implement Agile, especially when they look to infuse Agile beyond their development organizations.

Part of the reason is that Agile is not prescriptive. It provides a broad framework, but there is no map to follow, and organizations are developing their own processes to achieve the Agile goal of delivering software more quickly, of higher quality, and that meets the requirements of the user.

Also, it was originally designed as an approach to developing software. Now, the “Agile enterprise” is the goal. Why should only software development move quickly? Why can’t marketing, and sales, and business decisions, be made in a more Agile fashion.

They can, of course, but again, the devil is in the details. There are challenges in organizational leadership, in allowing teams to find their rhythms and then coming together at critical junctures and in using metrics to guide the business.

“Agile is front and center for everyone trying to make a digital transformation,” said Steve Elliot, CEO of AgileCraft, makers of a scaled Agile management platform. “But running an Agile mindset across groups of teams is the challenge.”

And therein lies perhaps the greatest challenge to achieving business agility. Few organizations are starting from scratch; most have development teams doing Agile projects and creating versions of software faster than release teams, marketing and sales teams, and even business decision-makers can keep up. “The agile methodology has been so successful that development teams are pushing code more quickly than IT can deal with,” said Anders Wallgren, CTO at Electric Cloud, a DevOps release automation software provider.

Some of this is due to the fact that whether most organizations embrace it or not, they are all becoming software companies. Pizza sellers want to be platform players. Automobile manufacturers use more software in their cars than ever before; mechanics are transforming from ‘grease monkeys’ to debuggers.  

And a lot of companies piloted Agile in software projects, and found that it works. But they never had a strategy to scale it throughout the enterprise, said, Sally Elatta, president of Agile Transformations Inc. and founder of Agility Health. “They never realized the organizational impact Agile would have. It was a bottom-up movement.”

Making the digital transformation
“The essence of Agile is adaption and change,” noted Boris Chen, co-founder and vice president of engineering at tCell.io, a company focused on real-time application security for DevOps. “But there are friction points if you can’t get all parts of the organization up to the same speed.”

“It’s almost like when mobile came along,” said AgileCraft’s Elliot. “Companies struggled to adapt. The ability to deal with changes in technology is critical to success.” Today, he said, organizations must conceive of a technology strategy – in this case, with the goal of business agility – and trace it down to deployment. “There has to be one view of the world,” he said.

Smart companies making the change understand that an Agile enterprise is more than having individual teams doing Agile. “That,” said Robert Holler, former CEO of agile project management software provider VersionOne and newly minted chief strategy officer at development tools maker CollabNet, “can create a tribal scenario where each team sets its own practices and tooling. That doesn’t scale very well. Organizations need to take a systems thinking approach, make decisions about improving and optimizing the whole system, not just the pieces. Lots of organizations are still wrapped up in tribal agile, and they’re not getting the full benefit. They’re not working for the benefit of the whole.”

It’s difficult to get teams working at the same speed, and Holler said it’s fine if they don’t. “It’s OK for teams to work at different cadences, but ultimately they have to align,” he said. Holler went on to say that organizations might be doing continuous delivery internally, creating software that is deployable every day but not necessarily deployed. Meanwhile the business is operating on a quarterly or annual cadence. “So you have to come together on monthly, quarterly and annual cycles. When the month rolls around, the organization has to operate on that meta-cadence,” he said, coining a new term.

Because the development group is working at a faster cadence than other teams, challenges lie in understanding when to pull PR and marketing, for example, in to discuss work being done in development. “There are different levels of releases,” said Shannon Mason, vice president of product management for CA Agile Central. “There are minor tweaks, changes. With the big paradigm-shifting things, you can get them out and hide them, then educate the field teams by turning it on for them. You can turn it on in pieces.”

tCells’ Chen suggests integrating marketing into some developer meetings, perhaps weekly, “so they can understand what’s delivered and what is coming.”

In the days of waterfall, teams would create requirements (with marketing and business input) for development, and after the 12-month development life cycle was complete, marketing would have the list of new features to promote. But in today’s world, since there is the notion that software is never done but simply improved iteratively, you need to break work down into increments of value, Mason said. “What’s the value proposition? Who are you targeting? What defines a ‘win’? If the uptake is good, the feedback is positive and the tech is solid, then we determine something is done.”

But the work has to be done at a sustainable pace, offered Ronica Roth, CA agility services advisor and team lead. “There’s a heartbeat and rhythm” to work, she said. “I don’t turn my heart off at the end of the day because it did a good job. But I don’t run 24 hours a day either because I’d fall dead.” Organizations, she said, need cadences and rhythms, just as people mark days, weeks and seasons. “When an organization has a set of rhythms that work together, that’s business agility.”

Leadership is critical
One of the biggest hurdles to scaling Agile across the enterprise and achieving business agility is a lack of business leadership experience in Agile processes. Pizza sellers never had to deal with Agile practices before. “You can teach people how to do Scrum in five seconds. It’s completely logical. But you don’t put a kid in pre-K and say, ‘We’ll see you in 12 years. They need continued instruction and training,” said Mason. “The people are the hard part. Getting people over their own egos and long-held beliefs, especially in leadership, is the real challenge.”

One of the reasons cited for a void in leadership is that training and education are “notoriously underfunded,” said CollabNet’s Holler. “Organizations say ‘Everyone’s doing agile, so let’s do agile.” But Holler said making a digital transformation for business agility is a change management process. “Training, education, centers of excellence are required for positive outcomes. Large organizations have lots of inertia.”

Programs have become more agile, but not the enterprise. Budgeting, for instance, “and the way organizations think about budget, is antiquated,” said AgileCraft’s Elliot.

In short, leadership has to change, as the role of an Agile manager is “changing in a huge way,” said Agile Transformations’ Elatta. “The role is more strategic and less tactical. They must remove organizational obstacles, form and support communities of practice. It’s a big deal.”

Elatta noted that organizations “haven’t invested in these people. They’ve forgotten about the middle management layer.”

Using metrics
For organizations to get to business agility, they need to measure what matters most to them, and to know if their investment in people and process is bring a desired return on investment. Without metrics, “you can’t tell if you’re faster, cheaper, better and healthier” with Agile, Elatta said. “You need continuous measurement for growth. Measure where you are today, gain consensus of where you want to go, then measure again. This cycle is critical.”

CA’s Mason pointed out that organizations collect massive amounts of data from their software that can be used “to save them from creating bad plans. We look at our data and see patterns from a human perspective. Envisioning what a product will look like 12 months from now is like looking for something that isn’t there. Knowing yourself a week from now is easier than knowing yourself a year from now.”

Businesses are looking to extend agility across their entire value stream, said CollabNet’s Holler. Data helps organizations deliver the products their customers want. “When your value stream is optimized, how do you optimize the feedback loop? Faster feedback can totally transform your business. It’s nirvana for Agile.”

Electric Cloud’s Wallgren echoed that sentiment. “Continuous improvement requires data,” he said, providing insights into release frequency, failure rates, user experiences and much more. “We ought to be doing more monitoring before production. If nothing else, you learn what a properly operating system looks like.”