It has been 20 years since the Manifesto for Agile Software Development was published, and even longer since the idea was first formed, and yet there still isn’t a clear understanding in the industry of what Agile really is.
“Far too many teams that claim to be ‘Agile’ are not. I’ve had people — with a straight face — tell me they are ‘Agile’ because they do a few Scrum practices and use a ticketing tool. There is an awful lot of misunderstanding about Agile,” said Andy Hunt, one of the authors of the manifesto and co-author of the book “The Pragmatic Programmer.”
According to Dave Thomas, co-author of “The Pragmatic Programmer” and the Agile Manifesto, just the way Agile is used in conversations today is wrong. He explained Agile is an adjective, not a noun, and while the difference may be picky, it’s also really profound. “The whole essence of the manifesto is that everything changes, and change is inevitable. And yet, once you start talking about ‘Agile’ as a thing, then you’ve frozen it,” said Thomas.
However, Alistair Cockburn, a computer scientist and another co-author of the manifesto, believes that Agile being misunderstood is actually a good thing. “If you have a good idea, it will either get ignored or misinterpreted, misused, misrepresented, and misappropriated…The fact that people have misused the word Agile for me is a sign of success. It’s normal human behavior.”
What is Agile?
One thing that is missing from the Agile Manifesto is an actual definition of Agile. In one of Hunt’s books, “Practices of an Agile Developer,” he defined Agile development as an approach that “uses feedback to make constant adjustment in a highly collaborative environment.”
“I think that’s pretty much spot on. It’s all about feedback and continuous adjustments. It’s not about standup meetings, or tickets or Kanban boards, or story points,” said Hunt.
But Thomas believes there is a good reason a definition wasn’t included in the manifesto, and that’s because Agile is contextual. “Agile has to be personal to a particular team, in a particular environment, probably on a particular project because different projects will have different ways of working,” he noted. “You cannot go and buy a pound of Agile somewhere. It doesn’t exist, and neither can a team go and buy a two-day training course on how to be Agile.”
Thomas does note he doesn’t mind Hunt’s definition of Agile because you have to work at it. “None of this can be received knowledge. None of it can be defined because it’s all contextual. The way of expressing the values that we had was so powerful because it allowed it to be contextual,” he said.
Dave West, CEO and product owner at Scrum.org, believes the real reason people don’t understand Agile is because of social systems, not the practice, the actual work or even the problems they are looking to solve. “Over and over again, we see this sort of pattern that agility is undermined not by the work, not even by the skills of the practitioners, but by the social systems and the context that those practitioners are working in…Bosses want to know when something is going to be done, but when you ask them what it is they want you to deliver, they can’t tell you that…but they want to know when it is going to be done,” he explained.
If we really want to take the opportunity that Agile presented, we need to change the system agility runs within, according to West. For instance, he said while Fidelity was one of the first companies to ever do Scrum, they are still wrestling with the ideas around it today because they didn’t necessarily change the way they incentivize people.
It’s about the core principles
To get back to the true meaning of Agile, we need to get away from the terms and get back to the four core principles, according to Danny Presten, founder of the Agile.ai community.. Delivering incremental value, having a good look at the work, being able to prioritize and improve cycles is “what really makes Agile hum. It’s not the terms. The more people get focused on the principles and the less they are focused on the terms, the better Agile will be,” said Presten.
A great starting point for teams that have only experienced waterfall or haven’t had as much success with software delivery is to start with Scrum, according to Hunt, but it should only be used as a starting point. “Modern Agile thought goes much further than Scrum, into true continuous development and delivery, committing directly to main many times a day… the goal has always been to shorten the feedback loops, to be able to change direction quickly, to leverage the team’s learning,” Hunt continued.
Presten compared learning to be Agile with learning to play an instrument. “As you start out, you read the sheet music. It helps make momentum happen for you and gives clarity, but if it stops there and all we do is mindlessly read the sheet music and go through the motions, then there’s a problem,” he said.
A good way to look at it is to look at how much feedback you are getting and when you are getting it, said Thomas. “The only way to be Agile is to be constantly adapting to your environment. Sometimes that can be minute by minute, sometimes it’s day by day and sometimes it’s week by week, but if you’re not getting feedback as often as you can, then you are not doing Agile,” he said.
Cockburn explained there have been three waves of Agile. The first was at the team scale, then Agile started to move to the organization scale, and now we are in the third wave which is at a global scale. The global scale includes finance departments, HR departments, legal departments, entire supply chains, governments, social projects, distributed teams and even different geographies. “It’s not just teams. In fact, it’s not merely organizations. It’s not merely software. It’s not really products. It’s global adoption,” said Cockburn.
Cockburn went on to explain that the reason Agile is being looked at on a global scale is because of VUCA: volatility, uncertainty, complexity, and ambiguity. He said the world is “VUCA” and that became even more evident with COVID, the lockdowns and the distributed ways of working for every person, team, industry, company and event country. Everyone needs to have the ability to move and change direction quickly with ease, he said.
“This is the new and current world. It is happening. Agile long ago stopped being only about software; it is now completely general. One can look at those values and principles and extrapolate them to any endeavor,” said Cockburn.
Looking back at the manifesto
The Agile Manifesto was created to uncover these better ways of working, developing and delivering software. It includes four core values and 12 principles.
From Feb.11-13, 2001, 17 thought leaders met at the Snowbird ski lodge in Utah to try to find some common ground on software development. That common ground became known as the Manifesto for Agile Software Development. At the time, those 17 software developers had no idea what was to come from the industry or how Agile would even play out over the next 20 years.
“The Agile Manifesto fundamentally changed or incrementally changed how people approached work and focused on the customer,” said Dave West, CEO and product owner at Scrum.org. “Twenty-five to thirty years ago, I worked for an insurance company in the city of London and we didn’t care about the insurance. We didn’t care about the customer. We just wrote code on the specs. The fact that today we have customer collaboration, the fact we now respond to change, all those behaviors have resulted in a lot of fabulous software.”
The world, however, is very different from when the Agile Manifesto was written that it has some wondering if it still is relevant in today’s modern, digital world.
According to Robert Martin, one of the authors of the Agile Manifesto and author of “Clean Agile: Back to Basics,” the manifesto itself is just a marker in time. “It does not need any augmentation because it is not a living, evolving document. It is just something that was said 20 years ago. The truth of what was said in the document remains true today,” he said.
Fellow manifesto co-author Dave Thomas believes that the manifesto actually applies even more today as software is moving faster than ever, people are adapting to remote work, getting feedback and adjusting as they go. “It’s becoming clear you can’t plan a year out anymore. You are lucky if you can plan a month out, and so you are constantly going to be juggling and constantly going to be reprioritizing. The only way to do that is if you have the feedback in place already to tell you what the impact is going to be of this decision versus that decision,” said Thomas.
If they could go back…
If Thomas had a chance to go back in time and change anything about the manifesto, he said he would remove the 12 principles and just leave the four values in it because they dilute the manifesto and give an idea that there is a certain way to do Agile. “I would make the manifesto just that one page and then possibly just because it may not be obvious to people, explain why it doesn’t tell you what to do,” he said.
Peter Morlion, a programmer dedicated to helping companies and individuals improve the quality of their code, believes the 12 Agile principles are still relevant today. “That’s because they’re based on economic reality and human nature, two things that don’t really change that much. If anything, some principles have become more radical than they were intended to be. For example, we should deploy weekly or even daily and we can now automate more than we imagined in 2001. On the other hand, some principles have been given a different meaning than we imagined in 2001: individuals no longer need to be in the same room for effective communication for example,” he recently wrote in a blog post.
Because of Agile, we have been able to adapt to those principles, and while we can’t be face to face in the wake of the pandemic, we can do video calls because of the software that was influenced by the idea of Agile, Agile.ai’s Presten explained.
If Presten were present at the Snowbird meeting back in 2001, he said he would probably give a hat tip to what outcomes can be expected from Agile, so that those principles can be mapped back to those outcomes to help people understand the what and why of Agile. “I am finding a lot more success and getting value from Agile by setting organizational goals like ‘hey, we want to get better at predictability,’ and then taking steps to get better,” he said
Scrum.org’s West, who was not one of the original authors of the manifesto, believes one thing the manifesto was very quiet on was how you measure success and feedback to inspect it, adapt it and improve it. There are a number of new initiatives coming out to provide organizations with better outcomes such as value stream management and BizOps. According to West, one thing these approaches and Agile all have in common is inspection and adaptation, and the idea of rapid feedback loops and observation. He thinks any of these approaches will help. If you are a software engineer, the Agile Manifesto may be better to look at. If you are on the business side of things, the BizOps Manifesto might be a better start, but ultimately he said to begin with the customer, the problem and the outcome you seek.
Looking back at the manifesto, co-author Hunt said if he had a chance he would add a preface to it that explains Agile is not Scrum. “Scrum is a lightweight project management framework. Agile is a set of ideals that a method should support. They are not the same, and you could argue that Scrum is not even all that Agile; it’s more like a mini-waterfall. Twenty years ago maybe we could wait weeks for feedback. Today, typically, we cannot,” he said.
Thomas would also add something about respecting individuals over respecting the rules in order to reflect that it is not the organization’s job to tell individuals how to behave; it’s their job. In retrospect, he also would have liked to have had a more diverse group of people involved in the manifesto. Cockburn, though, noted that if anything inside that room 20 years ago had been different, if anyone else would have been added, the outcome would have been completely different and it probably would have been more difficult to come to an agreement.
What Cockburn would change about the manifesto is the wording of responding to change over following a plan. “The discussion we had was that the act of planning is useful. [When] the plan goes out of date, you have to respond to change. People, especially programmers, use it to mean I don’t have to make a plan. I don’t have to have a date. And that’s just flat incorrect. There’s no way to run a company if you don’t have dates, prices and budgets,” he said.
Agile.ai’s Presten added: “I’m just so grateful for the founders, the folks in Snowbird and what they created. It really made the world a better place…It’s changing the world that we live in for the good, and then also the culture that it is creating at the companies we work at where decisions are getting decentralized. People are able to come in and grow and learn and fail fast to succeed, and having that safety net there has been a really cool thing, so I’m just super grateful for the founders and the work they did, kind of putting their neck on the line. I think we’ve all benefited from that.”
Technical Agile
While it is normal for ideas to get diluted over time, Robert Martin, one of the authors of the Agile Manifesto and author of “Clean Agile: Back to Basics,” believes that the meaning of Agile has become more than just diluted; it has lost its way.
He explained that Agille was originally developed by programmers for programmers, but after a couple of years there was a shift to bring Agile to project management.
“The influx of project managers into the Agile movement changed its emphasis rather dramatically. Instead of Agile being a small idea about getting small teams to do relatively small projects, it turned into a way to manage projects in some bold new way that people could not articulate,” Martin said.
Martin explained that the original goal at the Snowbird meeting, where the Agile Manifesto originated, was to bridge a divide between business and technology, but the business side took over the Agile movement and disenfranchised the technical side.
He said at one point the Agile Alliance tried to throw a technical Agile conference in addition to its annual Agile conference, which reinforced the idea that Agile fell off course. It was held twice — in 2016 and 2017 — and then discontinued.
“What we see today now is Agile is very popular on the project management side and not very popular on the technical programming side,” said Martin. “There are remnants of technical Agile such as Test-Driven Development and refactoring, but that’s prevalent in the technical community and not the Agile community.”
“Does the Agile Manifesto help the project management side of things? Yes of course, because about half of Agile was about project management, but the other half — the technical side — that part fled. And so the project management side of Agile is now lacking the technical side and in that sense, it has not been a good evolution from the early days of the manifesto till today. It has been a separation, not a unification. I’m still waiting for that unification,” he added.
He explained without that unification, there will be an increasing number of software catastrophes. “We’ve already seen quite a few and they have become fairly significant. We’ve had the software in cars lose control of the cars, kill dozens of people and injure hundreds of people. There have been a number of interesting lawsuits paid out because of that, just a software glitch has done that. We’ve heard trading companies lose half a billion dollars in 45 minutes because of software glitches. We’ve seen airplanes fall out of the sky, because of software that wasn’t working quite right and this kind of failure of the software industry is going to continue.”
If the business side and technical side of software development cannot be united again, Martin predicts the government will eventually step in and do it for us.
“We cannot have programmers out there without some kind of technological disciplines that govern the way they work, and that’s what Agile was supposed to be. It was supposed to be this kind of governance umbrella over both project management and technology, and that split. Now many [technologists] are free to do what they want without any kind of discipline,” said Martin.
“My hope is that we could beat the government there and that we can get these two back together before the government acts and starts legislating and regulating because I don’t trust them to do it well. ” Martin added.
