Today’s software release methodology involves multiple aspects of the SDLC: planning, scheduling, and managing a software build through the stages of developing, testing, deploying, and supporting the release. Release automation, as well as methodologies like Agile development, continuous delivery, and DevOps have greatly contributed to the evolution of release management.
It is a core function of operations, however, the more mature organizations are focusing on a DevOps approach, according to Jason Bloomberg, president of analysis and advisory firm Intellyx.
Release management oversees all the stages involved in a software release from development and testing to deployment, and is required any time a new product or even changes to an existing product are requested.
“Release management as a whole has actually been around for many, many years. It’s been around from the old waterfall days and all the way through to Agile, and it’s meant different things to each of these methodologies,” said Mike O’Rourke, the chief research and development officer at DevOps software provider Digital.ai. “The release process used to be kind of like you just did a release and it was on a DVD and you hand the DVD to someone when they bought your product, but now those releases can happen, you know, tens of hundreds of times a day.”
Developers shifted from thinking of just putting out a project at a set time, to getting involved in all of the processes that follow a release, whether that’s testing, updates, support, and more.
In previous methods of release management, everything went through a change advisory board and through a lot of manual checkpoints and guardrails, whereas now it’s much more of an automated capability, and this style of release management has really picked up momentum in the last five to seven years, according to O’Rourke.
The born-on-the-web companies such as Amazon started out the trend by constantly updating their main web-based product, and over time, earlier established companies started realizing that they have to shift their quarterly releases up to daily releases.
Heavily regulated industries took longer to get there as they had to deal with compliance issues in their release management. This includes financial industries, insurance, government, aerospace, defense, and health care, O’Rourke said. Release management has also evolved from being just about deployment. Whereas deployment only refers to moving code into some sort of environment (which could be the test or staging environment), release management goes beyond.
“The key difference here is that the end user experience is important with release, where it’s not something you can particularly focus on with deployment,” Bloomberg said. “So that is where software release management now has to go beyond CI/CD tools and now you have to worry about the operational side, in addition to the software deployment side.”
Effective software release management requires that all of the modules are interoperating at some level so that development teams have a good understanding of what’s really going on in the production environment with the code that they’re working on in the context of all the other code. This has spawned tooling that provides visibility into what’s going on with real-time metrics and logs.
“Tooling is now supporting developers who are able to get visibility into the behavior of their software in production and this is another part of the cultural shift that DevOps presents,” Bloomberg said. “You can’t just throw your code over the wall and now it’s an Ops problem, but rather developers have to be responsible for their code in production in a day-to-day practical way. They have to have the appropriate operational tooling, the user experience, application performance management tooling that is suitable for the role of the developer.
This is different from the typical role of the operations engineer, who is just making sure things are running. Developers have to make sure that releases are properly executed and they have some sort of canary testing or A/B testing that that’s done properly. They’re working with the SREs to ensure that software is properly tested in a production environment, according to Bloomberg.
Today, the part of software release management that deals with testing in production has changed dramatically with the use of feature flags or A/B testing.
“[These capabilities] allow us to say I have this person over here in this demographic and I want to try a couple of things with him, but I don’t want anyone else to see it. And maybe if he likes it, I’ll turn on some other capabilities for all these other people,” O’Rourke said. “If you don’t have that control to be able to pull it back and put the thing that used to work out there very quickly, you’re in trouble. These are the things that can take an app from five stars down to three, just like that.”
As this type of testing in production became more prevalent, companies realized that they need to plan well ahead of time if something goes wrong.
“No matter how well you test your software or how sophisticated your infrastructure-as-code approach is, there are going to be things that you need to deal with in production. So instead of just being caught by surprise by something breaking in production, you plan ahead,” Bloomberg said. “Testing in production is a phrase that’s always sort of struck fear in the hearts of any engineering manager because it’s like wiring your house with the power on. Who would want to do that if there’s an alternative. But, with this kind of software, dynamic software deployed to scale, there’s a lot of things that you just don’t know about the behavior of your software until you put it in a full production environment.”
In addition to testing in production, another major shift is the emergence of GitOps within software release management, which is a set of practices that empowers developers to perform tasks that typically fall in the hands of IT operations.
“You would think from its name that it’s about Git, but GitOps is more about how Ops is taking a Git approach. This involves how a Git-based development life cycle deals with multiple teams working in parallel, deals with bringing code together and bringing into production and deals with representing code in a declarative fashion: the infrastructure-as-code approach,” said Bloomberg.
Automation is integral for remote release management
While many organizations already know the benefits of automating their releases and have these types of tools working for them, the key challenge is to be able to orchestrate all of the information across all of the different tools, according to Digital.ai’s O’Rourke.
“You can’t just say, we only work with one of those tools. If you’re going to bring the organization together and automate them as a whole, you have to really work and orchestrate all those capabilities,” O’Rourke said. “So the ability to manage that process from start to finish is becoming increasingly important.”
In most organizations today, much of the release management process is still not fully automated, according to O’Rourke. However, many of these companies have improved hundreds of percentiles, meaning what might’ve taken six months and now it takes six weeks.
That’s great. But six weeks down to where Amazon is at multiple times a day, you know, you’ve still got a ways to go,” O’Rourke added.
The future of this more sophisticated automation will be primarily driven by AI, according to Intellyx’s Bloomberg.
“There are going to be new levels of automation and it brings new, more intelligent automation that leverages AI approaches so that it doesn’t boil down to human decisions, even on how you go about automating,” Bloomberhg said. “We’ll have our AI create the automations. And the humans now manage the AI, which is essentially managing the data that feeds the AI. So it’s a whole different way of thinking about dealing with automation at scale. And I think we’re just scratching the surface of that.”
The growing need for automated release management was already gaining steam before 2020, but now it has drastically changed the trajectory upward in a short period of time. With the pandemic forcing most software organizations to work from home, companies had to really invest in automating much of the process.
“COVID has made this even a bigger issue. It used to be that with a lot of these manual things I could potentially still do them very quickly. If I’m hanging around the office, I could say, approve this? Will you approve this? Well now I can’t do that,” Digital.ai’s O’Rourke said. “People have found that if they don’t automate, they’re basically losing ground with their competition and so the notion of COVID, I will tell you, it has dramatically accelerated this whole notion of release management and DevOps.”
