March 13, 2020. Friday the 13th. That’s when a large number of companies shut their offices to prevent the spread of a deadly virus – COVID-19. Many thought this would be a short, temporary thing. 

They were wrong.

The remainder of 2020 and 2021 were spent trying to figure out how to get an entire workforce to work remotely, while still being able to collaborate and innovate. Sales of cloud solutions soared. Much of the new software companies invested in required training just to get up to speed.

But training in the form of in-person conferences ceased to exist, and organizers sought to digitalize the live experience to closely resemble those conferences.

Fast forward to 2023. The software and infrastructure organizations have put in place enabled them to continue to work, albeit not necessarily at peak performance. Most companies today have figured out the ‘what’ of remote work, and some have advanced to the ‘how.’

But this move to a digital transformation has provided organizations with tools that can help them work even more efficiently than they could when tethered to an on-premises data center, and are only now just starting to reap the benefits. 

Thus, the editors of SD Times have determined that 2023 will be “The Year of Continuous Improvement.” It will, though, extend beyond 2023.

Bob Walker, technical director at continuous delivery company Octopus Deploy, said, “The way I kind of look at that is that you have a revolution, where everyone’s bought all these new tools and they’re starting to implement everything. Then you have this evolution of, we just adopted this brand new CI tool, or this brand new CD tool, whatever the case may be. And then you have this evolution where you have to learn through it, and everything takes time.”

Development managers, or a team of software engineers, or QA, have to worry about making sure they’re delivering on goals and OKRs, to ensure the software they deliver has value. So, Walker noted, “it’s a balance between ‘what can we do right now’ versus ‘what can we do in a few month’s time’? What do we have right now that is ‘good enough’ to get us through the next couple of weeks or the next couple months, and then start looking at how we can make small changes to these other improvements? It can be a massive time investment.”

Show me the metrics

Continuous improvement begins with an understanding of what’s happening in your product and processes. There are DevOps and workflow metrics that teams can leverage to find weaknesses or hurdles that slow production or are wasteful time sucks, such as waiting on a pull request. 

Mik Kersten, who wrote the book “Project to Product” on optimizing flow, holds the view that continuous improvement needs to be driven by data. “You need to be able to measure, you need to understand how you’re driving business outcomes, or failing to drive business outcomes,” he said. “But it’s not just at the team level, or at the level of the Scrum team, or the Agile team, but the level of the organization.”

Yet, like Agile development and DevOps adoption, there’s no prescription for success. Some organizations do daily Scrum stand-ups but still deliver software in a “waterfall” fashion. Some will adopt automated testing and note that it’s an improvement. So, this begs the question: Isn’t incremental improvement good? Does it have to be an overarching goal?

Chris Gardner, VP and research director at Forrester, said data bears out the need for organization-wide improvement efforts, so that as they adopt things like automated testing, or value stream management, they can begin to move down the road in a more unified way, as opposed to simply being better at testing, or better at security.

“When we ask folks if they’re leveraging DevOps or SRE, or platform methodologies, the numbers are usually pretty high in terms of people saying they’re doing it,” Gardner said. “But then we ask them, the second question is, are you doing it across your organization? Is every application being supported this way? And the answer is inevitably no, it’s not scaled out. So I believe that continuous improvement also means scaling out success, and not just having it in pockets.”

For Gardner, continuous improvement is not just implementing new methodologies, but scaling the ones you have within your organization that are successful, and perhaps scaling down the ones that are not. “Not every approach is going to be a winner,” he said. 

Eat more lean

Agile programming, DevOps and now value stream management are seen as the best-practice approaches to continuous improvement. These are based on lean manufacturing principles that advanced organizations use to eliminate process bottlenecks and repetitive tasks.

Value stream management, particularly, has become a new driver for continuous improvement.

According to Lance Knight, president and COO of VSM platform provider ConnectALL, value stream management is a human endeavor performed with a mindset of being more efficient. “When you think about the Lean principles that are around value stream management, it’s about looking at how to remove non-value-added activities, maybe automate some of your value-added activities and remove costs and overhead inside your value stream.”

Value stream management, he noted, is a driver of continuous improvement. “You’re continually looking at how you’re doing things, you’re continually looking at what can be removed to be more efficient,” he said.

Knight went on to make the point that you can’t simply deploy value stream management and be done. “It’s a human endeavor, people keep looking at it, managing it, facilitating it to remove waste,” he said. So, to have a successful implementation, he advised: “Learn lean, implement, map your value stream, understand systems thinking, consistently look for places to improve, either by changing human processes or by using software to automate, to drive that efficiency and create predictability in your software value stream.”

At software tools provider Atlassian, they’re working to move software teams to mastery by offering coaching. “Coach teams help [IT teams] get feedback about their previous processes and then allow for continuous improvement,” said Suzie Prince, head of product, DevOps, at Atlassian. In Compass, Atlassian’s developer portal that provides a real-time representation of the engineering output, they’ve created CheckOps, which Prince described as akin to a retrospective. “You’re going to look at your components that are in production, and look at the health of them every day. And this will give you insights into what that health looks like and allow you again to continuously improve on keeping them to the certain bar that you expect.”

Another driver of continuous improvement, she said, is the current economic uncertainty. With conditions being as they are, she said, “We know that people will be thinking about waste and efficiency. And so we also will be able to provide insights into things like this continuous flow of work and reducing the waste of where people are waiting for things and the handoffs that are a long time. We want to use automation to reduce that as well. All which I think fits in the same set of continuously improving.”

Key to it all is automation

Automation and continuous improvement are inexorably tied together, heard in many conversations SD Times has had with practitioners of the course of the year. It is essential to freeing up high-level engineers from having to perform repetitive, mundane tasks as well as adding reliability to work processes.

So whether it’s automation for creating and executing test scripts, or for triggering events when a change to a code base is made, or implementing tighter restrictions on data access, automation can make organizations more efficient and their processes more reliable.

When starting to use automation, according to John Laffey, product strategy lead at configuration management company Puppet (now a Perforce company), you should first find the things that interrupt your day. “IT and DevOps staffs tend to be really, really interrupt- driven, when I got out and talk to them,” he said. “I hear anything from 30% to 50% of some people’s time is spent doing things they had no intention of doing when they logged on in the morning. That is the stuff you should automate.” 

By automating repetitive little things that are easy fixes, that’s going to start freeing up time to be more productive and innovative, Laffey said. On the other hand, he said there’s not point in automating things that you’re going to do once a month, “I once had a boss that spent days and days writing a script to automate something we did like once a quarter that took 15 minutes. There’s no return on investment on that. Automate the things that you can do and that others can use.”