In the past, the CI/CD pipeline was simply a place to integrate code. Developers would write their code in GitHub, pass it through the pipeline, and then deploy it.

However, with the emergence of shift left security and newer automation practices, the pipeline has become a much more critical piece of the software delivery lifecycle.

According to Tim Johnson, senior product marketing manager at the DevOps solution provider CloudBees, there are two different aspects to the changes being seen within the pipeline. “One is the extent or breadth of what it does… and the other is the importance of what it does,” he said.

RELATED CONTENT:
A guide to CI/CD tools
How this company facilitates the tasks that need to be done inside the CI/CD pipeline

He explained that when the end user’s experience with an organization is primarily determined by the quality of software, delivering that is of the utmost importance.

“So the CI/CD pipeline has become that much more important… it has to work, you have to get the software out the door and so the importance of that has grown and the breadth and complexity of what the pipeline is being called upon to do has also grown significantly,” Johnson said.

He went on to say that while ensuring that features are delivering the expected value continues to be crucial, keeping security and regulatory standards in mind has only grown in importance as the pipeline has evolved.

“The delivery of the software through the pipeline also has to be secure and compliant,” said Johnson. “As well as what it is doing beyond just the simple CI aspect of it. So now you get into things like security and testing automation, software composition analysis, static analysis, dynamic analysis, and all these other things that have to be done to get that software through.”

An end-to-end process

According to Gartner research, security in the CI/CD pipeline needs to be an end-to-end process with certain team members responsible for monitoring potential problem areas in order to ensure code compliance.

This leads to the question of whether or not the software has passed these tests. Johnson explained that in order to deliver secure software through the pipeline, an organization now also has to worry about tracking and evidencing standards and exceptions in order to be sure that drift does not happen.

This results in increased complexity within the pipeline as keeping track of who accepts risks and makes changes as well as the reasons behind these choices has become paramount to the delivery of secure software.

“And then you can’t just go out and throw a party like ‘we deployed, yay it’s all over’ right? You have to keep track of what is going on in production. So, that requires an integration of not only tools, but teams and responsibilities,” said Johnson.

He also explained that as an organization works towards progressive delivery and looks at more features, micro components, and micro services, having that view into production is no longer a want, but a need.

Complexity in pipeline grows

According to Johnson, the need to make sure that the final product is performing the way it was intended to grows as the level of complexity within the pipeline does.

“The whole thing has gotten so much more complex, and there’s so many more stakeholders involved, and there’s so many more things that have to happen for this to come to market,” he said. “At the same time, the pressure on the market is constantly going up.”

Johnson also mentioned that there is a rising pressure to deliver to market quickly that has come with this consistent strain that the market is under.

All this to say that the need to innovate quickly in order to keep up combined with the complexities being added into the CI/CD pipeline has caused the software delivery process to change significantly in recent years.

The need for automation

Another change that has been made to the CI/CD pipeline is the need for automation. According to Johnson, automation is the essence of repeatability, predictability, and auditability and in order for automation to work properly, the whole organization has to be on the same page about those principles.

He explained that if there is a disconnect or a lack of proper communication on different organizational processes, automation cannot happen.

“You can automate bits of it and make incremental microcosm improvements and it’ll work a little better, but it’s still not going to be as fast and as responsive as it needs to be,” Johnson said.

He expanded on this saying that any time that there are gaps or missing pieces, more of a burden ends up being placed on the organization’s developers and shared services people to deal with these issues, leading to increased friction and a slowing of velocity.

Additionally, Johnson emphasized that when all of these new elements are done correctly, having them in the pipeline can be an overall positive change.

However, due to the inevitable increase in complexity, the need for every part of the organization to be on the same page has increased tenfold.

As far as the negative components of these additions, Johnson warned that organizations should be prepared for a rise in technical debt.

“Even though you may have your little bit of the world working well, there’s stuff that you haven’t done…and that is compounded by all of the other departments and all of the other stakeholders in the chain and the technical debt that they have yet to deal with,” he said.

On top of that, Johnson said that organizations run the risk of trying to implement these additions too quickly without thinking through how they will function within the context of the rest of the pipeline.

With this, he also mentioned that running a modern CI/CD pipeline requires a fair amount of courage from an organization.

“As problems arise, they need to have the courage to figure out how to deal with those, and not in the classic ‘shoot the messenger’ way. You have to have that culture that we are here to improve things… and it is everybody’s responsibility to pull the chain,” Johnson said.

This courage and bravery comes from different members of different teams not being afraid to mention when they notice an issue. According to Johnson, not making problems known is a much bigger time waster than the alternative.

“Even after you’ve detected the problem, there’s this gap until you fix it… do you have mechanisms in place to turn [the broken feature] off or roll it back, and do you have the bravery to do that?” he said.

“You have to have that bravery, because the consequences are so serious for something like that.”