Getting an end-to-end perspective with continuous integration and delivery

Velocity. Quality. Traceability. Scaleability.

For companies, it’s these characteristics that form the definition of “DevOps.” But it doesn’t stop there; doing DevOps right also means considering the processes, the culture, and the tools to deliver software and solutions for today’s value-hungry customers.

And it still doesn’t stop there, because DevOps is all about having end-to-end perspective while embracing continuous integration and continuous delivery. As an organization matures, it will most likely include some modern CI and CD practices, but to do this right, but it’s going to take more than just a few automated tasks to make development easier.

To be truly end-to-end, with visibility and constant feedback, the business needs to realize it’s not about one single tool, it’s about a chain of solutions and how to pull them all together into a giant tool that operates as one. This is what will fuel a continuous integration and continuous delivery workflow, with feedback, visibility, and the ability to deliver services to customers quickly.

Accepting DevOps
Regardless of the industry, change is always difficult, whether it’s changing a process, culture, the methodology, or a paradigm. It’s natural, and often expected, to feel a sense of resistance or hesitance.

Getting teams to accept change, especially when it comes to workflow or culture, is part of the overall challenge of DevOps. According to Rod Cope, CTO of Rogue Wave, some teams will embrace it quickly, agreeing that it is the key to efficiency and scalability. Those that resist, however, fear that automation will make them obsolete, and so they inject manual steps along the way to make sure they seem valued in the organization, even if this does slow things down, said Cope.

“It can be challenging to get those people over the hump to say well, there’s always more work that can be done so it’s not like we are going to need less people,” said Cope. “We are just going to move faster.” Instead, these automation tools allow organizations to be more flexible, keep up with rapidly changing technology, and it will let the organization move faster, he added.

The reality is, DevOps is not about the tools. The tools are the enabler of the “holy grail” of what companies are trying to get to, which is delivering more to customers faster with better quality, said Nicole Bryan, vice president of product management at Tasktop. DevOps is trying to broaden the scope of the conversation to include this culture shift, which Bryan believes will take some time. However, she does see the conversation becoming inclusive of more than just the development side.

“Ten years ago people didn’t realize delivering a software product was about more than just the code,” said Bryan. “And I think people do realize that now.”

Don’t be a ‘fool with a tool’
Unlike the days of waterfall development, continuous delivery refers to the process of improving the overall release pipeline, which includes deployment automation, provisioning, testing, and continuous integration. Developer changes on a shared repository are merged daily, sometimes several times a day, giving teams the ability to work fast and stay involved all the time.  

According to Thomas Hooker, vice president of marketing at CollabNet, the very ideas of continuous integration and continuous delivery flow very naturally into DevOps. The people and the processes are working rapidly together, and then the proper tooling supports these interactions, he said.

He said the right tool can have a tremendous impact in the process for not just the developers, but the QA test team, the build team, and everyone else involved. It’s these tools that give teams the visibility, the scalability, and the traceability that is needed to deliver value to the customer.  

“At the end of the day we work to serve the customer,” said Hooker. “The way we best serve the customer is to provide them value, and when we provide them real consumable value, they will buy more from us — not because they have to, but because they want to.”

Simply using a continuous integration tool or an automation tool does not mean you are doing DevOps or doing CI. There is no set of tools that companies can buy that will give them DevOps. Continuous delivery supports the promise of DevOps, according to Tim Buntel, vice president of products at XebiaLabs, but it’s the companies that “must be willing to change their organizational structures, communications and processes to be successful with DevOps,” he said.  

According to John Jeremiah, IT and software marketing leader at HPE, continuous delivery tools are insufficient if the company doesn’t have the discipline to commit frequently, build often, test often, and correct build issues.

“The tools help immensely, but without the right discipline, you can become a fool with a tool,” said Jeremiah.

Companies can easily become a “fool with a tool” if they think that there is a one-size-fits-all solution. That doesn’t exist, and frankly, it never did exist, said Bryan. For each team, there is often a mix of tools automating their continuous integration or their continuous delivery workflow.

RELATED CONTENT: Vendors make a case for their CI/CD tools

Focus on integration
While there is no tool to rule them all, leaders in the continuous delivery and continuous integration tool space say that there is one important aspect that a good tool chain must include, and that is the ability to integrate third-party tools.

That means, according to Stephen Feloney, vice president of products for application delivery at CA Technologies, if you have a continuous integration tool or orchestration tool, it needs to be able to integrate with say, Jenkins, and it should integrate with other third-party tools across the board, he said.

At XebiaLabs, Buntel said that integration is the most important part of a solid continuous delivery tool. Small teams have it easier, but the reality of most organizations is there are several different tech stacks involved in building software — lots of different environments and a mix of skills and languages can easily complicate things.

The best platform will be flexible enough to accommodate all these different user types, these features, these integration points, as well as different parts of the organization, he said.

Cope agrees, and said that integration is key for whatever tool is chosen. The tool should be able to work with the other tools in an environment, whether that’s Puppet, Docker, Jenkins, or other tools in the marketplace, including open-source tools.

“You are going to use a bunch, and I like to say open-source is like potato chips, you can’t have just one,” said Cope. “You’re going to end up with a whole bag [of tools] at one point, so make sure they place nicely with each other.”

And, flexibility is crucial here, said Cope. Every company has different packaging processes for their software, so preserve your choices and be modular. The only way to adapt, he said, is to not be locked into one process or tool chain.  

Scaling for the enterprise
When it comes to enterprises, which are large and often complex in nature, organizations need to find a tool chain that can actually scale. According to Feloney, scripting, coding and maintaining one application is different than scaling for an enterprise of say, 600 applications.

“From an enterprise point of view, you need something that can actually fully scale to that level and not have to be hard coded or scripted for every one of the applications,” said Feloney. “And you need something that follows dependencies.”

The scaling aspect of continuous delivery and continuous integration tools is not to be underestimated, says Tasktop’s Bryan. While there is the fear aspect of how a new tool might impact a job, the bigger concern should be with scaling, she said. Rob Elves, Tasktop’s senior director of product management, said “when you scale up, it’s not just simply volume; it’s scaling out to include inbound and outbound processes that are coming into that DevOps ‘infinity loop.’ ”

Scale is the biggest challenge Tasktop hears from its customers, because it’s a “whole different ball game” trying to do DevOps with 100 people versus 10,000 people. This is where Bryan sees a lot of tools falling short. At a large organization, a tool needs to offer that collaboration, that communication, and have all of the right information flow.

“It boils down to the ability to have that value stream inclusive of full traceability with full streams going back and forth, real-time, all the time, the be able to satisfy that,” she said. “Scale is the short answer.”

The need for speed
Today’s software delivery business initiatives are centered primarily around the idea of speed. The faster the continuous delivery pipeline runs, the faster the team gets feedback, which ultimately means, the consumer is getting their value, their product or service — faster.

According to HPE’s Jeremiah, enterprise IT has been “chronically” unable to deliver at the speed of business. Part of this is due to a legacy of monolithic applications, brittle architecture, and regulatory and compliance requirements, he said. Regardless, software is critical to digital-first businesses, and so the software delivery teams are going to have to embrace this pressure to increase speed, quality and security.

Since things are moving fast through the system, teams need to manage that entire release pipeline, said Hooker. Now that there are so many releases happening each day, a tool chain can help the team understand the realities of this new way of development.

While businesses want a tool to help them keep the continual stream of releases moving, both business leaders and technical people care about different metrics. Hooker said the technical teams want to see technical information on code and commits, but this means nothing to the business owner, who most likely wants to know whether an application is delivering value to the customer, he said.  

“The ability to show what is going on across your continuous delivery or continuous integration toolchain in terms of what is now being called a value stream, is very important,” said Hooker. “We can show how all this work taking place is delivering value to the business and value to the customer, and those are the big things.”

Feloney also hears customers saying they need to go faster, and his question to them is always “why?” Once you drill down, it’s obvious it’s not just about going faster, it’s about the company trying to achieve better business outcomes, he said.

What is inhibiting these companies from going faster? Feloney said it’s a multitude of things, but first and foremost, it’s the culture. Tools can certainly help customers “go faster,” but tools will not fix the problems.

“I can provide customers with tools all day long, but if they can’t fix their culture it’s not going to help,” said Feloney. “Most enterprises are so used to doing waterfall and having these hand offs and all these check boxes, and everyone has a say about things being released, there is all this bureaucracy, and that is what’s inhibiting this faster delivery.”

XebiaLabs’ Buntel said all of their customers’ initiatives aim for that goal — to deliver value to their customers faster. And it’s these high-level business goals centered around digital transformations that are translated down to IT and development, said Rogue Wave’s Cope. Those teams need to be more open and move faster and become more flexible, he said, which gets translated to having continuous integration and continuous delivery streamline development through production.

How does continuous delivery actually make development easier?
Just like DevOps is about more than just tools and processes, continuous integration and continuous delivery is more than just automating code and deploying changes. CI and CD can make development easier, but it’s going to take more than an automation tool to do so.

Getting CI and CD to make development easier means developers are checking code daily, and when the CI/CD pipeline fails, then new work stops, said John Jeremiah, IT and software marketing leader at HPE. Diagnosing and correcting the build becomes the top priority in this scenario, and either the code is fixed or it’s pulled from the repository to correct the build.

“Disruptive? Perhaps, but in the long run, quality improves, velocity improves, as does productivity,” said Jeremiah. “ Consider how security improves when every build is reviewed, scanned and validated from an app security perspective. Letting the build go red and not immediately correcting the issues is the recipe for a mess.”

Having this discipline to correct build issues before introducing new changes is the key to making development easier with CI and CD, said Jeremiah. And of course, automation is a huge benefit of CI and CD. Automating all the individual pieces makes it easier for the developer, and he or she can focus on what they need to develop, without having to worry about the external pieces, said Stephen Feloney, vice president of products for application delivery at CA Technologies.

Also, a good CD platform will remove unnecessary manual tasks and error-prone processes, which means developers can focus on functionality instead of non-coding tasks, according to vice president of products at XebiaLabs, Tim Buntel.

“Things like provisioning, deployment, compliance, approvals, status reports: all of that still happens but developers don’t have to deal with them because they’re automated and standardized,” said Buntel.

A full guide to continuous integration and delivery tools is available here.