How Jenkins is building up world of Continuous Integration

Remember CruiseControl? If you do, you were one of the cutting-edge adopters of Continuous Integration and Continuous Deployment. Today, however, you’re just another developer with fond memories of the past. Continuous Integration and Deployment are par for the agile course these days, and the growth and penetration of these principles closely mirror the growth of the most popular software for CI/CD: Jenkins.

It’s been a long road for the Jenkins Project. Like any legitimately good open-source project, Oracle attempted to murder it in its sleep by forking (or, rather, by not forking Hudson) the original project, Hudson.

In 2008, Hudson was a new, young and exciting software project. It won awards at JavaOne 2008, and went on to become the dominant Continuous Integration platform on the market, leaving CruiseControl behind.

Over the next three years, Hudson grew and expanded under the umbrella of Sun Microsystems—and its creator, Kohsuke Kawagushi. Shortly after the Oracle acquisition of Sun, however, things began to get ugly.

(Related: CloudBees releases Jenkins-based platform for CDaaS)

The funny thing about the history of Hudson and Jenkins is that, technologically speaking, there’s never been a real split between the two platforms: Both Oracle and the Hudson teams agreed on a great many technical issues. The sticking point in the end, however, was the trademark on Hudson: Oracle wanted to own it, while the project contributors wanted it to belong to the project.

In November of 2010, things came to a head, and in January 2011, Jenkins was its own project, leaving Hudson behind to rot away and die. Oracle attempted to maintain Hudson by contributing it and the trademark for its name to the Eclipse Foundation, but even that made no difference in popularity.

Today, Jenkins is maintained by more than 600 contributors. The project has its own small convention, its own steering committees, and hundreds of plug-ins. When it comes to CI/CD, there are other solutions, but Jenkins is by far the most popular. It’s the Linux of CI/CD.

Luke Kanies, CEO and creator of Puppet, was asked how many of his customers build their software with Jenkins. “I would guess nearly all of them,” he said.

“Almost everyone has a build system, and Jenkins is winning the race. Enterprises who have done the best job of having a fully automated build pipeline are all built on Jenkins. It’s a critical part of that orchestration piece. And when I talk about orchestration, it can mean 10 different things to 10 different people. Jenkins is a huge part of how people build modern software today, even though it is mature tech that isn’t as sexy as it could be.”

Jenkins as an enabler
When it comes to large enterprises and Continuous Integration, things can look a bit like they did for agile back in the early 2000s. The landscape may seem unaccepting—even unaccommodating—to continuous build and deploy cycles when you’re in a heavily bureaucratic corporation.

But Jenkins enables the kinds of changes needed to implement Continuous Integration. Bringing it into a development process means bringing the entire build, test and deploy cycle into a single, repeatable process. The Jenkins integration server itself also provides a single place to manage all the thousands of assets that can be required for a build process.

Steve Hazel, CTO and cofounder of Sauce Labs, also has kind words for Jenkins. “When we started Sauce Labs, we thought CI was not new in 2008, but it was not at the level of adoption it’s at now. The contenders for CI systems would have been CruiseControl and Hudson at the time. It was really just a mixture of different CI systems. As we were building the business, we didn’t really see a winner. A lot of folks we were talking to were talking about how they needed a CI server. It just enabled this transition a lot of companies went through to change their workflows,” he said.

Sacha Labourey, CEO and founder of CloudBees, has made a living offering enterprise-level service and support for Jenkins. “The market is moving toward DevOps and CD. It’s increasing the number of people we can talk to,” he said.

“What’s also radically changing is that as long as Jenkins remains focused on CI, it’s a machine under the desk that a developer uses. That won’t incur much revenue. As soon as people expand the scope to have a more CI approach, Jenkins expands to DevOps and to QA. They all use the same tool, and it means whatever is in the pipeline it’s going to production through Jenkins.”

And therein lies the major change in Jenkins 2.0: pipelines to production. Labourey said that the Jenkins project is an enabler of more than just CI. He said it’s an enabler for DevOps overall. Jenkins 2.0, he said, even further enables this transition.

“Jenkins has always been great at accommodating complexity and diverse use cases, but that can’t be the only justification,” said Labourey. “After more than a decade of Jenkins 1.x, the community has decided to release Jenkins 2.0. We want to tell the market to wake up: Jenkins is not just about CI. It’s about DevOps as well as CD, and the market needs to know.”

Some of those changes in the 2.0 branch are designed to make Jenkins friendlier and easier to use, said Labourey. “When you start it, it asks you what profile you need; do you want strict CI?” he said of the transition to pipelines in version 2.0.

“It will download the right plug-ins [and] pipelines. For 2.0, workflow pipelines are going to be the main Lego block—instead of jobs—on the platform.

“It’s all about CD and pipelines. That will remain completely backward compatible with 1.x. The UX is starting to be improved,” he added, stating that later this year a radical redesign of the UI will be released.

Those new pipelines, however, can’t come soon enough for Siddharth Jain, vice president of engineering at Animoto. He has worked with Jenkins since the very beginning, even going back to Hudson and CruiseControl. As a result, Animoto has used Jenkins for the five years it’s existed. Until now.

Jain has started to migrate the company’s build and deploy system to Atlassian’s Bamboo. The reasons, he said, are precisely the same as those that brought him to Jenkins in the first place: complexity.

“You can do a lot with Jenkins. You can script your way into it, but that leads to a lot of complexity,” said Jain. “Our existing Jenkins setup is complex, with over 400 jobs, and every job has different parameters. That makes it unwieldy, and makes it hard to manage. We are looking at tools where deployment pipelines are a first-class feature.”

That’s why Jain has decided to move to Bamboo: pipelines. While Jenkins 2.0 is adding them, it was too late for him and his team, especially since he was looking for mature pipeline support.

“We are actually more than halfway migrated off Jenkins at this point,” said Jain. “Bamboo gives you a few utilities to make it easier. With Bamboo, you have a way to import existing Jenkins jobs and configure them as Bamboo plans.

“We wanted this to be an opportunity not just to migrate away from Jenkins, but also to rethink our CI infrastructure. We didn’t want to carry over all the concepts we’d implemented in Jenkins. Thus far, we wanted this to be an opportunity for a fresh start on CI and CD. That’s why this becomes this big migration project. We probably carried over 20% to 25% of what’s existing in Jenkins.”

Still, with five years of almost daily Jenkins use under his belt, Jain has some thoughts on how the platform could improve. “Jenkins, for us, was more used as a job server as opposed to a CI server, where there are promotions of changes, testing of dependencies, and of different versions. Those were very difficult for us. You can do it with Jenkins, but its flexibility comes at a cost.

“I think I would do a couple different things. I think their plug-in ecosystem is so broad, it’s very difficult for someone to have an idea of what is a good plug-in. There might be 20 plug-ins that do the same thing. If I was product manager of Jenkins, I would focus on making it easy for someone who wants to do a really good job of CI. The other aspect of what Jenkins lacks right now is first-class support for deployment pipelines, defining stages, fan-in and fan-out.”

Those pipelines are here now, however, as Jenkins 2.0 should be generally available shortly after you read this. As of this writing, release candidate 1 is in the offering at Jenkins.io.