The long-distance runner rounds the track and starts down the final 200-meter straightaway, but stumbles dramatically. The back of the pack has nothing left to give. The leader lopes ahead effortlessly. Meanwhile, a surprise surge propels an athlete past four frontrunners. She slides across the line in time to win the silver medal.
The Olympics are an apt metaphor for peak performance of software deployment and release. Runners spend years circling the track, becoming ever more efficient at their sport. They work their way up through regional, national and world competitions, practicing delivery under pressure. But only the best make it to the Olympics, and once there, some fail ignominiously while others show that strategy is crucial when it comes to the final heat.
Avoiding dramatic blowouts in the “last mile” of the software life cycle has gained popularity thanks to David Farley and Jez Humble’s book “Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation.” Several new tools focus on elements of the life cycle after development and testing. And in 2012, the focus is increasingly on deployment, as open-source continuous integration tools such as Jenkins gain popularity.
Even the distinction between deployment and delivery is being actively debated, along with the utility of canary releases and feature toggles. These are all signs that the DevOps movement—bringing operational concerns into the early parts of the requirements and development cycle—is gaining mainstream awareness.
But like the weekend warrior whose training plan falls far short of competitive standards, is the average software team even ready to dream of continuous delivery? Has its prerequisite—continuous integration and build management—been met?
What continuous integration isn’t
A recent report by analysis firm Voke on agile that found only 49% of enterprises were using continuous integration. What’s worse, far too many teams are doing it wrong:
• It’s not just running a tool. “When people say they do continuous integration, what they actually do is they’re running a continuous integration tool, and that’s not the same as doing continuous integration. Crucially, continuous integration is a practice, not a tool,” said Humble. And what is that practice? “The essence of it is, you’re always pushing into mainline, into trunk, and working off trunk.”
• It’s not nightly builds. Development teams shouldn’t deceive themselves, Humble said: “A nightly build is not continuous integration. If you are not checking into trunk at least once a day, if you don’t have a suite of tests that runs in a few minutes, if you don’t pay attention when the build goes red and make it the highest priority of the team to fix it now, you’re not doing continuous integration.”
• It’s not hours of tests. Even the most extreme continuous deployment (see Timothy Fitz’s report, “Doing the Impossible 50 Times a Day”) depends on a test suite that takes just nine minutes (albeit distributed across 40 machines). At the other end of the spectrum, too many companies have no comprehensive automated testing in place.
• It’s not sifting through failure reports. “Friends don’t let friends grep log files,” said Anders Wallgren, CTO of Electric Cloud. “If you’re running a two-hour test suite and having to carefully watch the output, you don’t have to. We’re watching those builds in real time and can pinpoint what the problem is for you.”
• It’s not just running build and unit tests on a feature branch. According to Humble, continuous integration only applies to regular merges into the centralized code repository, or trunk, a subject of some controversy among developers fervently embracing distributed version control systems such as Git.
As support for his argument that continuous integration needs not require a special tool, Humble points to a 2006 article by James Shore, “Continuous Integration on a Dollar a Day,” which suggests all you need is an old workstation (to run the builds), a rubber chicken (to take to your desk to alert everyone that you have checked in changes and a build is running), and a desk bell (which you ring if everything passes).
Build management to deployment
It’s more likely, however, that you are using a tool to automate the integration and build processes. Whether it’s JetBrains’ TeamCity for .NET developers, Atlassian’s Bamboo or Apache Maven for Java developers, CruiseControl.rb for Ruby developers, or Jenkins (née Hudson) for any language, there’s something at every price point. While Jenkins is free and open source, there’s ThoughtWorks’ Go in the upper mid-range. UrbanCode’s uBuild and Electric Cloud’s ElectricAccelerator and ElectricCommander are at the high, six- to seven-figure end.
Further, build management and continuous integration tools have become increasingly indistinct: “There is enough overlap between the tooling that it was easy for CI tools to steal build management features, and vice versa,” observed Eric Minick, technical evangelist for UrbanCode. “In 2001, we released the Anthill open-source build management tool, which is long since dead. Within a month, CruiseControl came out as a continuous integration tool. I don’t think we and the CruiseControl guys saw us as being in the same space. But the net result was that the tools learned from each other.”
But there are crucial differences between the two activities. At heart, only developers care about continuous integration. Build management tools go further, creating standalone runtime artifacts that are traceable back to requirements and ready for handoff to QA or deployment. In fact, the extra information created by a build tool adds unnecessary overhead to continuous integration.
Build tools may be task-oriented (Ant, NAnt and MSBuild) or product-oriented (Make), which has repercussions in terms of speed and success. The product-oriented tools generate a series of files and execute them according to their timestamp order. “This is great when you’re compiling C or C++ (for example), because Make will ensure that you only compile those source-code files that have changed since the last time the build was run. This feature, known as an incremental build, can save hours over a clean build on large projects,” wrote Farley and Humble in “Continuous Delivery.”
There are additional details around optimizing builds for managed code and other platforms, but assuming you’ve got the right tool for the job, the slippery slope starts here. After the initial implementation of a check-in to version control launching a build loop and unit tests, it becomes attractive to stress-test the nightly build for performance.