DevOps is about two big worlds coming together, yadda yadda yadda. Now you’re deploying new code every day, right? Not quite. Just as in Seinfeld, when George’s girlfriend “yadda yadda’d” shoplifting, those are some very big yaddas.
Last month, we covered one of those yaddas when we discussed configuration-management systems, such as Chef, Puppet, Ansible, CFEngine and Salt. Those tools enable developers and systems operators to deploy fresh code on fresh systems in an automated fashion, it’s true. But the usage of those configuration and deployment-management systems covers only one of those three yaddas.
The other two are closely inter-related, and much more managerially driven. One yadda would be agile processes, taken to their logical extremes and including the operations side of the puzzle every step of the way.
The other yadda, as it were, would be unifying the actual medium through which these processes are expressed. While developers might keep their workloads and tasks in JIRA, Rational or Perforce, the operations side of the house might stash its tasks within Zendesk, ServiceNow, Zoho, Spiceworks, or any of a dozen other systems.
And yet, both of these worlds essentially run the same type of software to accomplish the same types of tasks, right? It’s like the episode of “Seinfeld” where Elaine discovers a whole other group of friends that mirrored Jerry and the gang: Bizarro Jerry. Bizarro development team has a coffee shop, too! But do these two parallel gathering places have the same menu? Can they both make a Big Salad?
Despite the fact that both of these virtual coffee shops perform the same basic functions, they’re not exactly the same. And even more importantly, it’s quite difficult to gather everyone up into a single virtual coffee shop. Just as you can’t replace a developer with a sys admin, you can’t replace Superman with Bizarro Superman, nor Kramer with Bizarro Kramer. So, too, can you not replace Perforce with ServiceNow, and vice versa. We’re all much more comfortable in their own universes.
In fact, best practices would advocate that teams build bridges between existing systems, rather than provide one collaborative platform to share.
Kurt Bittner, principal analyst for application development and delivery at Forrester Research, said that “It’s becoming more common to see these tools tied together. You can’t manage the life cycle without a common way to view the work. There isn’t one tool to rule them all, but this heterogeneous tool environment is being spanned by tools like Tasktop, CollabNet and the like.”
Indeed, a mandate for a single collaborative view of both IT and development is an extremely difficult goal, said David Jabs, CTO of AccuRev. “I think this problem’s going to be tough to solve not because of technical reasons, but because of organizational reasons,” he said. “It takes someone above both operations and development to do it, and that’s an organizational challenge.”
How DevOps is growing
And just as Bittner said, many companies offer products that bridge the gaps between social collaboration platforms. Mik Kersten created the Eclipse Mylyn project specifically to tackle this problem, and it resulted in the formation of Tasktop (with Kersten now as its CEO), a company to back and sell this open-source task-based development life-cycle tool.
This need for cross-platform integrations comes from the fact that other areas of the DevOps puzzle have matured, he said. “The key nut to crack [for DevOps] was to be able to deploy continually. I think with DevOps over the last year or so, that piece has matured. Things like going from the assets of the application—from the design docs and source code—to things that are built by your Jenkins, or your CI server. We’ve made tremendous strides on that layer and with DevOps,” he said of existing automated deployment tools.
“The other thing that is interesting to me is that we’ve already made strides in the early stages of DevOps. Continuous integration just works now. But of course there are varying stages of maturity on this front. Some can deploy daily, but at least there is mature automation technology out there from vendors and in products.”
Because the tools are now mature, the people and processes behind them are what require optimization in most organizations, said Kersten. “On that side of DevOps, in small organizations, you have thinking orchestrated by your support desk systems: Zendesk,” he said. “On the other side of Ops, you’ve got things orchestrated by ITIL systems like ServiceNow, which is providing the solution for the Ops people to talk to each other the same way JIRA and Rally and VersionOne have been successful on the development side.”
And this brings up a major pain point for DevOps: Those automated tools and systems can only go as fast as the teams behind them.
Said Kersten, “So the development team is using JIRA, the Ops team using ServiceNow, they’re able to push bits to production in real time, but the teams communicate in a biweekly status update meeting. Does that work?
“I think the first step to getting DevOps working is that organizations have to fix that first step. You have to automate going from source to production and running bits. Once that’s in place, however, the efficiency gain you get from that is only as good as your organization’s ability to learn from what’s in production.”
Laurence Sweeney, vice president of enterprise transformation for CollabNet, agreed. “I was talking with some folks in Europe before Christmas, and they really do have some great collaboration work,” he said. “They’ve got lots of people meeting each other; they’re co-located. But in a fairly small group—300 people—they have three different ways to track issues. Even though, on the surface, they’re collaborating very nicely, they struggle with their schedules, because they struggle with islands. It’s a perennial problem in IT.”
Bittner also sees a multi-vendor environment, and he expects it won’t be homogenized any time soon. “Mainly, everybody’s got these existing work-management solutions, and they want to leverage the value they’re getting out of that,” he said. “I don’t see an extensive move toward consolidation of the vendors. Beyond 2014, the work is so similar between these tools, some gradual consolidation of these tools is going to happen over time.”
The odd thing, Bittner added, is that open source hasn’t been the driving force in these collaboration platforms. “As far as the open-source tool, it’s been interesting because the open-source solutions—Bugzilla [and] other older things—they’ve been superseded by the commercial products. I don’t see open source coming in and supplanting major players like Rally and Atlassian. The other products have met the needs well enough,” he said.
That means development managers can now focus on pushing agile even further into the processes behind their applications. Randall Newell, director of capabilities marketing at IBM, said, “We’re seeing teams moving to agile doing two- or three-week sprints, with the expectation that the developer can see immediate results from their work.”
However, this velocity can be tough for other teams to keep up with if they’re not on the same two- to three-week release cycle. “Consider Cars.com, which has now gone from doing 20 or 30 releases a year, to now doing over 400 releases a year to stay fresh and current. That’s an agile process that goes very quickly from development to test to deploy,” said Newell.
“Contrast that with a much more complex environment like Nationwide Insurance. They’re in a regulatory environment, but they still have moved a huge portion of their delivery process to agile. Developers are doing very rapid builds, releasing the build to test internally, then they go through more rigid release processes. Developers need to be able to very rapidly check in and see the results of their most recent build and push that into automated testing as well.”
Addicted to tools
Perhaps they’ve met those needs too well. Said CollabNet’s Sweeney, “If you’re looking from the bottom up as project managers, we tend to fall in love with our tools. That’s how you end up with three different tracking systems: There’s one optimized for testers, one for developers, and one for Ops. As practitioners, we have to realize that is always going to be causing frictional loss and friction in relationships. We’ve all sat in meetings trying to do a postmortem and said ‘That’s not what my data says.’
“Let people use whatever tools they want. You need to say that it doesn’t matter what tools people use, we’ll make it work, and we’ll make it work with people. The thing we should do as practitioners [is make sure we don’t all have] different data models. The definition of what’s the most important thing to fix next can differ between the two teams… Having confusing data models in standard platforms can be just as bad as having multiple platforms.”
Igor Landes, vice president of engineering at Exadel, agreed. “Sharing the resources between pure development teams and maintenance and support teams is important,” he said. “Essentially, if the product has a longer life cycle and longer releases, some of the people [on each team] do simultaneously support a production-deployed product. They track the use of features, bugs and issues, and feed that back into the product development cycle.”
(Don’t let ‘troublemakers’ disrupt your DevOps plans)
Landes also stated that “tools are important for DevOps, but they’re not the most important aspect. But I do believe that even more important are the processes, and maybe even more important is the mindset. In other words, the deployment requirements should be part of the functional requirements for the application. If you don’t have that as a part of the functional requirements, no collaboration tool or effort will really help you much.”
David Myers, senior product manager for IBM’s DevOps team, said, “We actually are seeing the need for multi-level requirement management. There are some functional requirements, some nonfunctional requirements, some design requirements: All these tie back up to the question: ‘What’s the business reason we’re doing this in the first place?’ What’s the acceptance criteria from DevOps? What’s the feedback we get on that in the first place? It’s this virtuous cycle between when we’re doing something and finding out what are the customers saying about it.”
Indeed, Landes advocated further deep usage of these collaboration tools beyond their traditional roles. “More and more, organizations and enterprises understand that their applications that are being developed should be loggable [meaning, activities generate logs that can be sent to an external server], API-based, and their architecture should be such that you don’t need huge deployments. You have to be able to address specific features, preferably in a pluggable manner. The whole mindset of requirements should go into the architecture.”
Putting the actual system requirements for each application into the design documentation and tracking system is the key to keeping developers focused on their application’s requirements at deployment time, said Landes. “This is a good process that supports all the kind of interaction, and I think it is one of the processes that is being put by many enterprises into place now.”
Still, Landes said that even Exadel keeps development and production-tracking systems separate. “The reason we keep the things separately is to minimize the disturbance of the development team. Usually the production issues take precedence, or at least they do if they’re critical. You don’t want to insert the urgent issue into the development project in JIRA because it will then impact the velocity,” he said.
“We are tracking the burn-down rates, and we want to have a predictable rate of development. We want to have predictable velocity. If we start inserting the issues into the current sprint for development, it will impact that.”
Another aspect of DevOps and tools integration that is growing, according to Tasktop’s Kersten, is the area of performance monitoring. “We’re seeing some of our customers asking us to connect the performance info, connecting in ratings from app stores and such,” he said.
“In the end, the people doing the planning have that input, and they’re not consuming input in the old way with month-long release cycles. They’re doing it as part of this two- or three-week decision process.”
To that end, performance-monitoring systems like New Relic are being integrated into developer and operations dashboards, allowing everyone to have the same view on the same information.
What are the fundamentals?
Landes pointed out the fundamentals for developers as an enabler for DevOps. “There are other things that are interesting that have a very profound impact on this DevOps unity, if you will,” he said. “Unit tests, for example. What relationship does this have to DevOps? If you write your unit tests in a way that they can be executed in any environment, then when you deploy or if you have an issue in production, you can run the selective unit tests for the area under suspicion. In this way you can find issues easier.
“If you have test automation for your regular development life cycle, you can write these test cases and scripts in a way that they can be applied to a production environment. It’s not trivial, I’ll grant you that, but then it becomes much easier to troubleshoot.
“You don’t want to mess with the data in production environments, but if you do it properly, you can, for example, switch data sources in production, and point to the data sources in the staging environment. Then you can run the scripts kind of in production.”
IBM’s Newell said that the heart of all these process is “collaboration. We tend to look at DevOps as more than tools integration. It’s about culture and process and tools. What can a manager do? What we’ve seen, especially among our larger clients moving from or who have moved from waterfall to agile, they’re physically encouraging better collaboration by staffing teams that include business and infrastructure skills as well as developers, and they rotate the leadership through the team. They’ve impacted the physical location by locating these teams together in open pods encouraging integrations.”
Of course, all of these processes require you, the manager, to actually implement them. If your office is filled with a gang of Newmans and Kramers, with their Bizarro equivalents in Ops, you might be better sleeping under your desk. But if you can bridge the gaps and turn those wacky neighbors into one big happy family, you might just be able to push through some major process changes at your own private Vandelay Industries.
