The idea of having developers and operations work together towards one goal to achieve velocity and high-quality software sounds like a good idea in theory, but how has it been working in practice? A newly released report revealed that DevOps practices are actually paying off for organizations in terms of performance and quality outcomes.
The 2018 Accelerate State of DevOps Survey report comes from DevOps Research and Assessment (DORA) in collaboration with Google Cloud. About 1,900 professionals worldwide participated in this year’s study. The report is designed to find what issues matter the most to technical professionals and find new ways organizations and teams can improve. The survey looks at IaaS and PaaS, monitoring and observability, databases, testing, workflow, culture, security and reliability.
“Our goal is to help organizations understand how they can be high performers,” said Dr. Nicole Forsgren, founder and CEO of DORA. “We are at the state of work that drives technology transformations and changes the world. This year’s survey builds on the last four years of work to give us the most comprehensive view of IT impacts on the industry yet.”
According to DORA, this is the first time in five years the research is expanding to include availability. “This addition improves our ability to explain and predict organizational outcomes and forms a more comprehensive view of developing, delivering, and operating software. We call this new construct software delivery and operational performance, or SDO performance. This new analysis allows us to offer even deeper insight into DevOps transformations,” the report states. When it comes to SDO performance, DORA finds it unlocks competitive advantages such as profitability, productivity, market share, customer satisfaction and completed mission goals.
To capture SDO performance, the team looked at global outcomes rather than output. The four main measures of SDO performance the report looks at are deployment frequency, lead time for changes, time to restore service and change fail rate. When looking deeper into teams, the report found high, medium and low performers. This is the first year the data showed a fourth high-performance group: elite performers. “This new category exists for two reasons. The first is that we see the high-performing group growing and expanding, suggesting the overall industry is improving its software development and delivery practices,” according to the report. “The second is that the elite group demonstrates that the bar for excellence is evolving across the industry, with the highest performers still optimizing for throughput and stability.” Elite performers are more likely to deploy on-demand and take less than one hour for lead time changes. In comparison, high performances deploy between once per hour and once per day and take between one day and one week to make changes. Low-performing teams deploy between once a week and once a month and take between one to six months to make changes. In addition, the elite performers deploy code 46 time more than low performers, and 2,555 times faster. They have a 7 times lower change failure rate and are 2,604 times faster to recover from incidents.
When it comes to availability, which in this report means ability to make and keep promises and assertions on a software product or service, the report found availability correlated with the performance profiles. Elite performances are 3.55 times more likely to have strong availability practices, for example.
The report also found organizational performance is measured through profitability, productivity, market share, number of customers, satisfaction and quality of services. In addition, organizations have goals of improving quality, and can measure those through time spent on manual work, unplanned work or rework, security remediations and customer-support work. “Our analysis shows that high performers do significantly less manual work across all vectors, spend more time doing new work, and spend less time remediating security issues or defects than their low-performing counterparts. Because they build quality in, they spend less time fixing problems downstream, freeing up more time to do value-add work,” the report states.
Another interesting finding was how organizations implement cloud infrastructure has a big impact on performance. For instance, the report found teams that adopt essential cloud characteristics are 23 times more likely to be characterized as elite performers. The five essential characteristics of cloud computing are: On-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service.
“A stunning finding in the data this year is the compelling case of doing cloud right. Adopting essential cloud characteristics drives high performance and differentiates the highest performers from those who may say they are in the cloud, but are unable to realize performance gains because their technology and processes aren’t executed correctly.”
When looking at open-source software, the report found it is 1.75 times more likely to be used by higher performers, who are also 1.5 times more likely to expand open-source usage in the future. “Over the last two decades, open source software has become widely adopted. In our survey, 58 percent of respondents agreed that their team made extensive use of open source components, libraries, and platforms, with over 50 percent agreeing that their team planned to expand use of open source software,” according to the report.
Surprisingly, the report did find that outsourcing by function hurts performance. Lower-performing teams were 4 times more likely to outsource whole functions such as testing or operations than higher-performing teams.
Other key findings were that technical practices such as monitoring, observability and continuous testing drive high performance, and industry doesn’t matter when it comes to achieving high performance for software delivery.
In addition to Google Cloud, other sponsors of the report included: Amazon Web Services, CA Technologies, CloudBees, Datical, Deloitte, Electric Cloud, GitLab, IT Revolution, Microsoft, PagerDuty, Pivotal, Redgate, Sumo Logic, Tricentis, and XebiaLabs.
“Incorporating DevOps is becoming a mandate if you want to improve performance, productivity, and value,” said Bill Briggs, global CTO of Deloitte Consulting LLP. “The annual State of DevOps Report reinforces the importance of learning how to deliver technology differently in today’s world—from how you utilize open source to implementing cloud to best practices around testing, change management and security.”
