In the last few years, we’ve watched DevOps transition beyond a buzzword to become widely accepted by organizations large and small, global and startup. As we take stock of all the happened in 2018, it’s the perfect time to reflect on where we see things going in the new year.
Everyone is seeking systems unicorns
It is becoming increasingly easier to create and run complex scalable systems. Kubernetes and other frameworks have made what was previously a specialized practice into something obtainable by any developer with a credit card; this is an amazing and wonderful thing. Startups and small companies can build and scale at unprecedented rates.
However, as easily as these systems can be built, it’s become just as easy for lower-level details to slip through the cracks. Configuration management, logging streams, process management, scalable design patterns and performance tuning are just some examples of important considerations that are becoming frequently overlooked as organizations scale.
There is a growing demand for these systems unicorns who understand these low-level details, akin in many ways to the UNIX system administrators of the past, compiling kernels and working with low-level systems code. From Senior DevOps Engineers to the SREs at Google (who really are in a league of their own, doing some truly amazing things), we’ll see increasing demand for deep technical experience with operating systems, and distributed systems including Kubernetes.
The dangers of development drudgery
Organizations further along in their DevOps journey are implementing Site Reliability Engineering (SRE) practices.
RELATED ARTICLE: Defining the role of the Site Reliability Engineer
As a methodology, SRE seeks to improve the reliability of software and reduce the operational burden of running that software. There are many concepts in its methodology, but one thing we can all agree on is automating and reducing manual work.
Development Drudgery, referred to as “toil” by Google SRE, is manual and repetitive operational work, and often the day-to-day reality for many. Toil increases linearly as your service scales and can never truly be eliminated. High levels of toil create negative outcomes, burnout, increased risk for error, and ultimately, slow progress. If you’re spending more than 50 percent of your time on toil, it’s time to invest in turning this into positive outcomes through engineering work. As repetitive work, toil lacks enduring value.
In 2019, we’ll see more and more organizations realizing the need to incorporate SRE practices, including eliminating toil. This will allow them to focus on building software and delivering better outcomes for customers, freed from the burden of tedious manual work. Organizations will thus empower their teams to focus on creating real value instead of merely keeping things running. Let’s automate last year’s job.
Kubernetes takes the crown
Kubernetes is everywhere, and for good reason. Vendor-supported Kubernetes implementations like Google Kubernetes Engine, Amazon EKS, and open-source tooling such as kops have made it increasingly simple to deploy what is one of the most transformative tools in runtime and deployment orchestration to recent memory. With rumors of other vendors transitioning their own container orchestration platforms into maintenance, you cannot go anywhere without hearing about Kubernetes.
As an orchestration platform, Kubernetes has assisted dramatically in DevOps practices by being an approachable and standard deployment platform. When teams use the same tooling, they benefit from that shared knowledge. It has greatly assisted in the DevOps goals of common tooling, automation, infrastructure as code, and repeatability. Being usable almost anywhere, multi-cloud deployments are easier than ever and avoids vendor lock in.
The rate of innovation and community involvement has allowed Kubernetes to quickly rise to high levels of adoption, popularity, and explosive growth, and this will only continue in 2019. I am excited to see what happens next.
DevSecOps adoption struggles
It is difficult to argue against DevSecOps, but what is it exactly? DevOps adoption is still widely in progress, and just like DevOps it is a culture change, so is DevSecOps — including security in the development process.
In the earliest days of DevOps, corporate security policies frequently slowed development, and in some cases, these concerns were either bypassed or ignored in the name of progress. The systems and tools of the time frequently made security difficult to automate. Today, however, migration to cloud platforms or more controllable infrastructure is enabling increased security automation and enhancements, with API-controllable network access lists, DDoS mitigations, and access policy management.
There is also an influx of DevSecOps tools to address other concerns and a growing ecosystem of security, and while they may help, a culture of security must come first — and this is where organizations will continue to struggle. Any tooling must be as flexible and supportable as code like the rest of the DevOps ecosystem for it to be successful and adopted by practitioners.
Cloud providers who figure out a way to make this easier, analyzing and reporting on your existing software without requiring additional development, will have a winning combination, providing a valuable safety net.
Automated code analysis and basic vulnerability scanning has become easy, and there is little excuse not to include it in your development process. These bolt-on products can help your security, but are no replacement for a security-conscious culture and development process.
Competitive gaps grow
Now that DevOps has been accepted in the mainstream, the competitive edge of mature organizations who have already realized its benefits will increase, and dramatically. Their advantages from faster time-to-market and higher customer satisfaction will leave other companies wondering: “How are they going so fast?”
Organizations who are behind the curve will have a lot of catching up to do, and because DevOps is not an overnight change, those who delay may be left behind in an increasingly difficult proposition. This is not to suggest that DevOps is the only way operate, but certainly the benefits of time-to-market are difficult to deny.
Simultaneously, while skillsets grow and adapt, the opportunities for highly skilled DevOps practitioners and full stack developers will also increase, creating more demand for top talent who may not want to join an organization who isn’t practicing DevOps successfully.
Embrace the silo
The “You Build It, You Run It” mentality made popular by Werner Vogels, CTO of Amazon, has made a profound impact on the way service teams in DevOps organizations are created. As service teams are responsible for the life cycle of their software, including technical decisions, they may have complete control of all technical choices. Just as team cultures vary, so may their technology choices, implementations, and even workflows.
This can create many silos within a development organization, making portability of both software and people inside the organization challenging. However, the positives may outweigh the downsides. Broader spectrums of technology allow for multiple approaches and for the best ideas to win. It enables those teams to move quickly when needed because of their intimate experience with their services, and freedoms of choice.
It is best to temper this with some standards to gain efficiencies, such as common deployment platforms, datasources, and monitoring tools. Establish guilds and architecture plans, but listen to the teams when they complain. Your teams are usually right, and if they dislike something enough, they may just do it their way, either at your company or someplace else
Strike the right balance and encourage them continually to share, working together to let amazing software be built.
If one thing is for certain, it is that the way we work has changed dramatically. The operational technology available today has come so far to enable Dev and Ops to work together in ways that were simply not possible before. In speaking a common language, and using common tools like Kubernetes and cloud providers like Google or Amazon, it is easier than ever for teams to work together. It will be interesting to see how serverless and other approaches evolve DevOps further in 2019 and beyond, and allow us to work together in developing great software with even more abstraction from the underlying platforms.