In many large enterprises, software developers are in a tough situation. On one hand, business units demand new applications and services at the speed of market change. On the other, they work with central IT, whose limited resources and focus on system administration make it difficult for developers to get the infrastructure and platform resources they need in a timely manner. As a result, developers often can’t deliver quickly enough to address new market opportunities, and more agile competitors snap them up.

At the end of the day, developers just want to code and test and innovate. With the support of business units happy to hand over a credit card, many development teams have turned to shadow IT—contracting with public cloud providers for resources—as an easy release valve for the pressure from business units. Having spent years in a management role, where we regularly incurred a greater-than-US$10,000-per-month AWS bill because I couldn’t get what I needed from IT for infrastructure and other resources, I can categorically say that this is no way to operate.

Developers should not have to circumvent central IT in order to operate with agility. Development teams who leverage shadow IT are often seen as mavericks, which may be unfair given the organizational constraints they encounter. However, shadow IT poses serious risks to any large regulated enterprise. At a time when many enterprises face growing, nimble competition from around the world, it is crucial that enterprise IT, development teams and business teams gain operational agility and fluidity of resources to stay competitive. Or else, they will face margin compression, loss of market share, or even extinction.

And they must do this while maintaining central IT’s ability to control governance, compliance and security, which cannot be done with rampant shadow IT.

It is central IT’s manual, approval-based operating model that inhibits the allocation of developer resources. In my case, these slow processes and cost justifications meant there was no way we would have been able to obtain the resources we needed in a timely manner. I took a risk by using my corporate procurement card, and was able to leverage on-demand, cloud-based services from AWS, which enabled my team to pay just for what we used and to avoid latency in our development and delivery process.

Central IT did not have the sense of urgency we required, and the procurement process and planning overhead to stand up a single VM would take weeks. Fortunately, my expenses for shadow IT were never kicked back. I consider myself lucky to have learned a few key lessons that I’d like to provide in a constructive public forum for this conversation to move us all forward in working together to stay competitive and get the job done.
Let DevOps guide your way
I believe we need to close the loop between application development and centralized, enterprise IT operations. I am a huge fan of DevOps, which stresses communication, collaboration and integration between development and operations. Continuous integration and delivery enabled via DevOps, where your infrastructure is treated like code, hold the key to mitigating internal friction.

There are organizational challenges in implementing cross-functional processes that enable the provisioning of applications and platforms across the software development life cycle (SDLC). But when they are addressed, developers are freed from the bottlenecks that occur as code moves across dev, test, staging and, finally, to production. Well-managed, cloud-based DevOps unites product development, operations and the business, removing the painful cycle of process-induced delays.

A few key takeaways for me along my journey in Shadow IT:

Lead with the needs of the business when implementing cloud. Identify the operational problems that need to be addressed and the benefits of embracing a cloud-operating model. The development organization shouldn’t bear the burden of finding its own resources, and it should be exposed to the associated operational risks around security, compliance and governance.

Embracing a cloud IT operating model is a game changer. The development organization can move out of the “shadows,” and yet still experience the benefits of the cloud: on-demand, self-service access to elastic resources.

Implement DevOps methodology to speed up time to market. Everyone in the organization has a role to play. Security’s job, for example, is to mitigate risk. Business units are tasked with selling products, and so on. While development’s goal is to create and deliver applications and services as quickly as possible, they often feel that other teams’ goals get in the way.

However, by embracing DevOps, you can significantly reduce organizational friction between development and IT operations. Coupled with a move to a cloud IT operating model, access to development platforms can be provided on demand, either served up as IaaS or PaaS. Cloud-based DevOps will enable you to deliver faster, iterate more and be more competitive.

Set a vision and deliver on it. Depending on what your company does, developers probably create the products, which are monetized. Envisioning the competitive advantages that can be reaped by a cloud-based DevOps model should set the agenda for implementing the new processes and management tools you’ll need. Choose your cloud-management platform wisely and make sure it can meet your strategic goals.

There is a new breed of developers that are moving beyond traditional IaaS or PaaS models to obtain a truly agile development process and improve business responsiveness. It’s time to bring them the resources to innovate in a self-service, on-demand fashion by moving to a new, cloud-based IT operating model that utilizes DevOps methodologies.

Shawn Douglass is CTO of ServiceMesh, and he contributes to IT transformation at Global 2000 enterprises.