Building software, systems, and platforms is complex. With myriad choices and various paradigms and technologies to adopt, engineers have unrelenting (and critical) decisions to sift through. An increasingly large part of the learning curve for any developer is navigating the organizational tooling and processes to get their ideas into the hands of users, and much of that tooling is often outside their expertise. For example, with so many items shifting left, infrastructure and non-functional requirements, such as security concerns, are increasingly being shifted to the developer.
To combat this and to embrace a model where developers can focus on features (e.g. the Spotify Squad/Guild model), platform engineering is key, and internal developer platforms have become more popular. An IDP is the platform, or platforms, that engineers interact with to abstract delivery and infrastructure complexities. But IDPs are not without their own complexities, and it’s important to understand where things can go awry before jumping in.
A quintessential piece in distributed computing is the “Fallacies of Distributed Computing” by Lawrence Peter Deutsch. The fallacies are a set of eight false assumptions that developers invariably make when dealing with distributed systems, such as that bandwidth is infinite or that latency is zero. Similar to the eight fallacies of distributed computing, there are eight fallacies of your internal developer platform:
One – You Will Always Build/Deploy Successfully
A common pitfall in any system development is building for the happy path. Because software requires innovation and iteration, builds and deployments will fail – and a failure and recovery path needs to be accounted for.
Two – Your Administrators Will Stay
People never stay in the same position forever. Deep expertise in bespoke deployments is at risk when those with tribal knowledge move on. This also causes a steeper learning curve for those who onboard as platform administrators or onboard their application to the IDP.
Three – The Deployments Area is Always Homogeneous
Application deployment is the culmination of potentially multiple teams and their respective services. There are several approaches to deployment, but because of variations in the scope of changes, rarely are two changes exactly the same. Modern deployments take into consideration infrastructure (e.g. being ‘infrastructure aware’), and the rapid advancement of IaC (infrastructure-as-code) continues to add to the heterogeneous nature of today’s (and tomorrow’s) deployments.
Four – Platform Cost is Zero
There is an adage that supporting your internal customers will benefit your external customers. However, leveraging a provided IDP comes with costs not only to build and support but also for teams needing to adopt the platform – and especially so if the platform is bespoke to a team or organization.
Five – Technology Doesn’t Change
The only constant in technology is that there will be change. New paradigms are constantly appearing, and approaches and practices are evolving. Looking at the accelerating adoption of Kubernetes within enterprises, organizations need to continuously learn more granularity and nuances about the platform.
Six – Skillsets are Singular
The challenge with balancing self-service, abstraction, and capabilities is that no two users have the same skillset. For example, with shifting items left (like service mesh technology shifting networking concepts to developers), the assumption that every developer is an expert in networking is just false. Having to build for varying skillsets and opinions is a challenge.
Seven – There Will Only Be One Syntax
There is a need to orchestrate several disparate tools, and each tool has a different syntax. For example, authoring a pipeline has a different syntax than infrastructure-as-code, which has a different syntax than a Kubernetes Resource, and so on. Tying all these areas of expertise together should respect the underlying syntaxes, and the expertise required for each.
Eight – Access to the Platform Will Be Secure
When marching towards production, IDPs have elevated privileges to execute tasks. Like any system that has the ability to make an impacting change, good rigor is required. Perimeter control (e.g. if you can access a Pipeline, you could execute) is not an adequate control. Auditability and RBAC controls are necessary.
Creating an internal developer platform is an evolutionary journey that many organizations are embarking upon. But as more organizations embrace IDPs, being aware of the fallacies helps those make more robust systems and practices.
