Fellow architects and designers, I fear that we as an industry are moving our applications and data into the cloud without first having mastered service-oriented architecture, the basic discipline of building distributed systems. In the process, we’re setting ourselves up for failure.

If your experiences have been anything like mine, SOA has been a disappointment. Think back to a decade ago. We were promised lower operating costs, improved business agility and reduced operational risk if we would only adopt SOA. Filled with enthusiasm, and with the encouragement of reputed vendors and analysts, we persuaded our businesses to invest generously in technology and training around service-oriented architecture.

We’re ESB-ed and Registry-ed to the gills. But now, 10 years later, when we total up the benefits we received by adopting SOA—and there have been some—they haven’t been stellar. New systems still cost a lot to build, existing ones still cost a lot to support, it still takes time to create new business capabilities, and our systems still experience unexplained outages. So the promise of SOA hasn’t been realized, if we’re thoroughly honest with ourselves. Where did we fail?

SOA itself is not at fault. The basic principle behind SOA is “loose coupling,” which is based on perfectly sound software engineering experience. Think of “loose coupling” as the absolute minimum required set of dependencies that should exist between any two systems. Too many dependencies can make it harder to change anything because a large number of related changes needs to be made (high operating cost); these changes take time (lower agility); and changing anything risks breaking something unexpectedly (high operational risk). That’s why reducing dependencies to the minimum possible makes such good sense. SOA is therefore just common sense.

Why didn’t SOA succeed then to the extent that it should have?

The IT industry has seen SOA as just a bunch of cool technology products that can reduce the tight coupling between systems, and has forgotten about an equally important source of dependency. That’s the applications themselves, or, boiled down to the essence, it’s the nature of the data that’s exchanged between systems that is responsible for much of this tight coupling.

There is both a technology layer and a data layer in SOA, and we in the industry have been so fixated on the technology layer, so dazzled by the cool products we have, that we haven’t paid attention to what we do at the level of business applications and the kinds of data they exchange.

SOA, then and now
Again, mind you, I’m not knocking the significant advances made at the technology layer to reduce dependencies between systems. There was a time when data encoding was proprietary, and no one had standardized on the ASCII format that we take for granted now. Today, encoding standards like XML and JSON have taken that standardization a level higher up the stack, so systems can not only exchange data, but even make sense of its structure in a uniform way.