Boris RenskiHistorically, commercial software provided enterprises with reliability and scalability, especially for mission-critical tasks. No one wanted to risk failure in finance, operations, or any essential or enterprise-wide areas. So, enterprises considered open-source technology only for less important, more tactical purposes.

Recently, however, many large IT organizations have developed significant open-source strategies. Cisco, Dell, NASA and Rackspace came together to give birth to OpenStack. VMware acquired SpringSource and shortly thereafter announced Cloud Foundry, its open-source PaaS. Amazon, and others built solutions entirely on an open-source stack. Whole categories of technologies, such as NoSQL databases, made their way to mass adoption shortly after being open-sourced by Google and Facebook. There has been more activity in open source during the last two years than in the preceding decade. So what’s going on here?

Without a doubt, cloud is the IT topic that’s been grabbing headlines and investment dollars in the past few years. The recent high level of activity in open source noticeably correlates with the cloud movement, because there is a deep, synergetic relationship between the two. In fact, cloud is the primary driver for the increased adoption of open source.

In general, open-source projects typically require two components to get community uptake. First, the nature of the project itself has to be technologically challenging. Successful open-source projects are largely about solving a set of complex technological tasks vs. just writing a lot of code to support complex business processes, such as the case with building enterprise software. Linux, MySQL and BitTorrent are all good examples here.

Second, it requires a high rate of end-user adoption. The more people and organizations that start using the open-source technology at hand, the more mature the community and the technology itself becomes.

Cloud has created an enormous amount of technologically challenging fodder for the open-source community. The adoption of cloud translates to greater scale at the application infrastructure layer. Consequently, all cloud vendors, from infrastructure to application, are forced to innovate and build proprietary application infrastructure solutions aimed at tackling scale-driven complexity. Facebook’s Cassandra and Google’s Google File System/Hadoop/BigTable stack are prime examples of this innovation.

However, it is important to note that neither Facebook nor Google are in the business of selling middleware. Both make money on advertising. Their middleware stack may be a competitive advantage, but it is by no means the competitive advantage. Because companies want to keep IT investments as low as possible, a the natural way to drive down costs associated with scale-driven complexity is to have the open developer community help address at least some of the issues to support and growing the stack.

The result? Instances like Facebook’s open-sourcing of Cassandra, and Rackspace contributing its object storage code to OpenStack. Ultimately, cloud drives complexity while cloud vendors channel that complexity down to the open developer community.