Remember the reality presented in the movie “The Matrix”? While the population lived in a fantasy world, they were actually being used as an energy source. A small group of revolutionaries led by Neo would fight the powers that be to free the human race. The world as the machines and Agents knew it was changing, and the controlling powers faced a new reality. And they were nervous.

The world inside the data center has been changing too, and it is changing fast. The large, status quo storage companies are just as nervous. This group of large legacy system companies has ruled the data center for the past 40 years. They’re the ones selling all that pricey systems hardware—especially in storage found in every organization. They are pushing their brand of reality, and when those companies came knocking, you paid, even as you felt something was not right.

But the times, says Dylan, they are a-changin’.

(Related: Have we been looking at databases wrong this whole time?)

After four decades of legacy systems vendors selling massively expensive storage systems (we call them “MESS,” for obvious reasons), and customers reluctantly paying high prices for what Sage Weil called “commodity components wrapped in branded tin,” there is new excitement in the data center: the promise of lower costs and freedom from the tight grips of the storage Agents.

Relief comes in the form of open-source-driven software-defined data storage (OpenSDS). OpenSDS is a radically new approach that builds on innovations such as cloud storage, virtualization, open-source software, hybrid or all flash systems via a combination of hard disk drives (HDDs) and solid state drives (SSDs), and other storage-related developments.

OpenSDS includes one feature that is critical to software-defined storage (SDS), but which few vendors have embraced: open-source software that is developed and improved by large communities of researchers and developers. As Weil suggests, “The true potential of software-defined anything isn’t unlocked unless the software is also open source.” Sage advice indeed.

Why is open source so critical to SDS? We will answer that question in a minute, but first some background on the emerging importance of SDS in the data center.

‘Storage is at a breaking point’
When I go out and talk to customers and partners, they aren’t bashful about telling me what they think. They want to break the ties that bind them.

They say that infrastructure and data centers are too slow, too fat, too rigid, too complicated, and far too expensive—both to build and operate. They want solutions to transform the data center end to end, as they did server virtualization to transform the “compute” layer of the data center. They want low cost, simplicity and freedom from vendor lock-in. And they definitely want to reduce total cost of ownership via both CAPEX and OPEX, especially in the “storage” layer, as storage is the most “consumable” and fastest-growing layer in the data center.

Some others tell a similar story. “Storage is at a breaking point,” says Simon Robinson of 451 Research. He says that most organizations have an “unscalable hodgepodge of systems” that can’t support change and agility in the business environment. He believes that IT organizations “are looking to streamline their data storage infrastructure so that it can respond to emerging business requirements and support the new style of IT.”

If you think data storage is challenging now, consider the impact of all the new data devices coming online under the banner of the Internet of Things. According to analyst firm IDC, there will be approximately 212 billion Internet-connected things by the end of 2020, from healthcare instruments to tracking devices to wearables. IDC says data will continue to double every two years, which Information Age calls “scary” for CIOs. Already we’re hearing from our customers that data is growing so quickly they feel like they’re now in the storage business.

How do you regain control of data storage without being consumed by it?

Organizations know they have to be strategic about the data center. They want an architecture that meshes public and private clouds. They want the ability to save money by running on industry-standard commodity hardware, whether HDD or SSD. They want agility, flexibility, reliability and scalability so they can look beyond today’s immediate needs and plan for the future. Most of all, they want to spend less time, money and attention on storage than they currently do.

SDS has been proposed as the answer, but keep your wits about you. The term and concept are still relatively new, and there isn’t yet consensus about what SDS really means. Moreover (and this may surprise you) some vendors simply repackage existing solutions to capitalize on the popularity of SDS and its market momentum. Proprietary software and closed technology bundles can lock you into working with a single vendor.

Decoupling management software from the underlying hardware systems used to manage it is a key concept for SDS, which then allows an administrator to select from different options when provisioning storage. Newer technologies like flash-based storage or cloud storage can be added easily and quickly to the overall storage pool, and then made available to applications or workloads.

The logical extension (and truest expression) of that idea is an open-source-driven approach to SDS, and the next frontier—OpenSDS—which allows the software to run on a wide variety of hardware. Building on an open-source platform like OpenStack or CloudStack can ensure that the software is 100% hardware-, protocol- and app-agnostic, which in turn protects organizations against single-vendor lock-in on either the hardware or the software side. Using open-source-driven software also provides the benefit of involvement and improvements from a much larger community than any one vendor.

OpenSDS will transform data storage just like server virtualization has been transforming compute capabilities, and software-defined networks are changing networking realities and capabilities. These software innovations enable organizations to scale up and improve agility while saving money on CAPEX and OPEX.

Resistance is never futile
Until recently, not much had changed in the storage business over the past 40 years. As 451 Research points out, incumbents in the storage space have had little incentive to change what they’re doing. But that old way of doing business is quickly fading in the face of exploding data, rich media-driven social media, hyper-mobility, the Internet of Things, cloud computing, and so on. The data volume in the universe (as we know it) will be growing from 6 zettabytes to 41 zettabytes in the next five years according to industry analysts, while data storage budgets are being cut in the same timeframe.

Don’t expect the storage machine to go down without a fight. Be aware. Be brave. Join the growing community that has embraced open-source-driven SDS and, overall, the software-defined data center. With a little help from your friends, you can free yourselves from the Matrix and create your own reality. With a little bit of software, you can get your freedom and give tons of love to your internal and external customers in your extended organization. You can be your company’s Neo.