If you haven’t heard of polyglot persistence before, simply put, it means using different database technologies to handle specific needs. The term was derived from something else you may have heard: polyglot programming, which expresses the idea that applications should be written in a mix of languages to take advantage of the fact that different languages are suitable for tackling different problems. Polyglot persistence applies this same concept to databases, where you have an application that talks to different databases using each for a specific purpose—whether that is a NoSQL key-value store, document database or a traditional relational database.
Polyglot persistence means freedom for developers who should not be forced to use a single corporate-approved database for all their data-management tasks. More than one database type makes sense today and is being made even easier in this era of cloud computing.
(Related: How databases are changing the food industry)
The notion of having one or a very select number of corporate-approved databases is outdated, and that mindset needs to change. It’s inefficient and it’s probably costing your enterprise more than it should. There are great options available today that include both SQL and NoSQL data stores, including those that are optimized for both operational and analytic workloads, as well as both open-source databases and commercial database products.
There is some tradition of this concept in analytics, where data warehouses have long been used and where Hadoop is now making inroads, but this is a fairly narrow use case. Polyglot persistence is used much more broadly by the cool kids that work at Silicon Valley startups and at big Internet companies, who are always looking for better tools to do their jobs. Not so much in enterprise computing, however, but I believe that now the time has come for that to change.
To get started, let’s look at why using different databases makes sense by looking at some use-case scenarios for specific databases and examine the benefits.
- For querying time series data, like a stock market ticker or real-time events from an Internet of Things (IoT) stream, a vector database like InfluxDB could be many times faster than the typical relational database.
- For peer-to-peer messaging, a distributed key-value store can be extremely fast and use dramatically fewer resources. That is why Apple chose the NoSQL database Riak as the backing store for its iMessage app.
- To store social networking data, a graph database like Neo4j allows the program to traverse the networks of information much more easily and efficiently.
Beyond the benefits of choosing the right database technology for a given application’s data-management challenge, there may be economic reasons to choose a particular database platform. For example, in many cases, a freely available open-source database may be perfectly suited to the task at hand. This means your organization can avoid spending money on a commercial database when one isn’t required. Of course, at the time the application moves into production, you may choose to move to a commercial solution—but again, having that choice creates greater flexibility.
There is another side to this argument, of course; the most common objection being that the soft cost of people needed to operate multiple databases can be quite substantial. Let’s face it: DBAs don’t come cheap—nor should they, given their expertise. Each database technology also comes with its own set of tooling and often requires specialized expertise to administer. Staffing a team of DBAs for a dozen different database technologies would be prohibitively expensive.
While that had been true in the past, now Database-as-a-Service makes it possible to manage multiple types of databases economically. For example, Amazon Relational Database Service (RDS) provides the ability to provision single instances (possibly with replicas) of various relational database servers, including Microsoft SQL Server, MySQL, Oracle, PostgreSQL, and Amazon’s own proprietary MySQL variant, Aurora.
In the OpenStack environment, OpenStack Trove provides even more flexibility. Trove is a single framework within which it is possible to operate 13 different DBMS technologies in a consistent way. Trove offers self-service administration for not just relational databases, but also NoSQL databases and even a data warehouse. Trove currently supports Cassandra, Couchbase, CouchDB, DataStax Enterprise, DB2, MariaDB, MongoDB, MySQL, Oracle, Percona Server, PostgreSQL, Redis and Vertica, with several more currently under development.
Let me be more specific when I refer to self-service administration of multiple databases. The Trove DBaaS automates tasks like backup, replication, failover and cluster management, and does it in a way that leverages the native tools of the underlying database engines it supports. This automation reduces the need for a large staff of experts on each of the databases offered. The DBA does not need to be expert in every kind of database—that is left for the DBaaS.
Another argument for a small number of approved databases in the data center is that while databases on demand via the cloud are fast and convenient, they often operate on the fringes or outside the IT governance. There are potential risks to privacy and security, which are opened up when dealing with mission-critical enterprise data.
One way to address this is to take advantage of a private cloud, such as OpenStack. When OpenStack Trove is deployed on the private cloud, it is safely inside the envelope of corporate governance and data security. Because the database service is operating within the context of the company’s private cloud, the IT team can ensure conformance to corporate policies like data retention and privacy.
As a developer, you deserve nothing less than the ability to select the database of your choice to support your application. It’s time for enterprises to join the cool kids and embrace polyglot persistence. Today, with the advantages provided by the cloud and DBaaS, there is no reason to be locked into the shackles of corporate policies for using only one or a select number of database technologies. Demand that your organization is up to the times and taking advantage of polyglot persistence.