As the world becomes more connected, so does its data. The data-management tools and techniques of the past are not equipped to handle this non-uniform, semi-structured and highly interconnected data. Faced with this dilemma, software professionals have the option of struggling to fit everything into relational or key-value models, or instead to embrace the graph data model and steer their teams toward success.
It’s not always an easy decision to throw away decades of experience and replace relational databases altogether, but it is straightforward to move the highly connected parts of the system to a graph database. The size and complexity of the data is inevitably fueling a movement toward polyglot persistence (data housed in the stores best equipped to handle it) instead of a single (relational) database. While polyglot requires a broader understanding of data issues, it yields tremendous benefits for data architecture and governance.
To put the drivers for graph database adoption into context, we see that valuable data is generated by the spread of social networks outside their walled garden and into the rest of the Web. The social graph, which entangles the intent, interest and consumption graphs, is driving online commerce and advertisement. Furthermore, the latest advancements in smartphones have rapidly grown the mobile graph tying the online world back to the real. The data generated is already in graph form as people friend each other, like, want, and buy things, and check in to places. In other words, they are creating relationships, the very thing graph databases excel at.
Graph databases have a simple structure, using just two types of objects: nodes and relationships. Users, things and places are modeled as nodes, and various kinds of labeled, directed relationships are created between them (for example, when friend requests are accepted, or things are bought). These simple structures form paths that can then be easily used to compute which of your friends should also be friends, or what new music your friends are buying or what books they’ve recently liked. The data is connected in simple ways, but traversals of these connections can answer very sophisticated questions.
We can query highly connected data using tailored graph query languages that are designed to express intent over graph structures. A graph query language is much simpler than trying to describe such connections with hundreds of lines of SQL code (even with syntactic sugar like Oracle’s CONNECT AS). Finding how things are connected reaches out beyond basic social graphing and into areas like matching people to jobs based on skills and experience, patients to diseases based on symptoms, suspects to crimes, sellers to prospects, borrowers to lenders, etc. One benefit of graph databases is being able to solve problems by matching patterns in a graph.
Another benefit is speed of development. The schemaless nature of graph databases enables developers to rapidly evolve a system as requirements change to align with business needs. Adding a new type of node or relationship is typically painless, and it’s a more natural modeling approach than creating a bunch of join tables and the logic to handle them.
In terms of modeling approach, graph databases are whiteboard-friendly, since business experts can communicate with developers by simply drawing nodes and relationships, which can then be modeled directly in the database, yielding no semantic gap between the user’s data model and how it is expressed in the database. Conversely, with a relational or key-value store, the data model does not resemble the domain (since it is normalized then de-normalized or decomposed into disconnected values). The graph model facilitates the transfer of knowledge from the experts to the developers and business analysts by aligning all stakeholders around the domain.
While some might initially be apprehensive that graph thinking is a new technique to learn, in fact it’s a very natural technique that we simply need to rediscover. But once exposed to graph databases, it’s common to start seeing graphs everywhere, a testament to how natural and powerful the paradigm is.
Once we’re rewired for graphs, we find that graph databases are also able to scale to solve problems in ways other data models can’t. For example, the time required to traverse and return a subset of nodes and relationships remains proportional to the resultant set size (i.e. constant), even as the overall database grows rapidly. Attempts to replicate a graph within relational databases can work at trivial sizes, but performance quickly degrades and becomes unpredictable as the data set size or number of joins increases.
Graph databases also allow us to apply mature analytical techniques from graph theory and discrete mathematics. Graph theory dates back to 1736, when Leonhard Euler published a paper on the “Seven Bridges of Königsberg.” Since then, countless papers have advanced our knowledge of the field. Many such classic algorithms are directly applicable to the problems developers are trying to solve without them having to reinvent the wheel. The additional capabilities of building algorithms that take advantage of the property graph model however remain largely unexplored. This is where some are discovering their secret sauce.
Relational databases and key-value stores aren’t going away anytime soon, but they are impractical for connected data. Graph databases are ideal to help solve sophisticated data problems found in social relationships and highly interconnected systems. It is a new (old!) way of solving problems, and we’re finding many innovative ways of applying it every day.
Emil Eifrem is founder of the Neo4j graph database project, and CEO of Neo Technology.
