Microservices and containers have created a world of interoperable cloud-based services that can be moved seamlessly from one platform to another with minimal change. An area that has lagged, however, is providing data that can be used across multiple cloud platforms to power today’s modern applications.
The folks at SwiftStack, a 7-year-old data storage solution provider, have come up with a way to manage data across multiple clouds, based on three fundamental principles – that data is universally accessible; data decisions are based on policies, not manual operation; and metadata is the organizing principle rather than silo locations and hierarchical data structures.
“What we’ve noticed,” said Joe Arnold, co-founder, president and chief product officer at SwiftStack, “is that each public cloud is like its own operating system. Think about an application developer – they’re typically building the application in a specific programming language and targeting a specific environment – Windows, Linux, Mac, mobile devices, whatever. But what about the back-end infrastructure? We’re building applications in these public clouds to take advantage of the different capabilities they have. Google has a bunch of machine learning, Amazon has a whole host of services that they offer. The public clouds aren’t just a dumb place to run virtual machines. What we’re noticing now is that each of those clouds is going after really targeted applications that have unique functionality in those public clouds. It raises up the importance of being able to have access to your data even if it’s in another cloud, or even if that data is on-premises. You need to be able to get that data and manage that data into those public clouds so that you can take advantage of the services being offered there.”
The key to making universal accessibility happen is a new, open-source file system the company has developed – File Access, it’s called –that can provide data to both file-based and object-based applications. “When you look at how data typically can be stored in the public cloud, object is the very dominant way of storing that data, and application developers are building applications around using those Object APIs. Yet when you look on-premises, you see a lot of those applications still being predominantly file-based. There’s an impedance that needs to be worked through, so you can build your new applications with the same data that supports your existing applications.”
If you’re managing data across multiple clouds, one thing that’s critical is that an application shouldn’t have to worry about where data lives. “If it’s in the namespace, it should have access to the data. It shouldn’t care if it’s in a data center somewhere, or in AWS or Google or any other type of cloud,” Erik Pounds, SwiftStack vice president of marketing, said. “Placement of data needs to be policy-driven instead of manually driven. Based on the requirements for the data, different policies can be set for different data sets, which determine how it’s stored and protected and migrated or archived. From an application standpoint, it decides how the data is handled more so than an operator.”
Arnold said he’s seeing customers who tell him they don’t want to have to know necessarily where their data is, but instead want their system to manage and optimize where the data is placed. He said, “The system we have allows data being placed into the system, and policies can be applied to that data, based on age, for example, something gets archived, or something needs to be accessed in a public cloud, then let’s cache that and store that so it’s readily available and faster to access for that application.”
Finally, metadata plays a key role in managing data in a single, flat global namespace. “This,” Pounds said, “is what’s going to break down barriers that restrict value you get out of your data and the utilization of that data. Instead of the knowledge of data being owned by a single application or that data only being in a single location, once you have metadata associated with that data, it can be indexed and searched on. Now, anyone that has access to that data can get something out of it. They can run queries on massive sets of data and narrow that data down and start to utilize and act on that data.
“That’s something we’re seeing as a big change that’s going to enable the management and utilization of data across multiple clouds.”