Oracle wants to address machine learning problems with its newly announced open-source project GraphPipe. The project is a “dead simple machine learning model serving” solution.
“There has been rapid progress in machine learning over the past few years. Today, you can grab one of a handful of frameworks, follow some online tutorials, and have a working machine learning model in a matter of hours. Unfortunately, when you are ready to deploy that model into production you still face several unique challenges,” Vish Abrams, architect for cloud development at Oracle, wrote in a post.
According to the company, there are three challenges when it comes to deploying a machine learning model into a project:
- Lack of a standard for model serving APIs
- Inability to easily build a model server
- Not enough focus on performance
“We created GraphPipe to solve these three challenges. It provides a standard, high-performance protocol for transmitting tensor data over the network, along with simple implementations of clients and servers that make deploying and querying machine learning models from any framework a breeze,” Abrams wrote.
GraphPipe supports TensorFlow, PyTorch, mxnet, CNTK and caffe2 models.
Oracle explained that businesses normally train and deploy machine learning models individually with “bespoke” approaches. “This impacts an organizations’ ability to derive value from its machine learning efforts. If marketing wants to use a model produced by the finance group, they will have to write custom clients to interact with the model. If the model becomes popular sales wants to use it as well, the custom deployment may crack under the load,” Abrams explained. The standard aims to provide the tools necessary to derive value from investments.
In addition, the project includes: a set of flatbuffer definitions, guidelines for serving models, examples for serving models, and client libraries for querying models. “In essence, a GraphPipe request behaves like a TensorFlow-serving predict request, but using flatbuffers as the message format. Flatbuffers are similar to google protocol buffers, with the added benefit of avoiding a memory copy during the deserialization step,” wrote Abrams.
The flatbuffer spec is available on GitHub with clients and servers for implementing Python and Go. The team plans on adding a client for Java soon as well as a TensorFlow plugin.
