Google has announced the release of TensorFlow Serving, a flexible, high-performance and open-source serving system for machine-learning models. The solution was designed for production environments and optimized for TensorFlow, the company’s machine-learning system.
“Machine learning powers many Google product features: from speech recognition in the Google app to Smart Reply in Inbox to search in Google Photos,” wrote Google software engineer Noah Fiedel in a blog post. “While decades of experience have enabled the software industry to establish best practices for building and supporting products, doing so for services based upon machine learning introduces new and interesting challenges. Today, we announce the release of TensorFlow Serving, designed to address some of these challenges.”
According to the company, with TensorFlow’s ability to run multiple models at large scale, it enables model life-cycle management, experiments with multiple algorithms, and efficient use of GPU resources. The project aims to make it easier, faster and safer to take a model into production while using the same server architecture and APIs.
“TensorFlow Serving uses the (previously trained) model to perform inference: predictions based on new data presented by its clients,” wrote Fiedel. “Since clients typically communicate with the serving system using a remote procedure call (RPC) interface, TensorFlow Serving comes with a reference front-end implementation based on gRPC, a high performance, open-source RPC framework from Google. It is quite common to launch and iterate on your model over time as new data becomes available, or as you improve the model. In fact, at Google, many pipelines run continuously, producing new model versions as new data becomes available.”
The project is available on GitHub through the Apache 2.0 open-source license. It is written in C++, provides support for Linux, and uses minimal overhead.
More information is available here.
Top 5 projects trending on GitHub
#1. Free Code Camp: Still trending! Featured in our GitHub project of the week many moons ago.
#2. 30 Days of Swift: 30 days of 30 tiny Swift projects.
#3. Hosts: A repository that consolidates several reputable host files from a variety of sources.
#4. diff-so-fancy: Builds on output of Git contrib’s diff-highlight to upgrade your diffs’ appearances.
#5. Awesome Postgres: Inspired by Awesome MySQL, a list of PostgreSQL software, libraries, tools and resources.
