Facebook is trying to bridge the gap between reinforcement learning’s impact in research and its narrow range of use cases in production. The company’s new open-source project Horizon uses reinforcement learning to optimize products and services.
Reinforcement learning creates a system that can make decisions, take actions, and adapt based on feedback. This approach can potentially optimize sets of decisions without external input.
According to the company, the technique has shown promising results in research, but it has been difficult for the AI community to adapt the models to real-world requirements for production environments.
Facebook believes that Horizon and similar projects can potentially define how data scientists and engineers apply reinforcement learning in production. Horizon takes issues specific to production environments into account, such as feature normalization, distributed training, large-scale deployment and serving, and data sets with thousands of varying feature types and distributions, the company explained.
The workflows and algorithms in Horizon were built on popular open framework such as PyTorch 1.0, Caffe2, and Spark. This makes the solution accessible to those using reinforcement learning at scale, Facebook explained.
According to Facebook, it deployed Horizon within the company over the past year and has worked to improve the platform’s ability to adapt the decision-based approach of reinforcement learning to large-scale applications.
“Horizon is part of our commitment to the open development of AI — it was an internal Facebook platform that we’re now open-sourcing, and it works with tools that we have already made available to the community, including PyTorch 1.0 and Caffe2. And although Horizon is tuned for production purposes, our experience with the platform has also revealed important research related to integrating RL into other systems. We are leveraging the Horizon platform to discover new techniques in model-based RL and reward shaping, and using the platform to explore a wide range of additional applications at Facebook, such as data center resource allocation and video recommendations,” Facebook wrote in a post.
