Aviary helps developers choose the best large language model (LLM) for their particular application’s needs. Once chosen, the open-source tool can also be used to integrate it into the application.
“Our goal is to ensure that any developer can integrate AI into their products and to make it easy to develop, scale, and productionize AI applications without building and managing infrastructure. With the Aviary project, we are giving developers the tools to leverage LLMs in their applications,” said Robert Nishihara, co-founder and CEO of Anyscale, the company that created the tool. “AI is moving so rapidly that many companies are finding that their infrastructure choices prevent them from taking advantage of the latest LLM capabilities. They need access to a platform that lets them leverage the entire open source LLM ecosystem in a future-proof, performant and cost-effective manner.”
Developers using Aviary can submit test prompts to many different open source LLMs, such as CarperAI, Dolly 2.0, Llame, Vicuna, StabilityAI, and Amazon’s LightGPT so they can find the one that fits their needs.
It was built on one of Anyscale’s other open-source offerings, Ray Serve, which is used to serve and scale AI applications. This provides features like fault tolerance, dynamic model deployment, and request batching.
It can also provide dynamic cost optimization because it intelligently partitions models across GPUs.
Aviary also provides developers a unified framework for deploying LLMs that was designed to be used with multiple LLMs in an environment.
The project contains libraries, tooling, examples, documentation, and sample code to give developers all they need to get started with Aviary quickly.