Akamai has announced the launch of Akamai Cloud Inference, a new solution that provides tools for developers to build and run AI applications at the edge.
According to Akamai, bringing data workloads closer to end users with this tool can result in 3x better throughput and reduce latency up to 2.5x.
“Training an LLM is like creating a map, requiring you to gather data, analyze terrain, and plot routes,” said Adam Karon, chief operating officer and general manager of the Cloud Technology Group at Akamai. “It’s slow and resource-intensive, but once built, it’s highly useful. AI inference is like using a GPS, instantly applying that knowledge, recalculating in real time, and adapting to changes to get you where you need to go. Inference is the next frontier for AI.”
Akamai Cloud Inference offers a variety of compute types, from classic CPUs to GPUs to tailored ASIC VPUs. It offers integrations with Nvidia’s AI ecosystem, leveraging technologies such as Triton, TAO Toolkit, TensorRT, and NVFlare.
Due to a partnership with VAST Data, the solution also provides access to real-time data so that developers can accelerate inference-related tasks. The solution also offers highly scalable object storage and integration with vector database vendors like Aiven and Milvus.
“With this data management stack, Akamai securely stores fine-tuned model data and training artifacts to deliver low-latency AI inference at global scale,” the company wrote in its announcement.
It also offers capabilities for containerizing AI workloads, which is important for enabling demand-based autoscaling, improved application resilience, and hybrid/multicloud portability.
And finally, the platform also includes WebAssembly capabilities to simplify how developers build AI applications.
“While the heavy lifting of training LLMs will continue to happen in big hyperscale data centers, the actionable work of inferencing will take place at the edge where the platform Akamai has built over the past two and a half decades becomes vital for the future of AI and sets us apart from every other cloud provider in the market,” said Karon.
