BentoML has launched BentoCloud, an inference platform for enterprise AI teams that want control, customisation and efficiency in AI deployment, the company said on 30 June 2024.
BentoCloud lets teams deploy custom AI APIs with any open-source, fine-tuned or custom models. It offers scaling-to-zero, optimised cold starts, concurrency-based auto-scaling, external queuing and stream model loading.
A Bring-Your-Own-Cloud (BYOC) option runs the full serving and inference stack in a customer’s own cloud, so models and sensitive data never leave the customer’s virtual private cloud. The same setup can be deployed across AWS, Azure and GCP, with support for Oracle Cloud, CoreWeave and Lambda Labs planned.
“BentoML has helped TomTom maintain focus on its core competency in maps and navigation services, while also trying out the latest AI technologies speedily,” said Massimiliano Ungheretti, Staff Data Scientist at TomTom.