Amazon debuts AWS Inf1, an AI inference instance

Amazon Web Services today debuted Inf1, an instance that powers AI inference in the cloud that CEO Andy Jassy calls the lowest cost inference offering available in the cloud.

“[I]t will have lower latency, it will have 3 times higher throughput, and up to 40% lower cost per instance compared to our G4 instance, which is based on an Nvidia chip which previously was the lowest cost inference instance in the cloud,” Jassy said.

The vast majority of costs for operations using cloud services to power AI solutions comes from inference, Jassy said onstage today at the AWS re:Invent conference in Las Vegas.

The news follows the release of the Elastic Inference service and plans to release the Inferentia AI chip. Inf1 will also be powered by the Inferentia chip made by Annapurna Labs, an Israeli company AWS acquired in 2015.

Inf1 instances are generally available today and will integrate with PyTorch, MXNet, and TensorFlow. Inf1 will be made available for EKS instances and Amazon’s SageMaker for machine learning in 2020.

The news was announced today onstage at AWS re:Invent alongside Graviton2, a 7-nanometer, 64-bit chip made to rival Intel’s X86 in data centers. Jassy said Graviton 2 will power M6G, R6G, and C6G instances. Graviton 2 will have 4 times more compute and 40% better price performance than Intel’s X86 processors, Jassy said. News of a new AWS data center chip first emerged last week.