Amazon Web Services today debuted Inf1, an instance that powers AI inference in the cloud that CEO Andy Jassy calls the lowest cost inference offering available in the cloud.
“[I]t will have lower latency, it will have 3 times higher throughput, and up to 40% lower cost per instance compared to our G4 instance, which is based on an Nvidia chip which previously was the lowest cost inference instance in the cloud,” Jassy said.
The vast majority of costs for operations using cloud services to power AI solutions comes from inference, Jassy said onstage today at the AWS re:Invent conference in Las Vegas.
The news follows the release of the Elastic Inference service and plans to release the Inferentia AI chip. Inf1 will also be powered by the Inferentia chip made by Annapurna Labs, an Israeli company AWS acquired in 2015.
Inf1 instances are generally available today and will integrate with PyTorch, MXNet, and TensorFlow. Inf1 will be made available for EKS instances and Amazon’s SageMaker for machine learning in 2020.
The news was announced today onstage at AWS re:Invent alongside Graviton2, a 7-nanometer, 64-bit chip made to rival Intel’s X86 in data centers. Jassy said Graviton 2 will power M6G, R6G, and C6G instances. Graviton 2 will have 4 times more compute and 40% better price performance than Intel’s X86 processors, Jassy said. News of a new AWS data center chip first emerged last week.
VentureBeatVentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact. Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:
- up-to-date information on the subjects of interest to you
- our newsletters
- gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
- networking features, and more