Sony AI launches the Gastronomy Flagship Project to apply AI to cooking

Sony launched Sony AI in April to handle AI R&D with a focus on gaming, imaging, sensing, and more. The organization has offices in Japan, the U.S., and Europe and works in collaboration with Sony’s other business units. Today, Sony AI unveiled its first major research initiative in the Gastronomy Flagship Project, which aims to enhance the “creativity and techniques” of chefs around the world.

Sony began investing in AI with cooking applications in April 2018, when the company’s U.S. division — Sony Corporation of America — signed an agreement with Carnegie Mellon University to work on AI and robotics research. At the time, Sony said its initial research and development would focus on optimizing food prep, cooking, and delivery because the technology needed for a robot to handle “the complex and varied task of food preparation” could later be applied to a wider range of industries. According to a McKinsey & Company analysis, 73% of the activities food service workers perform have the potential for automation.

The Gastronomy Flagship Project comprises an AI-powered app for creating recipes, a robot that can assist chefs with cooking, and a “community co-creation” plan that will serve as the foundation for the first two efforts.

To create the app, Sony AI says it will leverage a range of data sources, including recipes and ingredient information like taste, aroma, flavor, molecular structure, and nutrients to train “proprietary AI algorithms” that will assist “the world’s top chefs” in pairing, recipe design, and menu creation. The company notes that recipe creation is a challenging research area for AI because of the infinite combination of ingredients, as well as constraints such as location, climate, season, and health and food preferences that must be taken into account. “Through this app, [we aim] not only to assist in making delicious food, but also to contribute to people’s health and the sustainability of the environment,” Sony.AI wrote.

Above: Sony AI’s gastronomy app concept.

Image Credit: Sony

An AI system that can whip up food pairings and recipes might not be as far-fetched as it seems. IBM recently announced that it is teaming up with McCormick & Company to create new flavors and foods with machine learning. IBM’s Chef Watson, a research project that sought to create new recipes by analyzing the chemical composition of hundreds of different ingredients, produced more than 10,000 novel dishes. And New York startup Analytical Flavor Systems’ platform — Gastrograph —  taps sensory data and machine learning algorithms to suss out products’ flavor profiles and identify areas for improvement. There’s also Los Angeles-based Halla’s I/O platform, which uses AI to generate Netflix-like recommendations for grocery, restaurant, and food delivery apps and websites, in part by leveraging a database of restaurant dish, recipe, ingredient, and grocery item taste and flavor attributes. Others — like Foodpairing, Plant Jammer, Tastewise, and Dishq — offer proprietary recommendation systems that take personal preferences into account.

As for Sony AI’s forthcoming cooking robot, tentatively dubbed Chef Assisting Cooking Robot, Sony says it will help with the entire cooking process from prep to plating through the use of sensors and machine learning approaches like imitation learning. The ambitious goal is to replicate and in some cases exceed the skills and techniques of chefs with “high precision and speed” and to develop robots that can be teleoperated to serve meals to people in remote locations.

Sony AI says that it will support these initiatives with partnerships involving universities, research institutes, and companies “at the forefront” of machine learning research. The division is also pledging to “drive dialogue” with creators and experts in a “wide range” of food-related fields. Experts include Eneko Atxa, chef owner of the three-Michelin-star restaurant Azurmendi in Bilbao, Spain; sommelier, chef, author, and aroma specialist François Chartier; and restauranteur Joan Roca. The aim is to apply any learnings to the development of AI apps and robots.

Robots in the kitchen are nothing new. At the Consumer Electronics Show in January, Samsung unveiled Chef Bot, a collaborative robot with arms designed to help chefs with ingredient prep. Startups like Chowbotics, Creator, Miso Robotics, and Picnic have taken the concept commercial with fast-service robots that can quickly prepare pizzas, french fries, burgers, and more fresh to order. But some robotic chef concepts have struggled to get off the ground. In June, Zume, a robotics startup focused on pizza-making, shut down its business and pivoted to food packaging, production, and delivery systems. Sony’s multifaceted approach is perhaps reflective of the reality that robots designed for the kitchen aren’t a surefire investment.