Royal Carribean’s Spectrum of the Seas — the third-largest class of cruise ship in the world by gross tonnage — can accommodate over 6,000 people and 2,500 crewmembers during its nine-night Asia voyages. That’s a lot of mouths to feed, particularly mornings prior to port, when throngs of passengers nab breakfast before deboarding.
So how does the crew keep things humming along smoothly without impacting guests’ experiences? Computer vision plays a role — it’s the subfield of AI that helps computers “see” objects in digital photos, videos, and other media. To the extent that Royal Carribean employs it, it’s enabling customers and staff alike to keep tabs on crowd congestion at the ship’s top onboard dining destinations.
“We don’t want guests’ first experience on the ship to be waiting in line,” said David Milone, project manager and lead UX strategist at Levatas, an AI consultancy firm that’s working with Royal Carribean to implement an AI-powered people-counting system built in partnership with Microsoft, at VentureBeat’s Transform 2019 conference in San Francisco last week. “We [looked] at where we could get the biggest bang for our buck.”
Milone and team narrowed on the Windjammer to start, a buffet-style marketplace found in most of Royal Carribean’s vessels that’s included as a part of every vacation package. It’s reliably popular — the Windjammer can seat over 1,000 patrons — and it’s equipped with the same connected security cameras that monitor the Spectrum of the Seas’ other high-value assets.
CCTV worked to the team’s advantage in the end. They annotated tens of thousands of images captured from them to train a computer vision algorithm to recognize people, even those partially occluded by walls, tables, and chairs.
An outsize challenge was eliminating dead zones — i.e., spots where the algorithm was likely to miss passengers — while working within the constraints of the cameras’ picture-distorting fisheye lenses. That wasn’t the only blocker the team had to contend with; unreliable connectivity at sea meant the algorithm, once trained, would have to ingest new footage on local computers instead of banks of remote machines.
The solution to the connectivity problem turned out to be on-ship servers custom-tailored for AI workloads, which laid the groundwork for a retraining pipeline the team used to adapt the system to Windjammer locations on other ships. (Royal Carribean now has three ships with such servers.) The system isn’t perfect by any stretch, conceded Milone — it achieves about 90% accuracy with images containing 80 people or less — but work on higher crowd densities is ongoing, and the team recently piloted a more efficient model that tracks venue entrances and exits as opposed to whole areas.
“[With this data, we] can say how people are waiting to get into each venue [and] start to direct guests where they should go,” said Milone, “and [we can] also feed that data to our strategy teams.”
Royal Carribean’s investments in AI extend beyond crowd analytics. The cruise line taps computer vision-equipped cameras to recognize travelers as they board ships, cutting down the need for verifying travel documents manually. Another machine learning system helps Royal Carribean’s food services teams make decisions about orders to ensure wastage is kept to a minimum. And more recently, the company began piloting a chatbot that allows passengers to ask questions about activities and destinations.