Worried your go-to hole-in-the-wall might not have a stellar food safety record? Google’s new artificially intelligent (AI) system can help lay your fears to rest — or confirm the worst of them.
A study led by researchers at the Mountain View company and Harvard’s T.H. Chan School of Public Health describes a machine learning model — FINDER (Foodborne IllNess DEtector in Real time) — that leverages search and location data to identify “potentially unsafe” restaurants. Their paper (“Machine-learned epidemiology: real-time detection of foodborne illness at scale“) was published today in the journal npj Digital Medicine.
“Foodborne illnesses are common, costly, and land thousands of Americans in emergency rooms every year. This new technique, developed by Google, can help restaurants and local health departments find problems more quickly, before they become bigger public health problems,” Ashish Jha, K.T. Li Professor of Global Health at Harvard Chan School and director of the Harvard Global Health Institute, said.
As the study’s authors explain, FINDER takes in anonymous and aggregated logs from users who opt to share their location data. It identifies search queries indicative of food poisoning (e.g., “how to relieve stomach pain”) and then looks up restaurants visited by the users who performed those searches. Lastly, for each applicable restaurant, it calculates the proportion of folks who stopped by and later showed evidence of foodborne illnesses in their searches.
One challenge was dealing with search term ambiguity. As the researchers noted, a Google search for “diarrhea” could be related to food poisoning, but doesn’t convey information about the details of the symptoms. Their solution was a supervised machine learning classifier that leveraged additional signals — including results shown in response to queries, aggregated clicks on those results, and the content of the opened web pages — to identify illness-related searches with 85 percent accuracy.
FINDER was tested in two cities: Las Vegas and Chicago. Their respective health departments were provided a list of restaurants automatically identified by FINDER, which inspectors probed for health code violations. In Las Vegas, a total of 5,038 inspections were completed, 61 of which were prompted by FINDER; of the 5,880 inspections completed in Chicago, 71 were brought on by FINDER.
Of all the restaurants identified by Google and Harvard’s AI system, 52.3 percent were deemed unsafe upon inspection, compared to 24.7 percent for baseline restaurants. Moreover, FINDER restaurants were more likely to be rated unsafe across all risk designations and had a greater number of critical and major violations.
In the end, FINDER outperformed complaint-based inspections and routine inspections in terms of precision, scale, and latency (the time that passed between people becoming sick and the outbreak being identified). And it managed to better attribute the location of foodborne illness to a specific venue than customers did.
“Previous research shows that people tend to blame the last restaurant visited, and therefore may be likely to file a complaint for the wrong restaurant,” the researchers wrote. “The FINDER approach is more robust than individual customer complaints, as it aggregates information from numerous people who visited the venue.”
It’s not a perfect system, that said. Because foodborne illness tends to have a relatively lengthy incubation period, with symptoms occurring after a delay, FINDER achieved peak confidence only after sufficient time had elapsed. And the odds of a restaurant being identified by FINDER as unsafe was higher in lower risk-level venues than in high risk-level ones.
But the tests demonstrated a 68 percent improvement over advanced complaint-based systems that use Twitter data mining, including Chicago’s own.
“Our results provide evidence that this type of tool can be used by health departments today to more rapidly pinpoint and investigate locations where outbreaks may be occurring,” the researchers wrote. This model can be expanded by public health departments to reduce the burden of foodborne illness across the United States, and can also be employed to assist in monitoring a variety of other diseases globally.