Although augmented reality is most commonly associated with overlaying digital imagery and information on top of the real world, researchers have continued to explore potential audio applications of AR, generally using headphones or audio-backed glasses to weave data into a wearer’s exploration of spaces. On October 20, Foursquare will jump into the audio AR field with Marsbot for AirPods, a “lightweight virtual assistant” that whispers location-specific guidance into your ear as you walk through unfamiliar places.

Digital assistants have gone mainstream over the past decade as Google’s Assistant leveraged cloud servers to proactively organize users’ emails and calendars, Amazon’s Alexa facilitated frictionless voice-controlled shopping, and Apple’s Siri promoted on-device processing to reduce user privacy concerns. But for better and worse, these assistants have been passive — evoked only on request — and non-persistent in the sense that they otherwise stay silent. Inspired by Scarlett Johansson’s humanlike AI assistant in the movie Her, Foursquare is exploring how a more active and location-aware virtual aide could work if it was always able to communicate with you, using audio cues rather than on-screen notifications as flags. The AirPods project builds on the company’s prior Marsbot, which uses text messages to offer location-aware tips.

Walking through New York City with Marsbot for AirPods, users will hear tips automatically surfaced for specific nearby locations, attractions, and people, ranging from the rating of a restaurant to the presence of a Foursquare contact. Some of the tips are provided directly by Foursquare, including useful facts that might draw users into businesses, but there’s another layer. Users will be able to record their own audio clips to share with the public, and Foursquare’s samples — “Get the burger!” and “Don’t get the burger!” — barely scratch the surface of what individual- or business-generated recordings might contain. Anticipating some concerns, the company says it “will be proactively monitoring uploaded clips for abusive or offensive content” and let users flag content for human review.

Foursquare bills Marsbot for AirPods as an “experiment” that’s “meant to demonstrate what is possible” with location-aware audio, and the company plans to update the app with additional features based on user engagement results. This could end up paving the road for more useful versions of Google Assistant and Siri — or kick off an era of even more annoying advertising, as McDonald’s tries to steer you away from a local restaurant or five different stores compete for your attention as you walk through a neighborhood. For now, the app is meant to be light, delivering “no more than 10 spoken syllables” per tip, and typically only a few tips per day.

In its “version 1” release, Marsbot will work “anywhere in the world” as users walk, run, bike, or drive. But it has “the most preloaded content” in major cities such as Chicago, Los Angeles, New York City, San Francisco, and Seattle, with tip timing that’s optimized for walking outdoors. Unlike audio AR experiences designed for art galleries or museums, Marsbot isn’t meant to provide walking tours of specific spaces, but rather to offer tips and facts as you walk around, automatically lowering music and podcast volume levels to offer guidance. Foursquare notes that the app won’t interrupt voice or video calls.

Marsbot for AirPods will be available at 8:00 a.m. Pacific on October 20 as a free download from the iOS App Store, and despite the name it will actually work with any pair of headphones. Foursquare has also set up a waitlist for a yet-to-be-named Android version of the app, which should be available in the near future.


The audio problem: Learn how new cloud-based API solutions are solving imperfect, frustrating audio in video conferences. Access here