Update 11:10am: We’ve corrected this story in several places with some additional information Amazon provided us about how Firefly works. The changes are noted in italics.
Amazon is a fascinating company, and the Amazon Fire Phone is a fascinating machine for connecting you with stuff to buy. It’s probably also the biggest single invasion of your privacy for commercial purposes ever.
And no one seems to have noticed.
There’s a lot of gee-whiz gadgetry in the new Fire Phone: a 3-D screen, head sensors, dynamic perspective shifts as you move, and real-time identification of over 100 million objects. That last part, the real-time identification, is the new Firefly function.
Firefly is a seriously impressive combination of hardware, software, and massive cloud chops that delivers an Apple-like simplicity to identify objects like books, movies, games, and more, just by pointing your Fire Phone’s camera at them and tapping the Firefly button.
We’ll be exploring how you can grow your mobile business at MobileBeat 2014 in San Francisco on July 8-9.
Lest you noticed a common denominator to those items and get the crazy idea that Firefly is only for stuff you can buy at Amazon, it also recognizes songs (oh, you can buy those on Amazon too) and TV shows (ditto) as well as phone numbers, printed information, and QR codes.
How do you think it recognizes those things, including text on images, for which Amazon says it will offer language translation features later this year?
Well, the Firefly button and the camera button are one and the same. Meaning that whenever you’re using Firefly, you’re using the camera. Plus, of course, you’re turning on audio sensors that capture ambient sound.
And then you’re transmitting all those pictures and sound files to the grandaddy and global leader in connected cloud technology, the company that pretty much invented what we now call big data analytics for customer insights, and the largest online retailer in the wild wild west.
(Update: Amazon has clarified that while Firefly does use the camera and microphone, the camera app is separate from Firefly. Your personal photos and videos are not uploaded to the Firefly system; they’re stored separately in your personal storage account in Amazon’s cloud, and not used by Amazon in any way, Amazon says.)
All of those pictures require processing, analysis, and matching, presumably at a level — if they can identify 100 million objects — that can only be done in the cloud, and not on a small handheld device with 2 GB of RAM and 32 GB of on-board storage.
Fortunately for you, dear consumer, Amazon has kindly consented to store all your photos, forever, in its vast cloudy server farms. How gracious Amazon is, providing that massive service for free! How lucky are you, getting all that for free!
(Update: Amazon says you can delete your photos and recordings from Firefly at any time. However, until you delete them, it does use any Firefly photos and recordings to enhance its recognition system.)
Probably not as lucky as Amazon.
By storing all the photos you’ll ever take with Firefly, along with GPS location data, ambient audio, and more metadata than you can shake a stick at in Amazon Web Services, Amazon will get unprecedented insight into who you are, what you own, where you go, what you do, who’s important in your life, what you like, and, probably, what you might be most likely to buy.
Babies in your pictures? Sell that dame diapers. Lots of old-school hot rods? See if you can sell Billy Bob some NASCAR shwag, or maybe beef jerky. Outdoorsy, are you, with your pictures of remote mountaintops and idyllic forest meadows? Clearly you need hiking boots and granola. Looking at a business card? Perhaps things she likes will be things you’ll like, too.
Big data? This is gargantuan data.
Privacy violations a la three-letter U.S. intelligence agency? This is the NSA’s wet dream.
Firefly is “instant gratification,” says TechCrunch. Fire Phone is an “amazing piece of hardware,” says Wired. Amazon’s Fire Phone APIs are a dream for developers, we said. Firefly lets you “easily price-check items,” says GigaOm. Firefly is the phone’s “sexiest feature,” we said.
It also might just make you Amazon’s bitch.
“We care about consumers’ privacy,” the Amazon press release announcing Fire Phone does not say.