StatMuse uses conversational AI and a database of sports trivia and statistics to answer your sports history questions on the major virtual assistant platforms, like Alexa and Google Assistant. The answers come through your speaker in the voices of sports bigshots like Peyton Manning and Scott Van Pelt, which is very fun indeed, but the company took a step into predictive territory by adding fantasy football tips and helps to its roster.

The system that StatMuse built is intriguing for its uncomplicated approach and appealing feature set, and it speaks to how a company can build an engaging AI-powered product without tumbling down an AI rabbit hole. StatMuse focused on building the core technology it needed, and then it developed features on top to make the product appealing to its target customers. And adding a predictive element to the platform turned out to be fairly simple.

There are three parts to StatMuse: natural language processing (NLP), a database of static and real-time information, and celebrity voices. The company’s proprietary NLP stack includes natural language understanding (NLU), natural language generation (NLG), and text-to-speech (TTS).

When you ask a question via a virtual assistant like Alexa, it gets converted from speech to text, and then the NLU layer extracts meaning. The system looks up the question in StatMuse’s database, generates a reply using NLG, and selects the appropriate voice (eg, a player from the team an answer is about). Then it’s converted to voice audio with TTS, the system adds a little visual (like a player’s illustrated headshot), and you hear the reply. There’s generally an “ask” for some kind of follow-up.

statmuse nlp process

Getting the voices

The latest celebrity voice — Matthew Berry — marks a move for StatMuse beyond offering facts and figures and into predictions.

Matthew Berry is ESPN’s fantasy football guru. He cohosts the Fantasy Show on ESPN+, wrote a book, and made a cameo in Avengers: End Game. His voice will pour forth with fantasy football knowledge, driven by the actual predictions he creates for ESPN, from your Alexa-powered device.

In an interview with VentureBeat, Berry described how his collaboration with StatMuse came to be. He and StatMuse cofounder and CEO, Eli Dawson, had been friends for some time. Around a year ago, Dawson approached Berry about working with StatMuse to create the fantasy football feature.

“I loved the idea,” said Berry, “ …because the most common question I get asked all day every day — from my friends, from my kids’ friends, from my coworkers, from people on Twitter or Instagram, when I go out in the street, when I’m in line at the grocery store, whatever — ‘Who do I start?’”

“It is constant,” Berry emphatically reiterated. “I’ve been asked that question in public restrooms, at funerals — like, you have no idea. I can tweet something completely generic that has nothing to do with anything, like hey, happy birthday to my podcast co-host Field Yates. And, you know, half of the responses are like, ‘Yeah yeah yeah, but listen, I got a problem with the flex.’”

For Berry, the idea was that he could essentially multiply himself and make the answers to those questions available to millions of people was a “no brainer.”

In order to get Berry’s voice, and the voices of other sports celebs, into the system, StatMuse brought each person in and recorded them saying a number of different words and phrases. In our interview, neither Dawson nor Berry would elaborate on how much time and effort that process actually took, but Berry did say that he did multiple recording sessions. And Dawson was clear that the task was about creating a library of the person’s lexicon, rather than endless rote sentences that would be replayed.

Here’s a video of Berry in one of those recording sessions:

The end result is, anecdotally, fairly impressive. In the course of chucking fantasy football questions at Alexa, I got answers in the voices of multiple sports figures, and if you weren’t paying close attention, you could mistake some of them for real responses. Other responses were more stilted, with small glitches that betrayed the artificiality of the voices.

Real-time and contextualized data

As with any AI-powered system, data is key. StatMuse has an extensive database of historical sports information — even from digitized decades-old newspapers and the like — but it’s been evolving to include more types of data, like live updates.

“We first [got] really good at kind of answering historical fact questions, where we could go all the way back to the inaugural season — you know, kind of serve those diehard stat junkies within sports,” said Dawson. “And then we kind of evolved to be able to do live stats. So during a game that you could ask, ‘How is Pat Mahomes doing,’ [or] you know, ‘What’s the score of the Niners game,’ and then the natural evolution was to then go to subjective and predictive [answers], particularly focused on fantasy [football].”

But using NLP to pull answers from what is a mostly static database of information is one thing. On gameday, information changes constantly, from scoring to injuries. That’s a whole new sort of challenge.

“So when it comes to things like a live score or a live stat, it’s up-to-the-minute updated,” Dawson said. “In the grand scheme of the technology here, you know, that’s a pretty straightforward process in the sense of just — we have these APIs that, as soon as data input changes, it just runs a process — effectively, every minute.”

Berry has to keep pace, too. Fantasy football recommendations can change dramatically over the course of an afternoon, for example if a player listed as “questionable” to play does or does not end up starting a game.

“My rankings on ESPN.com are constantly updated, right? As are projections, as is my information, so — you know, this is why my wife jokes that I see my phone more than her during the football season. Because I’m constantly near a device at all times. Because football news happens 24/7,” said Berry. “And so, it’s a grind throughout the football season! But, yes, when news happens in the NFL that affects how I perceive their fantasy value will be changed, I make that public. I make that known to ESPN, and obviously that gets known to StateMuse users, as well.”

He did allow that StatMuse has added another layer to his process, and he said that it took he and StatMuse a little while to find some workarounds. “It is a little bit of extra work, but honestly, the heavy lifting is more on the StatMuse side,” he added.

The other side of the data coin, though, is how useful the data actually is to you as a fantasy football owner. There’s a certain amount of information you can get just by adding the Matthew Berry Alexa skill and firing off questions, like what running back has the most fantasy points on the year so far. But if you want more specific answers for your team, like deciding who to start at wide receiver, you would have to upload your roster and tell StatMuse what scoring system your league uses, etc.

That privilege costs you money. It’s not much — $12 per year, with a couple dollars off if you have an Amazon Prime account — but it adds a layer of friction. These predictions are already part of Berry’s job with ESPN, so you can get the same essential information there, but the premium subscription through an Alexa-powered device is the only way you’ll get those contextualized, immediate, audible recommendations in Berry’s voice.

A simple solution

The most clever aspect of StatMuse’s foray into predictions is how easy it is for the company to do it, because it’s just a matter of the data and how it’s processed, and how answers are served up to users.

Regarding predictive data, Dawson said, “It’s just like another stat or piece of data for us.” In the case of fantasy football help, it’s just that this data happens to be predictive. The system is already capable of processing language, extracting meaning, finding answers, and serving them back. The system itself doesn’t need to change much to provide the predictive part; it’s performing the same basic processes, just on a different type of data.

Dawson said the StatMuse pulls in predictive fantasy football data from multiple sources, including Berry. He also said that StatMuse might attempt to create its own projection capabilities at some point, and may get into sports betting.

At launch, the fantasy football features are only on Alexa devices — notably, Dawson said that Amazon approached StatMuse about the idea — but because StatMuse is platform agnostic, it will be coming to additional virtual assistant platforms soon.