Why do 87% of data science projects never make it into production?

“If your competitors are applying AI, and they’re finding insight that allow them to accelerate, they’re going to peel away really, really quickly,” Deborah Leff, CTO for data science and AI at IBM, said on stage at Transform 2019.

On their panel, “What the heck does it even mean to ‘Do AI’? Leff and Chris Chapo, SVP of data and analytics at Gap, dug deep into the reason so many companies are still either kicking their heels or simply failing to get AI strategies off the ground, despite the fact that the inherent advantage large companies had over small companies is gone now, and the paradigm has changed completely. With AI, the fast companies are outperforming the slow companies, regardless of their size. And tiny, no-name companies are actually stealing market share from the giants.

But if this is a universal understanding, that AI empirically provides a competitive edge, why do only 13% of data science projects, or just one out of every 10, actually make it into production?

“One of the biggest [reasons] is sometimes people think, all I need to do is throw money at a problem or put a technology in, and success comes out the other end, and that just doesn’t happen,” Chapo said. “And we’re not doing it because we don’t have the right leadership support, to make sure we create the conditions for success.”

The other key player in the whodunit is data, Leff adds, which is a double edged sword — it’s what makes all of these analytics and capabilities possible, but most organizations are highly siloed, with owners who are simply not collaborating and leaders who are not facilitating communication.

“I’ve had data scientists look me in the face and say we could do that project, but we can’t get access to the data,” Leff says. “And I say, your management allows that to go on?”

But the problem with data is always that it lives in different formats, structured and unstructured, video files, text, and images, kept in in different places with different security and privacy requirements, meaning that projects slow to a crawl right at the start, because the data needs to be collected and cleaned.

And the third issue, intimately connected to those silos, is the lack of collaboration. Data scientists have been around since the 1950s — and they were individuals sitting in a basement working behind a terminal. But now that it’s a team sport, and the importance of that work is now being embedded into the fabric of the company, it’s essential that every person on the team is able to collaborate with everyone else: the data engineers, the data stewards, people that understand the data science, or analytics, or BI specialists, all the way up to DevOps and engineering.

“This is a big place that holds companies back because they’re not used to collaborating in this way,” Leff says. “Because when they take those insights, and they flip them over the wall, now you’re asking an engineer to rewrite a data science model created by a data scientist, how’s that work out, usually?”

“Well,” Chapo says, “It doesn’t.”

For example, one of his company’s early data science projects created size profiles, which could determine the range of sizes and distribution necessary to meet demand. Four years ago the data science team handed the algorithm to an engineer, and it got recoded in Java and implemented. Two weeks ago, they realized that that it’s been broken for three and a half years.

“It’s broken because nobody owned it, we didn’t have the data science team to be able to continually iterate on the models, think of it as an asset, and have data operations making sure it’s working well,” Chapo said. “We’re starting to bring those ways of working to life. But it’s hard, because can’t just do it all overnight.”

“One of the biggest opportunities for all of us today is to figure out how we educate the business leaders across the organization,” Leff said. “Before, a leader didn’t need to necessarily know what the data scientist was doing. Now, the data scientist has stepped into the forefront, and it’s actually really important that business leaders understand these concepts.”

AI is not going to replace managers, she adds, but managers who use AI are going to replace those who don’t.

We’re starting to see that awakening of business leaders wanting to understand how machine learning works, and what AI really means for them, and how to leverage it successfully. And those leaders are going to be the most in demand, Leff said.

Another essential key to success, Chapo added, is keeping it simple.

“Oftentimes people imagine a world where we’re doing this amazing, fancy, unicorn, sprinkling-pixie-dust sort of AI projects,” he said. “The reality is, start simple. And you can actually prove your way into the complexity. That’s where we’ve actually begun to not only show value quicker, but also help our businesses who aren’t really versed in data to feel comfortable with it.”

It’s not necessarily the sophistication of the model at the beginning, it’s about creating a better experience for customers. Companies actually no longer compete against their closest competitor, they’re actually competing against the best customer experience someone else has provided, even if that’s in an entirely different sector. If you can call up a ride sharing service on an app in just a few moments, you begin to want the same level of experience, when you call the bank, or file an insurance claim, or place an order online.

There are three ways to get started, and avoid becoming one of the 87%, Chapo said. Pick a small project to get started, he says — don’t try to boil the ocean, but choose a pain point to solve, where you can show demonstrable progress. Ensure you have the right team, cross-functionally, to solve this. And third, leverage third parties and folks like IBM and others to help accelerate your journey at the beginning.