What’s next for big data after a turbulent 2019?

When thousands of data scientists and business analysts converge on New York next week for the year’s main data industry event, Strata, they will be looking for answers to the current chaos in our business and technological ecosystem.

We’ve seen tectonic shifts in the data and analytics space in 2019, fueled by a seemingly endless wave of acquisitions, some of which were truly epic in proportion: Salesforce acquired Tableau for $15.7 billion, and Google acquired Looker for $2.6 billion. Further downstream in valuation but no less significant, CommVault purchased Hedvig for $225 million, Cloudera bought Arcadia Data, Logi Analytics bought Zoomdata, Sisense bought Periscope Data, and Alteryx bought ClearStory for $20 million. That’s more than $18 billion worth of acquisitions — and we’re only in three-quarters of the way through the year.

At the same time, Hadoop’s precipitous decline further increased this year with Cloudera’s stock price dropping in June and MapR’s value plunging. With so much enterprise data enmeshed within Hadoop, its decline raises a lot of uncertainty.

Given all of those shakeups in the data world, I expect this year’s Strata experience to be markedly different from previous years. Here are the five things I predict will be most discussed on the show floor, over coffee, and in the after-parties.

1. The acquisitions have only just begun

The wave of 2019 acquisitions are just the beginning. That said, we’re not looking at a fundamental shift in the business intelligence landscape. Rather, we’re seeing a natural progression as businesses seek to maximize their investments in business intelligence (BI) — gaining faster, simpler, and more accurate insights. BI is moving out of the hands of data scientists and into the hands of decision makers – which is exactly why we’re going to see more consolidation on the interpretation side of BI and less on the presentation side.

2. Keep your eye on AI

It’s an open secret in the entrepreneurial community: if you don’t have “AI” in your tagline, you get less funding. But there’s more than hype behind this. Enterprises are moving forward in their big data life cycle. The phase of establishing data discipline – i.e. achieving the capability of not throwing away any data — is behind us. Now, companies are at the stage of seeking actionable and accessible insights from their data. And only AI-powered machine learning apps can make sense of such massive data sets. And I’m not talking about AI-fueled Netflix viewing recommendations based on your past history. I’m talking about putting AI’s rubber to the road to seriously impact revenues and competitive edge.

3. Big time investment in apps for the cloud

The on-prem processing power that would be needed for effective real-time BI over massive data sets is prohibitively expensive. On-prem tech was built for batch processing, but today’s businesses can’t wait for jobs that take tens of hours or more. This is why data migration to the cloud has gone into overdrive. And now it’s time for companies to reap the fruits of this move with cloud-specific versions of the apps that run their businesses. New app projects start with cloud-native versions. Yet 90% of enterprise apps were built to run on prem – and these need to be massively customized or completely rebuilt to work on the cloud.

4. Hadoop isn’t quite dead yet

Hadoop’s freefall this year – especially given Cloudera’s stock price collapse and MapR’s precipitous value drop – is indicative of tech from a different era. Today, the world has moved from on-prem storage and billions of batch-based queries to real time analytics over massive cloud-based datasets. Yet despite the hype, Hadoop is not quite dead yet. And although the name Hadoop will eventually disappear from our lexicon, at Strata this year I expect to hear a lot of discussion about it — mostly, from frustrated data stakeholders asking “What are the options outside of Hadoop?” and “What do I do after Hadoop?”

5. Security and privacy: not here

In the outside world, there is almost no discussion of data that doesn’t touch on privacy and security. At Strata this year, however, I expect privacy and security issues to be raised less in the context of data itself. This is interesting because in Hadoop’s heyday, there were endless discussions of obfuscation, data lineages, and other security considerations. Today, because of the shift to the cloud, data security is perceived in the data and analytics community as a concern for big tech and infrastructure companies — not their direct responsibility and not a pressing conference issue.

The bottom line

Any time thousands of data scientists and business analysts meet, things inevitably get interesting. Yet this year, we’re talking the equivalent of a shipbuilders’ conference following the sinking of the Titanic. Hadoop’s demise and the rapidly evolving BI landscape will ensure that the quest for wisdom at this year’s Strata will be a uniquely engaging one.

David Richards is co-founder and CEO of WANdisco.