This article was contributed by Oliver Schabenberger, chief innovation officer at Singlestore.

Microsoft CEO Satya Nadella coined the term tech intensity, a combination of technology adoption and technology creation. Companies can accelerate their growth by first adopting best-in-class technology and then building their own unique digital capabilities.

Over the past decades, technology innovation has followed a familiar pattern toward digital transformation in nearly every industry or application area. Innovation shifts from industrial technology (machines, manufacturing) to computing technology (hardware) to data technology (software). Connecting has evolved from building roads and railroad tracks to wiring between computers to software-defined networking. Automating intelligence has evolved from industrial machines to replace muscle power to translating known logic into machine instructions (e.g., tax preparation software) to modern AI systems that program their logic based on data (e.g., natural language interaction).

Even computer science as a discipline experienced this transformation when it shifted its focus from computing to data about 20 years ago. The computer science-driven approach to data brought us the modern incarnations of machine learning and data science. 

The shift towards data technologies does not eliminate the other stages of technology innovation. We still use roads. Underneath a software-defined network, there are wired computers somewhere. Computerized knowledge systems still have their place — who would want their taxes done by a neural network trained on last year’s returns?

But as the world transforms digitally and turns into data, data-driven technologies are a logical consequence. The increase in tech intensity we experience today is an increase in data intensity.

Data intensity

In physics, intensity is the magnitude of a quantity per unit. For example, sound intensity is the power transferred by sound waves per unit area. More colloquially, intensity is understood as a high degree of strength or force. Both the colloquial and the theoretical definition of intensity are useful in our context, although we will not attempt a mathematical formula of data intensity. 

Data intensity is about the attributes and properties of the data such as volume, velocity, types, structure, and how you transfer the energy in the data into value.

In his book Designing Data-Intensive Applications, Martin Kleppmann distinguishes data-intensive from compute-intensive applications, depending on the nature of the primary constraints on the application. In compute-intensive applications, you worry about CPU, memory, storage, networking, and infrastructure for computation. In data-intensive applications, the data becomes the primary challenge and concern. 

This shift follows the familiar pattern towards data technologies. The underlying compute infrastructure is still essential, but automated provisioning and deployment, infrastructure as code, and auto-scaling of resources ease computing concerns. When you worry about auto-scaling the application database, adding real-time text search capabilities to a mobile app, adding recommendations based on click-stream data, or managing data privacy across cloud regions, then your application has become more data-intensive.

The concept of the data intensity of applications extends to data intensity in organizations. The data intensity of an organization increases as it manages a greater diversity of data (e.g., by volume, type, speed), becomes more data literate, adopts more data-driven technologies (e.g., data integration, data flows, no-code ELT), and builds its unique data-driven content (e.g., predictive models).

A good thing

Increased data intensity should be a good thing. As focus shifts from operating data centers to being data-centered, the rate of innovation should increase. An increase in data literacy should result in better decisions. The software-defined technologies should make processes programmable, eliminate risk, and make the organization more adaptive. Building your own predictive models should increase differentiation and enable a better customer experience through personalization.

Alas, that is not the experience in many organizations. Instead of focusing on how to get the most from data, the challenges surrounding the data create massive bottlenecks. Rather than fuel digital transformation, data intensity seems to choke it. 

    When data intensity leads to complexity and friction, the outcomes tend to be negative. People, processes, and technology adapted to one level of data intensity might not be able to cope with the next level of intensity: when the number of users grows tenfold, or the data volumes triple, or predictions are required where descriptive statistics are computed today. 

    Data intensity becomes a surrogate measure for digital transformation, with complexity it is a measure for digital maturity and resilience. In the coming years, many organizations will have objectives, key results and KPIs tied to data intensity to capture that maturity level.

    Oliver Schabenberger is chief innovation officer at SingleStore.



    Welcome to the VentureBeat community!

    Our guest posting program is where technical experts share insights and provide neutral, non-vested deep dives on AI, data infrastructure, cybersecurity and other cutting-edge technologies shaping the future of enterprise.

    Read more from our guest post program — and check out our guidelines if you’re interested in contributing an article of your own!