Head over to our on-demand library to view sessions from VB Transform 2023. Register Here

ss dog on computer

From healthcare and financial services, to government and academia, “big data” has gotten unprecedented attention as industries across the board struggle to answer a key question about their data: what does it all mean?

Now, more than ever, companies are seeking out talent with deep analytical skills who can find the value in their data. And while machine-based learning algorithms and advanced math is a starting point for breaking down data sets, finding meaningful relationships and connections in data points requires human skill and intuition to extract the most value. Enter the latest prized commodity — data scientists.

Demand for data scientists is heating up

Demand for data scientists is sharply on the rise. The U.S. alone will need 140,000 to 190,000 people with deep analytical skills by 2018 just to keep up with the pace of innovation, according to the McKinsey Institute. Though it’s clear that companies are hungry for this kind of talent, there simply aren’t enough data scientists to go around.

We’ve already begun to see academia capitalize on this opportunity, like the data science and statistics initiative that New York University just launched. While a person can certainly get a leg up by taking these courses, data science doesn’t lend itself to a defined career path. Granted, there are real technical requirements needed for a successful career in data science, but knowing how to manipulate data is arguably one of the most critical skills.

Algorithms will only get you so far

The important part of the term data scientist isn’t “data”, but “scientist.” Scientists work to find results that are correct, compelling, and convincing. That’s what data scientists do, too — they just do it with bits. They need to be knowledgeable about machine learning and statistics, but beyond analytical talent and computational ability, they must be observant and persuasive.

There’s no doubt that computer algorithms are getting more powerful. Already, they are being used to predict outcomes like customer retention, average patient hospital visits, and whether or not an individual will default on a loan. However, computers have their limits. As one New York Times columnist recently observed, “capable as these machines are, they are not always up to deciphering the ambiguity of human language and the mystery of reasoning.”

Computers can’t recognize context or nuance, but people can. Whereas a machine will do exactly what you tell it to do — no more and no less — the role of data scientists is more complex. It’s the data scientist’s job to raise objections and to question any result that the machine yields. That is the real value that they bring to the table. This kind of creative approach to analyzing data is an art itself. And when you marry science (machine learning models) with art (human inference), you’ll get more accurate relationships and conclusions you can trust.

What to look for in a data scientist

I work with a team of several modelers and, interestingly, no one on our team has a core background in machine learning. We have team members with degrees in physics, psychology, and even pure math; most, but not all have doctorates. That’s because data science is a performance-based field: a person’s creative approach to a problem is the key skill, more important than simply another PhD. That may be counterintuitive to those making the hiring decisions, who might typically assess a candidate based on paper, and it can certainly complicate the hiring process, but it’s a critical fact.

Whether a company is looking to build a more robust internal data science team or to outsource its analytical talent, where’s the best place to look? Though you might be tempted, I advise that you avoid looking through the traditional channels for fresh hires. The computer science major from that Ivy League school may look good on paper, but there are also a lot of high-performing computer science majors that don’t necessarily attend the best universities. I guarantee that you’ll end up with a more well-rounded person and end up paying much less for someone with equal, if not greater, talent.

And if you’re going to cherry pick candidates working in academia, you’ll want to grow your team with a biologist or psychologist-type — someone who is not only familiar working with computers, but also with running data tests. In some cases, enterprises might already have the talent at their fingertips — they just haven’t realized it yet. I’ve found that forecasters and business analysts are often equally good working with data sets, but they’re often underutilized internally.

Capitalize on your data scientists

It used to be that company executives made business decisions based on their own intuition and hunches. Today, with the help of data scientists, these decisions are now being made on the basis of data.

It’s not that big data is new — a treasure trove of database information has been lying untouched for years. Only now, companies recognize the importance of data. As data science continues to evolve into a highly prized industry, the most valuable thing that enterprises can do is to capitalize on your analytical talent and do it before the competition.

John Merrill is a software engineer and modeler at ZestFinance, a Los Angeles-based financial services technology company that uses big data to help make better credit underwriting decisions in order to provide credit alternatives to the underbanked.

Dog on computer via Annette Shaff/Shutterstock

VentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Discover our Briefings.