REDWOOD CITY, Calif. — Do you design by data, or design by your own intuition? Or somewhere in between?
“When we think about what data can do, it’s important to help people understand the world around them,” Gourley said. “If you’re building systems to make changes in the world, understanding is really core to that.”
In terms of algorithms versus intuition, he continued, “Where does big data leave us as humans? Are we out of the game?
“Biological limitations mean we can’t think quickly enough … but we also know that if you look at a system like weather prediction, we can track data from the last few years, and computers can play a massive role in prediction. But meteorologists adjust the outcomes of the models and give it a 16 percent lift by interacting with that model.”
The conclusion Gourley has come to, he said, is, “Humans and machines perform better than machines alone. … Big data can’t predict everything.”
When it comes to chess, Twitter, or world peace, Gourley said it’s “naive” to imagine that algorithms don’t have any limitations. For example, there’s the issue of data quality.
“Data is messy. You start to understand that the information out there is more than any organization can collect on its own. … There’s huge amounts of information owned by publishers hidden behind paywalls. But if you make all data free, who’s producing it?”
Case in point: Wikipedia.
On the other side, human biases also get in the way of complex problem-solving, especially for topics like geopolitics or casualties of war.
Ultimately, he concluded, practice with arranging that balance will lead to near-perfection.
“The technology we create will modify us. And we have to be okay with that. But we should design it to change us in a way that’s beneficial and will make us smarter, designing it with humans in mind. How do you project this data in a way that can be comprehended?”
Stay tuned for more from the Data Science Summit today in Redwood City, Calif.
Quid is radically changing how people interact with information. The Quid Intelligence Platform combines search, premium data, and high-performance algorithms to create visualizations of mar... All Quid news »
Richard Snee is chief marketing officer at Pivotal. He is also cofounder and advisor at Data Science Central, an online community for Big Data practitioners. Prior to Pivotal, Snee served as... All Richard Snee news »
Originally from New Zealand and am now based in San Francisco where I split my time between Mathematical research and my venture backed startup Quid. I have a PhD in physics from Oxford wher... All Sean Gourley news »