Esther Perez is a systems engineer and database administrator.
I’ve been a systems engineer and database administrator for fifteen years now, and it’s fair to say that there have been more than a few large-scale shifts in information technology in that time. Perhaps no technology trend has driven a greater amount of change in that period, though, than the rise of big data.
Big data, when harnessed properly, is undoubtedly a good thing. It enables people and organizations alike to make more accurate, data-driven decisions based on past and expected behavior. While organizations have been analyzing large data sets for better business and operational insights for some time now, it wasn’t until forward-thinking companies like Facebook and Google began building custom databases that were specifically intended for larger data sets that the big data movement really took off.
The big data ecosystem is comprised of a large pool of modern databases and software packages, all of which require specialized training or certification to be harnessed properly. With this in mind, it’s no surprise that the Harvard Business Review recently referred to the data scientist, whose primary responsibility is to analyze complex data sets with these tools, as the “sexiest job of the 21st century.”
Moreover, all signs point to the continued growth of storage and big data analytics as the Moore’s Law of the next decade.
As exciting as this is for engineers, it also poses new challenges. For one, while organizations across the board have largely acknowledged that they should leverage the increasing volumes of data they’re generating and storing, there’s still a considerable amount of confusion around how exactly to do that. According to a March 2013 study from European research firm Interxion, 69 percent of European organizations view big data as a moderate or significant business challenge.
Additionally, most engineers, myself included, were trained before the rise of the big data ecosystem. There were no courses in our respective engineering schools on Hadoop, and we didn’t receive specialized training on HBase or Hive. While there’s a growing group of recent engineering graduates that had the luxury of learning these skills in university, the majority of engineers out there today still predate the rise of big data.
In an effort to attract top engineering students, a growing number of high-profile universities – Harvard, MIT and Columbia among them – have begun to incorporate big data and data science-centric majors. While this does in theory make it easier for mid-career engineering professionals to gain the requisite skills needed to stay competitive in a fast-changing field, it’s not always so simple.
Taking the necessary time and resources to pause your career and go back to school for one or two years is an extremely risky professional move for most; it’s not as if your employer is simply going to wait two years for you to complete a program and hand your job back once you have a degree in hand.
There is an alternative to this approach, however. A growing number of online learning platforms aimed at providing technical training and certification have appeared in recent years. Some notable examples include Coursera, Khan Academy and Big Data University, all of which are free of charge (a stark contrast to the wave of for-profit “universities” that had previously dominated the online education landscape).
These platforms are an extremely useful medium for mid-career professionals who are looking for a cost- and time-effective means by which they can gain necessary training for big data tools without sacrificing their existing jobs. Given the fragile economic conditions that exist in most of the world today, this is an especially invaluable feature.
In my case, I have enrolled in Big Data University, where I recently completed a free course on integrating Jaspersoft’s reporting tool with Hadoop to effectively display large data sets in a manner that allows non-engineers to easily digest and understand it. I completed the course and gained the subsequent certification in just two weeks, and am now in the process of enrolling in additional courses there.
I live in Spain, where the economy is particularly dire. While I’m finding myself tasked with working with big data more and more on a daily basis, the idea of walking away from my current job and investing money in another degree is unthinkable. With the highest unemployment rate in the EU, job competition can be cutthroat to say the least and high level education is the best way break out from that trap.
The rise of online learning platforms offers great potential, not only for engineers looking to stay ahead of the big data curve, but for any professional interested in pursuing new skills mid-career, without sacrificing their job and going back to university full time.
VB's research team is studying web-personalization... Chime in here, and we’ll share the results.