And like in all big rushes, the companies providing the picks and shovels can make a killing.

Lately, a number of enterprising startups that sell big data analysis tools to large companies are seeing explosive revenue growth — the latest sign the big data gold rush is fully on.

One of them is Datameer, a Silicon Valley (San Mateo, Calif.) company that offers a tool to help other companies analyze and visualize their big data quickly and easily — by providing it in a spreadsheet-like user interface that most employees can understand.

Datameer has tripled revenues over the past 12 months and looks on track to hit a $10 million run rate by early Q3, according to back-of-the-envelope math. Datameer CEO Stefan Groschupf says the company will reach 100 enterprise contracts by that time, and the average contract is $100,000 in value. The company offers annual software subscriptions, including an enterprise version can cost $100,000 or more, depending on data throughput.

Groschupf says many of his customers like to keep the details of their contracts confidential. But if you check Datameer’s web site and other announcements, you’ll see it serves companies like Sears and Visa, as well as emerging companies like hot gaming firms Kabam and Kixeye. Datameer says it also has four of the five largest global banks as paying customers as well as three of the four largest credit card companies and the U.S. government, though Groschupf is mum on specifics, citing confidentiality agreements.

Datameer’s traction comes as news emerges about other big data companies with similar growth. GoodData, a San Francisco company that provides business intelligence from data, said it too has tripled revenues over the past year. GoodData has been around since 2007. Datameer was founded in 2009. The coinciding explosion of growth this year suggests we’re seeing the “hockey stick” growth part of the big data cycle.

Datameer has raised $17.8 million from firms like Kleiner Perkins and Redpoint, but it may raise an expansion round to ensure it can grab more market share even as other competitors pile in. It can reach profitability based on the money it has already raised, said Groschupf.

Datameer’s value comes in offering an easy-to-use application layer on top of Hadoop, the popular open source software framework that allows companies to store and organize data on the fly. While companies can try to hire hard-to-find and expensive Hadoop experts to analyze the data themselves, Datameer’s app layer dumbs things down. Just about any manager in any unit can integrate, analyze, and visualize any data for their own needs. Does an HR manager want to track the performance of sales executives in new ways? Well, they can query that.

The advantage of the Hadoop-spreadsheet approach is that it allows companies to interact directly with new or existing data sources, without actually changing the underlying data. And it lets them run new types of queries on-the-fly. This contrasts to the cumbersome technology that has dominated until now, where data queries are constrained by slow and expensive data manipulation process: Generally, ETL (extract transform load) technology is used to feed data into enterprise warehouses for subsequent manipulation by business intelligence apps. This process can take up to 18 months to set up, and requires preconceived data modeling. In other words, if your intended queries aren’t thought up beforehand, you’re out of luck. No peering into your data with new sorts of queries.

Datameer’s CEO Stefan Groschupf recently shared evidence of his company’s traction during a panel discussion I moderated at Interop Tuesday (see a good summary here).

Here are five interesting ways Datameer, or services like it, are creating value:

Making money

Datameer helped a large anti-virus software company double the conversion rate of some of its marketing campaigns, netting it more than an additional $20 million in revenue within six months. Before using Datameer, the company had sunk more than $1 million into advertising campaigns that weren’t converting very well: It had used Google ads and tried to market through weblogs, its download logs, and sales accounts. Datameer helped it optimize across all of these areas, with no additional investment in infrastructure or advertising outlays. Similarly, Datameer is helping fast-growing gaming companies like Kabam and Kixeye respond more dynamically to data collected by their users, giving them a leg up on more mature companies like Zynga, which isn’t using Hadoop and is reliant on the more traditional, less flexible ETL/warehouse technology. By encouraging Kabam and Kixeye game developers to write to log files that can be instantly analyzed, Datameer can run 1,000 reports a day on them, checking to see say, what color — red or blue — is best used on game buttons for enticing gamers to click. Similarly, hundreds of other variables can be tracked on the fly, about how gamers are clicking, buying virtual goods, and then leaving.

Identify fraud

Datameer worked with a credit card company to help it save billions by identifying fraud before it happened. Groschupf wouldn’t identify his customer. However, the WSJ recently reported how Visa has used big data to save “billions” by identifying fraud that could have cost it $2 billion; it mentions Visa is using Hadoop but didn’t mention the app running on top of it. The technology allows credit card companies to view data at the individual merchant terminal level and across hundreds of attributes from average authorization volumes to the frequency of purchases that turn out to be fraudulent.

Prediction

Datameer helped an unnamed “hardware company” analyze server usage among its customers to predict when those customers will need a new server. Instead of waiting for the customer to realize servers are at full capacity, Datameer’s data knows capacity will be hit in, say, two weeks, and ships a new server to the customer before they realize they need it. That has helped the hardware company improve sales by more than $100 million “over a few years,” according to Groschupf. Datameer can perform a range of other needs for customers based on trends. One example is price optimization. While online retailer Amazon is using price optimization techniques to update prices up to 12 times a day, traditional retailers have typically optimized pricing once every 12 weeks. Datameer recently helped Sears shorten that time to three days.

Matching people

Datameer’s principal competition is not other providers, says Groschupf, but when companies decide to do the job themselves. Relationship site eHarmony, for example, was an early adopter of Hadoop, and claims this has helped eHarmony allow 550,000 U.S. marriages to happen. eHarmony matches people by running Hadoop every night on the questionnaires filled out by its customer base.

Another big step for the big data industry occurred late last year, when Datameer announced an OEM deal with online human resource software vendor Workday to embed the Datameer analytics platform as part of their big data analytics application for customers. “This marks one of the first deals to integrate Hadoop analytics into the enterprise packaged application ecosystem,” said Tony Baer, an analyst at research firm Ovum at the time. The firm called it a “key milestone in the process for big data securing mainstream enterprise acceptance.”

---

HealthBeat 2013 is a new conference showcasing how technology is transforming health care. We'll explore how IT is driving out inefficiencies on the hospital, practice, and patient levels. Check out full event details here, and register here.

---

“Big data” startup Concurrent has raised $4 million in its first round of funding with a goal to help app developers use Apache Hadoop to build more robust data-intensive applications.

San Franciso-based Concurrent was founded in 2008 and offers a popular open-source product called Cascading, which it claims is the “most widely used and deployed application framework” for building apps with Hadoop. Companies including Twitter, eBay, Etsy, FlightCaster, iCrossing, Razorfish, Trulia, and TeleNav use Cascading.

Today’s investment was led by True Ventures and Rembrandt Venture Partners. Combined with seed funding, Concurrent has raised about $5 million in total.

Concurrent also used today’s announcement to reveal that it has appointed Gary Nakamura as its new CEO. Before Concurrent, Nakamura served as SVP and general manager of Terracotta, which was acquired by Software AG in May 2011.

“Our investors’ confidence in Concurrent validates our strategy and the pioneering work we have done since the launch of Cascading in 2008,” Nakamura said in a statement. “This new investment will enable us to build on this foundation and accelerate our growth as we work to deliver our commercial product and grow the core team. I am excited to be joining Concurrent at such a pivotal time and look forward to driving the company’s continued growth and development.”

This is a guest post by Walt Maguire, Analytics Director at ParAccel

The emphasis on “big data” has grown mightily over the last year, as more companies strive to draw useful intelligence out of increasingly massive data volumes from web clickstreams, sensor data, social media data and other large datasets.

One technology approach has dominated the discussion: MapReduce. MapReduce is open-source technology used for distributed programming, and its current incarnation “Hadoop” (named for its inventor’s son’s stuffed elephant), has been trumpeted as the new solution on the scene, the silver bullet for getting value from big data.

But while MapReduce and Hadoop are interesting and useful, the approach is nothing new, nor a panacea. While often cost-effective for inexpensive data storage and lightweight data processing, running analytics on Hadoop data has been challenging. Early adopters report that analytics in Hadoop are very slow to process — a big problem for analyzing giant data sets — and complex to write, due to not supporting SQL (structured query language, the lingua franca of analysts.)

Newsflash: Other technologies that solve many of the same problems have existed for decades, namely Massively Parallel Processing (MPP) databases, which are known for speedy processing of analytics and robust SQL support.

However, the MPP and Hadoop approaches are not mutually exclusive. Hadoop and MPP databases are increasingly used together by forward-thinking companies for a complete big data infrastructure that is cost-effective and leverages the best of both technologies.

Let’s compare the two approaches and look at a few specific examples of how they can be combined.

Map/Reduce and Hadoop evolution

At the heart of MapReduce are two functions called, unsurprisingly, Map and Reduce. A Map function’s role in life is to take some input data such as a list of words, apply some function and then map those inputs to output data. A Reduce function will take the outputs from a Map, and apply a function to reduce the input data into usable output data.

In the world of big data, divide and conquer is a must if we’re to cope with the data volumes generated today.

Example of a Map/Reduce function:

The initial driver behind the development of MapReduce was a paradigm shift in computer programming during the 1990s towards an approach called “functional programming.” Not long after it was first used at Google to speed up its indexing of the World Wide Web in 2004, the open source MapReduce platform, Hadoop, was developed. Hadoop delivered a reasonably complete way to develop distributed MapReduce programs. It had numerous gaps, but for those analyzing 10,000x as much data as they were five years ago, it helped.

The recent uptake of Hadoop has been driven in part by necessity. With the exponential growth of the Internet, machine data and the trend toward “saving everything,” organizations have more data than ever before, much of it in unstructured forms. So an innovation first created as a programming technique has been pressed into service as a specialized platform for distributed data processing. While it’s good at functions traditionally performed by ETL tools, it’s not as good at providing fast answers to questions.

Organizations risk finding themselves with a large repository of data in Hadoop that they can’t analyze very well.

Massively Parallel Processing (MPP) evolution

For many of the tasks necessary in processing and analyzing big data today, the Massively Parallel Processing (MPP) database is better. MPP databases also split up complex, large volume jobs into units processed across multiple nodes. While they don’t act exactly like MapReduce, they accomplish many of the same things, and are far better at some things.

MPP databases provide things taken for granted by database users for decades such as ACID compliance — meaning you will get predictable answers to questions. This isn’t enforced in Hadoop. Also, MPP databases include cost-based optimizers and monitor the distribution of data within the system; and as a result, they are generally an order of magnitude more efficient than Hadoop. So you can do things ten times more quickly, or do the same thing with one-tenth the infrastructure.

MPP databases do not solve every problem. For example, when the structure of incoming data is unknown or variable, an MPP database requires that this be structured at load time. So a measure of data manipulation must take place to prepare it. Also, appliance-based MPP systems can be difficult and costly to expand, whereas Hadoop is designed to run on any hardware. Software-based MPP database solutions don’t have this problem.

The following table compares and contrasts Hadoop/MapReduce with MPP databases.

Comparisons from the real world

Many firms have brought in both technologies for their big data infrastructure.

One large retailer found that Hadoop and an MPP platform are complementary. The company ingests large amounts of unstructured data and archives it at low cost with Hadoop; it then loads the data needed for analytics into the MPP platform via the vendor’s proprietary, high-speed integration module. Now, this retailer can run jobs 200x faster than its previous data warehouse, enabling more granular market basket analysis and customer segmentation. It leveraged Hadoop for low-cost storage and an analytic platform for doing the actual analysis,  cost-effectively solving a number of key problems with this combination of technologies.

This is a common model these days. Evernote, a Redwood City, Calif.-based developer of note taking and organization software, has a similar architecture, using Hadoop for low-cost data storage and processing of web application log data, combined with an MPP platform for analytics. For them, it was faster to move the data to a platform purpose-built for analytics than to try to run the analytics within Hadoop.

Evernote CTO Dave Engberg provides much more detail and a summary in the company’s Tech Blog:

Hadoop is great for cheaply storing a ton of data and performing parallel batch processing jobs in minutes instead of hours (or days)…But it’s not particularly quick for more complicated analyses that combine multiple different sets of data.…

Overall, the new infrastructure has met our goals. We can load and transform hundreds of millions of records in two hours instead of 10+, we’re generating far more (and far better) reports, and we can safely perform much more complex analyses of user trends than we could before.

In summary, there is a useful place for MapReduce and Hadoop in the big data landscape, but MPP technologies also offer significant advantages. Companies should strongly consider using both together to deliver big data infrastructures.

---

HealthBeat 2013 is a new conference showcasing how technology is transforming health care. We'll explore how IT is driving out inefficiencies on the hospital, practice, and patient levels. Check out full event details here, and register here.

---

But let’s face it: That train is in the station. What’s next?

Also see: Forrester’s top 10 mobile trends for 2013

Analyst Bryan Hopkins gave us a peek into what Forrester thinks is next, and much of it builds on those four horseman of disruptive change. “We went a level deeper in our research by examining how today’s hot technolog[ies] create platforms for future disruption,” he wrote this morning in a blog post.

Here they are, in four groups:

End user computing technologies

1. Next-generation devices and UIs
   New sensors and new user interfaces. Think Leap Motion
2. Advanced collaboration and communication
   Think social inside, like Yammer or other social-inside-the-enterprise solutions
3. Systems of engagement
   Real-time data, in everyone’s hands. Think Roambi

Sensors and remote computing technologies

1. Smart products
   Thing that can sense, react, and communicate. Think operating system for places and buildings
2. In-location positioning
   GPS and in-building location sensors
3. Machine-to-machine networks
   Background intelligence on people and things. Think ReelyActive

Process data management technologies

1. Smart process applications and semantics
   Real business processes are a lot messier than your flow charts. Smart process apps know that.
2. Advanced analytics
   Smarter, more predictive data. Think Cloudera’s Impala tool for Hadoop
3. Pervasive BI
   People need business intelligence that comes every hour, not at the end of the month
4. Process and data cloud services
   Scalable, burstable, and cheap computing capability. PaaS, BaaS, etc. 

Infrastructure and application platforms

1. Big data platforms
   Infrastructure to handle big data and high speed … and use all that data you’ve been uselessly storing
2. Breakthrough storage and compute
   Yes, hardware may still be necessary, even if you’re never going to be like Google
3. Software-defined infrastructure
   Software that dynamically routes your networking and data center capabilities
4. Cloud application frameworks
   Technologies for deploying and running distributed apps in the cloud, like, perhaps, a multi-continent-spanning database
5. New identity and trust models
   New federated trust and identity models for a changing world of jobs and careers … and maybe even killing all usernames and passwords

An interesting thought for executives:

If you want a good look at the future of end user computing technologies and sensor and remote computing devices, check winning Kickstarter and IndieGoGo campaigns in the technology and gadget categories. And for a picture of the future for the last two groups above, process data management and infrastructure and application platforms, look at Google and Facebook.

photo credit: ginnerobot via photopin cc

After three years of operating under the radar, Think Big Analytics is ready for its big debut.

The Mountain View, Calif.-based startup leverages open-source computing frameworks Hadoop and NoSQL to bring “big data” to large companies. The company may also offer consulting services, such as assisting clients in prioritizing use cases before working with a Hadoop vendor like Cloudera, GigaOm reports.

To bring its product to market, the founding team has raised a massive $3 million seed round from former Cisco executive Dan Scheinman, with participation from the venture firm WI Harper Group.

The market is full of big data and analytics startups, which are competing for customers against legacy vendors like SAP.

However, investors see massive opportunity for these startups to appeal to companies that are grappling with vast quantities of potentially valuable data. Analysts have reported that both chief marketing officers and chief information officers will carve out a slice of their budget to spend on big data technologies in the coming years.

Early customers include NetApp and Quantcast — the former company of Think Big founder Ron Bodkin. In an interview, Bodkin said inspiration hit on in spring 2010. He announced to dinner guests that he intended to quit a steady job at as the vice president of engineering at Quantcast to help companies use analytics tools.

“Our goal is to be to enterprises what Google is for consumers — the fastest response engine for ad-hoc queries,” said Bodkin. The technology is different from legacy vendors due to its emphasis on helping companies generate real value from the data, and the speed — “questions [are] answered in minutes, not hours,” he said.

---

HealthBeat 2013 is a new conference showcasing how technology is transforming health care. We'll explore how IT is driving out inefficiencies on the hospital, practice, and patient levels. Check out full event details here, and register here.

---

This is a guest post by technology executive Bruno Aziza

“Big data” is everywhere. From social media startups to New York’s Central Park, everyone seems to be deploying big data analytics these days.

Gartner, one the biggest analyst firms in the world, backs up the trend with big numbers: a recent report shows that $28 billion was spent on big data technologies this year, and over $230 billion will be spent through 2016. $230 billion is almost as much as the GDP of Portugal.

However, you’ll need big bucks to be able to deploy big data technology solutions. Most companies don’t have the IT budget, and can’t afford to hire a data scientist or data services team.

If the trend is to succeed the companies of all sizes, there are a few problems that will need to be addressed and ironed out.

Big data is too expensive!

You might have heard all about the exploits of the biggest players: Facebook stores about 100 terabytes of data about its users, and NASA streams approximately 24 terabytes every day (full disclosure: NASA is one of my company’s customers). These numbers are indeed impressive.

How much does it cost to work with so much data? Even with Amazon Redshift’s aggressive pricing, NASA would have to pay more than a $1 million for 45 days in data storage costs alone. This number is consistent with New Vantage Partners survey, which evaluates the average big data project to cost between $1 million and $10 million.

Most CIOs, according to a recent survey, responded that their budget won’t cover the cost of a big data deployment. We need to approach big data differently, and design solutions that allow smaller companies take advantage of this opportunity. The cost of storing and processing this data is simply too high.

Big data does not necessarily have to be big 

This brings us to the second reason why the big data market is flawed. Today, a big deal is made of the vendors who partner with the largest technology firms to work on petabyte-scale data. Yet, even SAP’s own research shows that 95 percent of companies only use between 0.5 terabytes to 40 terabytes of data.

The amount of data that Facebook and NASA are crunching remains the exception, not the norm. Truth is, you don’t have to be a large company to leverage your data. If you looked at range of companies in the U.S., you’ll find that there are over 50,000 that only have between 20 and 500 employees – most of which, I’d argue, are trying to solve data problems at scale.The biggest market for big data is not just with the Fortune 50, it is with the Fortune 500,000. Why do we then focus so much on the exceptional few, when the majority of companies that need help are neither Fortune 50 and do not have petabyte-scale problems?

Sometimes, I wonder what would happen if we changed the definition of big data. What if, instead of focusing of the proverbial 3 V’s (velocity, volume and variety), we tried something like this: “Big data is a subjective state that describes the situation a company finds itself in when its infrastructure can’t keep pace with its data needs.”

This definition might not be as glamorous as others, but it sure would be closer to the reality most companies are trying to grapple with today.

Bruno Aziza is Vice President of Marketing of Big Data Analytics Company SiSense.

Big data image via Bruce Rolff // Shutterstock

---

HealthBeat 2013 is a new conference showcasing how technology is transforming health care. We'll explore how IT is driving out inefficiencies on the hospital, practice, and patient levels. Check out full event details here, and register here.

---

This is a guest post by enterprise technology executive Jeff Carr

“Big data” is without a doubt the hottest trend in technology today, possibly surpassing social media, which has held the tech hype crown for years. At its broadest, the definition of big data includes any aspect of harnessing, analyzing and monetizing the massive amounts of data being generated by web and mobile based applications. The sheer scale of the data being generated dwarfs what was considered ‘large’ amounts of data as recently as ten years ago, and all indications are that this trend line will continue.

Most observers would agree that the era of big data started around 2007, when Google’s MapReduce programming framework was integrated with Apache Hadoop, an open source project founded a couple of years earlier to help developers efficiently and cheaply process large amounts of data. Used together, Hadoop and MapReduce made it faster, easier and cheaper to process and analyze massive volumes ofdata than ever before.

At this juncture companies started adopting various forms of Hadoop/MapReduce to capture and filter their data. Companies like Yahoo and later Facebook were some of the earliest to announce petabyte stores of data in Hadoop.

Rapid commercialization of the Hadoop ecosystem, however, has only occurred in the last two or three years, as the revenue opportunity began to reveal itself. As with any big trend in technology, including the RDBMS/client server, internet, and web security trends that preceded it, big data has correspondingly evolved into the technological equivalent of a gold rush. Hundreds of companies have entered the fray with hopes to quickly cash in.

The majority of these companies, which include pre-big data enterprise technology incumbents and a slew of data-focused technology startups, are positioning themselves as the suppliers to the miners of big data. Instead of picks, axes and gold pans, they supply the tools, technologies and services that will help companies monetize huge amounts of data. Needless to say, there is a lot of data to be mined and a lot of money to be made.

A slightly closer look at the big data market demonstrates an obvious, yet often overlooked, truth about where we are in the big data innovation and maturity cycle. Most big data products available today are UI’s, management tools and integration tools focused around open source projects being developed within the Apache Hadoop ecosystem, including Hive, PIG and Zookeeper. Big data revenue, on the other hand, is being driven primarily from services that help design, architect and implement big data solutions using the Hadoop ecosystem. In fact, many of the largest and fastest growing companies in big data today are pure services companies (Opera Solutions and Think Big Analytics come to mind).

I’m not suggesting that there is anything wrong with a services approach to the market. Most of these companies are providing solid value for their customers, which can translate into healthy revenue streams. It does help, however, to have some historical perspective to understand where we are in the big data innovation cycle, and what comes next.

Open source services and support as a primary revenue stream originated in the 90’s when there was a similar gold rush around the commercialization of Linux. Early companies such as VA Linux and Red Hat capitalized on this. Nearly 20 years after the open source movement started, however, there is exactly one company with “pure” open source roots that has more than $1B in annual revenue, and they achieved this in 2012. By contrast, there are many billion dollar technology companies that have innovated new IP-based solutions that monetize major technology trends.

In that sense, it’s clear that we remain in the earliest days of the big data movement. Larger companies looking to monetize their big data assets lack the expertise and know-how to do so, and they are turning to services companies to help them bridge those gaps. As the market evolves, so too will labor skills democratize and broader product innovation begin to take hold, creating less reliance on services-centric companies. To relate this back to my gold rush analogy, the early winners were people selling tools and mining expertise, but the long term winners were the people that actually found the gold!

So what’s next for big data? For any developer team that has felt the pain of building a big data infrastructure, one clear next step is simplification. The diagram below is a basic flow most companies go through to leverage big data:

Building this solution requires a small army of vendors and consultants to combine solutions and technologies in various ways to analyze and (hopefully) monetize their data. It takes months, and in some cases, years. It’s expensive. In short, it’s a pain in the butt, and the result often does not help monetize big data directly, it’s just the first step in the process.

Yet, no one can argue that this is not where the “action” is in big data today.

Simply put, the current state of big data is great for service vendors, and not always so great for big data buyers. It’s a market ripe for innovation. In the immediate future, we can expect an increasing number of product-centric companies to begin to disrupt the patchwork of services-centric solutions that currently exist.

In summary, the “secret” of Big Data is that today it suffers from a dearth of expertise, so the majority of the revenue is coming from a services centric approach combined with open source technologies. I am in no way diminishing the importance and value of open source projects like Hadoop. To the contrary, I’m a huge supporter, always have been.

What I’m pointing out is that the market will evolve beyond an open source, services-driven revenue model when companies begin developing highly disruptive technologies that solve the hardest problems of big data. While open source solutions like those from Apache may play a role in this, history indicates that the most innovation will come from companies engineering entirely new ways to solve the most difficult problems.

Jeff Carr is COO of Precog. Precog is a data science platform designed for developers and data scientists to turn data assets into data-driven features and products inside an application.

Jeff has worked in technology for 25 years with a focus on business development, market assessment, strategy and operations. For the past 11 years he has worked exclusively with early stage companies in markets including network security (Vericept, CipherTrust), VOIP (Borderware SIPassure), and Big Data (Precog).

---

HealthBeat 2013 is a new conference showcasing how technology is transforming health care. We'll explore how IT is driving out inefficiencies on the hospital, practice, and patient levels. Check out full event details here, and register here.

---

So you’ve got an idea for a “big data” application, but building it seems to be a nightmarish prospect.

A startup called Continuuity, which raised $10 million today, has struck a chord with developers who are struggling to build big data apps on top of Hadoop. Continuuity promises to make it easier by offering a suite of tools that extend across the various stages of software development from “prototype to production.”

In a nutshell, the company makes it easier to build, scale, and deploy Hadoop apps both in and outside the firewall. Continuuity’s AppFabric hosting platform is built on top of Hadoop and related technologies.

The founding team of seven engineers had previously built Hadoop and HBase (Hadoop’s NoSQL database) deployments at technology giants like Yahoo, Microsoft, and Facebook.

This is one of several young companies that claim to unlock Hadoop’s untapped potential. For instance, Platfora recently launched a product to help its users “make Hadoop amazing” by crunching raw data and turning it into actionable business insights.

Hadoop is viewed as an unruly problem child in certain traditional circles and has been criticized for failing to integrate well with traditional business intelligence (BI) applications.

“Continuuity is poised to disrupt the market by merging big data and cloud to introduce a new application development opportunity for all developers,” said Mike Dauber, Principal at Battery Ventures, the firm that took part in the funding. He praised Continuuity for its lofty mission to “become the center of gravity for the Big Data application development ecosystem.”

In addition to Battery Ventures, investors in the round included Ignition Partners, Andreessen Horowitz, Data Collective, and Amplify Partners. Battery Ventures’  Roger Lee will be joining Continuuity’s board. The Silicon Valley-based startup closed a sizable $2.5 million seed round in January and has been focused on building out the product ever since. With this infusion of capital, the company will roll out an aggressive go-to-market strategy and expand its team of developers.

---

HealthBeat 2013 is a new conference showcasing how technology is transforming health care. We'll explore how IT is driving out inefficiencies on the hospital, practice, and patient levels. Check out full event details here, and register here.

---

Platfora, the startup that made a splash at the recent Strata conference with a promise to “unleash the potential of Hadoop” has raised its second round of funding.

The San Mateo, Calif.-based company has been baking its technology for over a year. In a nutshell, it crunches raw data in Hadoop, the popular open source framework, and turns it into intelligence for business users.

Ben Werther, CEO, Platfora

At a recent dinner hosted by its investor, Andreessen Horowitz, Ben Werther, the company’s CEO told me he was astonished by the reaction when Platfora launched at Strata. When it comes to this “big data” trend that everyone is talking about, companies are navigating uncharted waters.

Data can no longer be stored in columns and rows in relational databases. To be truly useful, the most unwieldy data (whether it’s texts, emails, or records) will need to be processed and visualized, so it can inform business strategy. Plenty of companies claim to offer a solution, but are still in the nascent stages of development.

According to Werther, Hadoop is the answer, but only a small cadre of developers use it. “We want to make it possible for everyday business users to interact, analyze, and drive Hadoop,” he said in a recent interview.

Startups like Datameer and Karmasphere also claim to bring Hadoop straight to your desktop; but for Platfora, speed of deployment is a competitive edge. “People can visualize and ask questions about data within hours,” said Werther. “There is no six-month cycle time to make Hadoop amazing.”

Customers (10 signed up for the beta) can view the data on a web-based interface. Platfora claims to be the first big data startup to use HTML5 canvas technology, which makes it easy to use and enables collaborative data analysis on any device.

Battery Ventures led this round of funding, with participation from existing investors Andreessen Horowitz and Sutter Hill Ventures.  In-Q-Tel, the CIA’s venture arm is also an investor, but is not specified in this round.

“We believe that Platfora represents the future of business analytics. With the emergence of Hadoop, legacy ETL, data warehouses, and BI solutions no longer make sense,” said Mike Dauber, principal at Battery Ventures in a statement.  “The early demand for Platfora speaks to the company’s vision and understanding of how businesses need and want to access and analyze data.”

Roger Lee of Battery Ventures will join the company’s board.

Top image courtesy of Shutterstock

---

HealthBeat 2013 is a new conference showcasing how technology is transforming health care. We'll explore how IT is driving out inefficiencies on the hospital, practice, and patient levels. Check out full event details here, and register here.

---

In traditional circles, Hadoop is viewed as a bright but unruly problem child.

Indeed, it is still in the nascent stages of development. However the scores of “big data” startups that leverage Hadoop will tell you that it is here to stay.

Cloudera, the venture-backed startup that ushered the mainstream deployment of Hadoop, has unveiled a new technology at the Hadoop World, the data-focused conference in New York.

Its new product, known as “Impala”, addresses many of the concerns that large enterprises still have about Hadoop, namely that it does not integrate well with traditional business intelligence applications.

“We have heard this criticism,” said Charles Zedlewski, Cloudera’s VP of Product in a phone interview with VentureBeat. “That’s why we decided to do something about it,” he said.

Impala enables its users to store vast volumes of unwieldy data and run queries in HBase, Hadoop’s NoSQL database. What’s interesting is that it is built to maximise speed: it runs on top of Hadoop storage, but speaks to SQL and works with pre-existing drivers.

“If you ask questions, you will get answers ten times faster,” Zedlewski explained.

These answers or “insights” can mitigate risk and inform strategy for a company’s sales, marketing and business development teams.

Travel booking site Expedia is one of the companies that is using Cloudera’s new product to manage four pedabytes of company data.

“We are able to work on one single platform for big data rather than many disparate systems for archiving, ETL and analytics,” said Jeff Prather, Director of Global Business Intelligence and Data Warehousing Platforms at Expedia.

“This evolution of Hadoop has enabled us to reduce our latency by 50 percent and produce a new real business insight service not previously viable,” said Prather in a statement.

Cloudera burst onto the scene four years ago. It was among the first to capitalize on the growing popularity of the open source Hadoop framework. (Fun fact: Hadoop was invented by independent programmer Doug Cutting, and shortly thereafter was brought to Yahoo. It was named after his son’s top elephant.)

“The reason why people are attracted to it [Hadoop] is because it’s scalable, extremely flexible, easy to get started with, and inexpensive compared to what else is out there,” said Zedlewski.

Cloudera’s founders used Hadoop for the purposes of data analysis and business intelligence. For developers, Hadoop is free to download under the Apache software license, but the company sells a broad spectrum of technical support services.

The Silicon Valley-based company has signed on business intelligence partners to test the product, including Tableau and QlikTech. (Learn more in our inaugural list of top “big data” companies). It will bring in revenue through this new product by charging customers for support services.

Cloudera has 400 employees and has over $76 million of venture funding in the bank. Cloudera’s chief scientist, Jeff Hammerbacher, conceived and led Facebook’s data team.

---

HealthBeat 2013 is a new conference showcasing how technology is transforming health care. We'll explore how IT is driving out inefficiencies on the hospital, practice, and patient levels. Check out full event details here, and register here.

---

Skytree, a “big data” company that came out of stealth mode in February, has filed for additional funding from existing investors, Javelin Venture Partners.

The startup has a tool to bring machine learning techniques to a mainstream market and has raised $1 million of a $2 million round, according to the security and exchange commission (SEC).

Machine learning lets systems get smarter over time, which helps data scientists and developers uncover patterns in a chaotic mess of data. Skytree’s eponymous product is known as “Skytree Server.”

The company has hired a team of machine-learning Ph.D.s to design the server and integrate with existing data stores, including Hadoop. The hope is that it will process and query data 10 times faster than is currently possible.

The startup is interesting to investors as it has taken on a meaty technical challenge. Machine learning is a particularly complex approach to big data that has not been fully leveraged. Only a handful of tech companies, government agencies, and financial services firms have integrated machine learning.

The San Jose-based company raised its series A round from Javelin Venture Partners in February, 2012. Read the SEC filing.

Image via Shutterstock

NEW YORK: Platfora, the Andreessen Horowitz-backed “big data” startup, unveiled the product it has been baking for over a year at the Strata conference today.

The Silicon Valley-based company publicly launched its technology that crunches raw data in Hadoop, the open source framework, and turns it into intelligence for business users.

For years, data was contained in structured columns and rows, and stored in relational databases. Today, companies are dealing with large volumes of unwieldy data. Every big data startup worth its salt will tell you that once it’s processed, this unstructured data contains potentially valuable insights.

Ben Werther, Platfora’s CEO

In a phone interview, Ben Werther, Platfora’s CEO, made a similar case: “Businesses want to know if they can derive business intelligence from Hadoop [and] the answer is yes.” Werther told me that the startup’s first customers (there are 10 companies signed on to the beta) are struggling to manage petabytes of corporate data in the form of texts, emails, reports and so on.

He explained that Hadoop is a feasible solution but can only really be of use to a small cadre of developers. ”We want to make it possible for everyday business users to interact, analyse, and drive Hadoop,” he said.

Werther says it’s the speed of deployment that makes Platfora unique. “People can visualize and ask questions about data within hours,” he said. “There is no six month cycle time to make Hadoop amazing.” In addition, there is an analytics component; users can visualize the data on an HTML interface.

Werther claims that the closest competitor is the “status quo” — the companies that still rely on traditional data warehouses. However, there are a number of other startups that claim to bring Hadoop straight to your desktop, including Datameer and Karmasphere.

Platfora has over $12 million in first-round venture funding in the bank from In-Q-Tel, the CIA’s venture wing, Andreessen Horowitz, and Sutter Hill Ventures.

---

HealthBeat 2013 is a new conference showcasing how technology is transforming health care. We'll explore how IT is driving out inefficiencies on the hospital, practice, and patient levels. Check out full event details here, and register here.

---

RainStor is the second startup today to announce a multimillion-dollar fundraise in the “big data” market (Trifacta, which just raised $4.3M, is the other).

San Francisco-based RainStor pulled in $12 million today for relational database products that allow new ways to store large amounts of data.

Companies of all sizes are increasingly looking to exploit big data — vast stores of both unstructured (texts, emails, reports, and so on) and structured data – for business insights. As a result, there is no shortage in venture capital funding for big data startups that have the security and scalability required to sell their product to large enterprises.

RainStor runs on the Hadoop Distributed File System, which means it can scale out its data storage and analytics capabilities, potentially to multi-perabyte data volumes. According to the company, its technology is easier to use than most and doesn’t require a staff member to have specialized database administrator (DBA) skills.

They call it “NewSQL,” as it’s a novel approach to a decades-old technology and is designed to run on-premise or in the cloud. What’s interesting here is that most startups in this space have heralded the death of the relational database.

The startup will use the funding to step up its sales and marketing efforts. To succeed, it will need to win customers from competitors in the big data space, which include traditional players like EMC, and startups such as CopperEye.

RainStor’s third round was led by Credit Suisse and Rogers Venture Partners, with participation by existing investors Doughty Hanson Technology Ventures, Storm Ventures and The Dow Chemical Company. This most recent round brings the company’s total funding to $23.5 million.

Photo via Shutterstock

SuccessFactors, a company that makes cloud-based human capital management software, was acquired by SAP for a whopping $3.4 billion last December. In February, Oracle bought Taleo, which develops web-based candidate recruiting software, for $1.9 billion. Even tech giant Microsoft is cashing in on the enterprise startup scene, as is evidenced by its acquisition of employee social network service Yammer for $1.2 billion in cash this year.

These billion-dollar acquisitions make sense when we consider the big money enterprise tech companies and startups are pulling in from venture capitalists. In fact, the enterprise software industry is expected to be a $4.5 billion market by 2016, a huge increase from $800 million in 2011.

Businesses all over the country are using enterprise software to break down barriers in communication. Cloud computing, software as a service (SaaS), and the overall rise of the tech industry are helping enterprise tech companies flourish. Their programs are becoming standard in the workplace — one study found 77 percent of enterprises now use the cloud in some form, and 16 percent of applications used today are SaaS — and they’re changing the way we do business.

What are the next companies to make the list? Below are six stars in enterprise tech likely set to lead the industry:

1. Kaltura

Kaltura, the open source online video platform, offers enterprises a wide array of video-related software services, including comprehensive video player, video distribution, video editing, video sharing, and video communication tools.

Companies can incorporate the tools for a wide array of uses, whether for marketing, corporate communications, media management, or even internal training. Tech giant Groupon recently announced it will use Kaltura’s software for internal communication with its 12,000 employees.

Founded in 2006, Kaltura has more than 150,000 customers and is currently increasing its presence in Europe, making the company a global leader in video software. Kaltura has raised more than $40 million venture capital and is headquartered in New York City.

2. Hadoop

Hadoop is an open source software framework that allows companies to process large amounts of data. Named after the founder’s son’s toy elephant, the company’s open-source software is used by the likes of Facebook, Yahoo, eBay, and LinkedIn.

Processing “big data” has traditionally been a problem for businesses because the various formats involved make large amounts of data difficult to scale. Hadoop allows companies to gain value and insight from their data by giving them a way to easily and inexpensively process it. Because Hadoop allows engineers and developers to work together to solve the big data problem, it’s even been referred to as not just a technology, but a movement, meaning this is a platform that could forever impact the enterprise tech industry.

3. GitHub

GitHub offers a social network for programmers, allowing them to collaborate on projects and monitor development. The git-based site, which has 1.7 million users, allows programmers to see the full repository so they can view the history and track revisions. Businesses like Facebook, Twitter, Microsoft, Mozilla, and LinkedIn use the site, which was founded in 2008 and is now valued at around $750 million.

Perhaps most importantly, GitHub recently received $100 million from Andreessen Horowitz – the venture capital firm’s largest investment to date. The funding will help GitHub to determine a marketing strategy to bring the service to businesses, so we’re sure to be seeing this software development company growing in the future.

4. Box

Box began as a college project in 2005 and has grown exponentially since. It raised $284 million in venture funding and boasts 80 percent of the Fortune 500 companies as customers. The cloud collaboration software is used by more than 11 million people and allows users to share, organize, and store files on a web-based platform.

The company hopes to use its startup capital to build tools for large businesses and increase its international presence. Currently, about 20 percent of the company’s user base is international, but Box hopes to increase this number to 40 percent. With big aspirations and plenty of capital under its belt, Box is certainly becoming one of the next billion-dollar tech companies to watch.

5. Zuora

Zuora provides a subscription billing and payment service that alleviates the need for businesses to create their own customer billing platforms. Businesses can adopt Zuora’s subscription-based service, which is cloud-based, to monitor customer subscriptions and payments.

Founded in 2007, the company has raised $77.5 million in funding. As of fall 2011, the company was valued between $300 and $500 million, but that’s sure to have increased over the past year. As Zuora expands in Europe, it’s clear this company will continue to make great strides in enterprise technology.

6. Marketo

Founded in 2006, Marketo raised an impressive $108 million in startup capital. The company sells marketing automation and sales effectiveness software to businesses of all sizes, so they can sell and market with increased efficiency. The company also focuses on social technologies, and helps companies to capture sharing information and analytics to drive social marketing campaigns.

In the first half of 2012, Marketo added several Fortune 500 companies to its customer base as well as companies like Panasonic, Boston Children’s Hospital, GigaOm, and SurveyMonkey. The company plans to unveil an initial public offering after the November election, meaning Marketo is a company to keep your eye on as the year progresses.

These are just a few companies slated to forever change the way our businesses manage data and communicate. As more and more money is funneled into enterprise technology startups, growth and innovation are sure to be staples in this industry for years to come.

Zev Gotkin is an entrepreneur and founder of L’Mala, a writing firm specializing in website content development, blogs, branding and social media promotion. He can be reached at wgotkin@gmail.com.

[Top image credit: nadirco/Shutterstock]

Sept. 9 - 10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Everybody wants big data, but nobody wants to muck around in SQL queries. And while Hadoop is a hot topic, most people have no clue what it really does, let alone have the skill to use it.

That’s where business intelligence tools like Bime come in. Bime, which just launched version 4.0 of its browser-based data mining and data visualization tool, promises to make diving into enormous databases as pleasant as taking a swim in a cool city pool.

Well, almost. You still have to understand a little about database fields and pivot tables, but in a demo I saw yesterday, Bime was remarkably simple — and fast — at turning a database with more than 400 million rows into pretty bar charts, pie charts, and network visualizations. Essentially, it’s a more user-friendly front end for whatever gigantic databases you already have, and it’s one that marketing managers and sales executives can use, without a lot of handholding from the IT department’s database gurus.

The data on the back end can come from Salesforce.com, Google Apps, Google Analytics, and more traditional data repositories, such as MySQL, SQL Server, or Oracle.

Bime 4.0 adds support for Google BigQuery, a web service that lets you do queries against enormous data sets — databases with billions of records, for instance. The company also offers BimeDB, a hosted database that’s optimized for analytics, so companies can store their data with Bime as well as using it as the BI front-end.

Bime costs $60 to $240 per user, per month, depending on the options. For more details, check out this page of video demos that show what Bime does.

The company that makes Bime is called We Are Cloud, although it’s changing its name to BIME (in all caps, though I refused to spell it that way) to match its leading product. We Are Cloud has 15 employees and is based in Montpellier, France. It has raised several small rounds from angel investors in the U.S. and Europe but has not disclosed the total amount of its funding. We Are Cloud debuted in the U.S. at Demo Fall 2011.

---

Big Data and Predictive/Real-time Analytics startups: Are you looking to jumpstart development & accelerate market traction? Sign up for the SAP Startup Focus program to receive technology, support, resources and community to help you develop new applications on SAP HANA, a cutting edge database platform. Get started here, and enter promo code “VB2013″ on the form.

---

Sept. 9 - 10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Hewlett-Packard-owned cloud computing business Autonomy has released a new set of cloud-based solutions created to help businesses organize and analyze their data, the company announced today.

The new solutions are based on HP’s Converged Cloud and Autonomy IDOL 10. One of the biggest new programs is the Optimost Clickstream Analytics app that will let companies to view customer engagement and visits with greater ease.

Autonomy has also released new capabilities for leveraging IDOL technology with Hadoop deployments. Through this technology, companies will be able to combine Hadoop data with other platforms (like Java) to get the most out of their data. Vendors will also be able to embed the Autonomy IDOL 10 engine in each and every Hadoop node.

“Autonomy and Vertica offer an unparalleled platform from which to automatically understand, optimize, and act on 100 percent of an organization’s data,” said Rafiq Mohammadi, head of the Promote unit at Autonomy, in a statement. “Today’s announcement further extends this platform, by enabling organizations to get more value from their Hadoop deployments, and blend clickstream data with unstructured, human information. Autonomy Optimost Clickstream Analytics uniquely enables marketers to see and act on customer trends and sentiment—as they emerge—in order to optimize offers and campaigns, and to ultimately accelerate revenue growth.”

Since being bought by HP, Autonomy has been helping companies and marketers with a full-scale platform that delivers data solutions across many channels. Autonomy’s private cloud for businesses has now surpassed 50 petabytes in size to create “the world’s largest private cloud.”

Photo credit: IvanWalsh.com/Flickr

10gen, the startup that created the popular MongoDB open-source data application, has raised a new $42 million round of funding to improve its data-harnessing technology and build out its team.

The New York- and Palo Alto-based company generates most of its revenue by offering support, training, and consultation for MongoDB. Customers come from a wide cross-section of industries and include Foursquare, Disney, Viacom, eBay, Telefónica, and several governments across the world. The response from these customers has been tremendous, as the company has seen 50 percent quarter-over-quarter sales growth during the past five quarters.

There are quite a few similarities between 10gen and another big database startup, Cloudera, which has made a name for itself by developing and supporting open-source data platform Hadoop. However, MongoDB and Hadoop are used for different things and some customers use both data apps to help make sense of and manage their data.

“MongoDB handles real-time interactive queries, while Hadoop is used for analytics and data analysis,” 10gen President Max Schireson told VentureBeat. “MongoDB is fast and scalable on one hand and very flexible on the other.”

The major new round of funding was led by New Enterprise Associates, with participation from prior investors Sequoia Capital, Flybridge Capital, and Union
Square Ventures. The new round brings the company’s total raised to about $73 million.

“I’ve had my eye on 10gen and their amazing trajectory for several years,” said Harry Weller, general partner at New Enterprise Associates, in a statement. “They’ve solidified themselves as the dominate player in the NoSQL market and a real disrupter in the database space in general. The NoSQL market is exploding, and we see MongoDB becoming the de-facto database for both Web 2.0 companies and large enterprises.”

One other big thing the company will do with the funding is invest in new hires. The company has around 130 employees now, but Schireson said it intends to have at least 200 by the end of the year. It also plans to open an office in Sydney soon.

You can view a video outlining MongoDB below:

Photo credit: nokhoog_buchachon/Shutterstock

Sept. 9 - 10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Google’s BigQuery application has launched into general availability with an aim to help businesses crunch “big data” sets easier and cheaper than ever, the company said Tuesday.

BigQuery, as we previously detailed in November, gives companies of all sizes a powerful cloud-based tool to analyze data. Traditionally, crunching big data sets has taken more IT investment than simply spinning up and uploading data to a Software-as-a-Service (SaaS) program.

The SaaS software stands in stark contrast to open-source data software Hadoop and companies like Cloudera that help companies get a handle on Hadoop. It’s also quite different from on-premise data crunching software like HP-owned Vertica and IBM-owned Netezza.

Google product manager Ju-kay Kwek, who is overseeing the company’s big data efforts, told me companies that rely on data and business intelligence would likely prefer BigQuery over other options because it’s easier to set up and it costs less.

“On-premise options like Netezza and Vertica are fast and powerful, but they will cost you,” Kwek said. “And with Hadoop, you need more heads and you have to build out a custom Hadoop system.”

Kwek said one of Google’s largest advertising customers used BigQuery verus its own on-premise Hadoop cluster as a test to see how well it worked. BigQuery crunched the data ten times faster, showing how much faster data crunching can be when using Google’s monstrous computing power.

While in limited preview, BigQuery was free to try for 30 days, but now that the service is all the way live, companies will have to pony up. Thankfully, the prices appear fairly reasonable. Here’s the breakdown:

Let us know in the comments if your company might give BigQuery a shot for its big data needs or if you prefer Hadoop, Netezza, or Vertica.

Data image: Toria/Shutterstock

Sept. 9 - 10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Industry analysts like to refer to 2011 as “the year the cloud arrived.” But now that it’s here, what are we going to do with it?

We’ve got a few ideas.

Vendors are tripping over themselves to bolster their product lineups with cloud-hosted software and services, while customers in the public and private sectors alike are realizing the cost saving benefit of letting someone else worry about their servers and applications. And that’s not even mentioning the burgeoning consumer cloud market, where even Apple sees ample opportunity.

Despite the hype, there’s a lot of substance to the cloud. Here are five trends that you’ll want to keep an eye on this year.

Hybrid clouds

Pop quiz: You’re an IT administrator at an insurance company, where strict internal mandates and Federal regulations alike require you to keep sensitive customer data on-premises and in your care. But you want to take advantage of affordable, scalable, externally-managed public cloud services, too.

Enter the hybrid cloud. Every vendor has their own definition of what exactly “hybrid cloud” means, but at the core, the idea is that on-premises resources and the public cloud are joined for the best of both worlds. That way, data and applications that need to stay local can do so, while those apps that can be outsourced can get many of the benefits of the public cloud. As cloud computing picks up steam in 2012, more and more businesses are going to find that they need this mixed approach to meet their security and privacy guidelines.

And vendors are ramping up to meet the challenge. On stage at CloudBeat 2011, Oracle technology product marketing VP Rick Schultz listed hybrid cloud enablement as a key priority for the recently-unveiled (and succinctly-named) Oracle Public Cloud. And speaking of CloudBeat, a survey we took at the event found that IT pros had hybrid clouds on their minds. Also, Cloud operating system Nimbula makes hybrid cloud management its specialty.

With this much momentum, it seems likely that plenty of other vendors are going to be putting the hybrid cloud model into the spotlight this year.

Consumer cloud services

Unlike hybrid clouds, this is a trend you can see every day. Chances are pretty good you already have a Dropbox or a Box account for cloud file storage and sharing. Everyone has their choice of Google Apps or Microsoft Office Web Apps for everyday document creation and editing in their browsers. Android devices can choose between Amazon Cloud Player or Google Music for MP3s on the go. And perhaps most influentially, the Apple iPhone 4S brought with it the Apple iCloud, enabling the hordes of iOS customers to keep their music libraries, bookmarks, calendars and other files in sync wherever they go.

As XKCD presciently pointed out, people are increasingly finding that all they need is a browser to get stuff done.

If you need proof that 2012 is only going to make that ball rolling, then just look at this month’s CES coverage. As usual, where Apple goes, the technology market follows, and Acer took the lid off AcerCloud, its shameless iCloud competitor. LG and Gaikai are teaming up to bring video gaming straight from the cloud into your television set. Even Mercedes-Benz is putting a cloud-connected console straight into the dashboard. And so on.

Pretty soon, there’s going to be no escaping the cloud, whether you’re at home, at the office, or even in between.

Virtual Desktop Infrastructure

Before CES, I would have pegged this as another enterprise-focused, behind-the-scenes kind of avenue of cloud innovation. But then OnLive, best known for streaming video games from the cloud, debuted OnLive Desktop, and opened the door for the consumer, too.

Virtual desktop infrastructure (VDI) is an acronym that pretty much what it sounds like: Essentially, it gives you a remotely-accessible virtual desktop that simulates a computer that doesn’t (physically) exist. For businesses, the value can be immense: Rather than buy five hundred desktops, just build a VDI cloud or contract one out from a third-party provider, and a dozen people can share one computer’s worth of resources.

As an added benefit, employees can often log on from any computer that has an Internet connection and have their exact same work desktop waiting for them wherever they go. And when the size of the workforce changes, it’s easier and cheaper to provision and delete accounts than it is to buy a new machine or reformat it for a new user. Plus, in the rising “bring-your-own-device” era of IT, the ability to run any enterprise app on a tablet or smartphone is too good a bonus to pass up.

OnLive isn’t the only company that sees market potential here. Startup dinCloud kicked off 2012 by raising a cool million in seed funding for its cloud-hosted VDI service, with investors no doubt drawn by the fact that it launched with support from major players like NetApp. But that’s small potatoes next to the $70 million Goldman Sachs invested in AppSense in the early part of 2011, as it predicted that the VDI market would hit $2 billion over the next several years.

And as the Google Chromebook, the Apple iPad and other mobile devices continue to rise in popularity this year, VDI is in a good place to help make them business-worthy, since what’s under the hood matters a lot less than the strength of the network connection.

Open source and open standards

The OpenStack open source cloud platform may have started in 2010, but 2011 was the year that it really kicked into high gear. Built on community-contributed code, OpenStack aims to let any enterprise deliver its own infrastructure-as-a-service (IaaS) platform on standard hardware. OpenStack project founders Rackspace and NASA were joined by a community of over 110 other vendors, including heavyweights like HP, Dell and Citrix, as it debuted no less than three major feature releases. And while OpenStack’s leadership openly admits that there’s still a ways to go before it can compete with entrenched vendors like Microsoft and VMware in the data center on its own terms, the platform is maturing quickly and 2012 is going to see many companies build real, functional, salable cloud offerings on top of OpenStack.

But OpenStack isn’t the final word on open source in the cloud by a long shot: Apache Hadoop came out of beta earlier in January, giving companies tools to manage huge amounts of data, and the Oracle Big Data Appliance is already using it. The Open Data Center Alliance is going to continue its mission of improving and standardizing more efficient cloud facility designs. Node.js is only getting more popular for developing web applications. Even US Federal Chief Information Officer and Administrator Steven VanRoekel has publicly trumpeted the development of open standards in the cloud as a priority for his office.

And there are many more initiatives out there, besides. Vendors are moving to both open up and standardize the cloud, with an end goal of completely eradicating the concept of cloud vendor lock-in – which has stood as one of those major obstacles to cloud adoption mentioned before.

Cloud legislation

Here’s where things get a little sticky. There’s a reason Rackspace CEO Lanham Napier was originally slated to testify against the Stop Online Piracy Act (SOPA) on January 18th – though with the January 17th announcement that the debate over the controversial would be tabled for a month, Napier never wound up following through.

As a prominent cloud service provider, Rackspace stands to suffer in unforeseen ways under the act. If a customer stores infringing material in their cloud, is Rackspace liable? If so, would they be required to turn off that customer’s access with no warning? And so on, and so forth. SOPA is problematic in many ways, and 2012 is going to bring a lot of confusion before it brings answers.

Meanwhile, across the pond, European businesses are rethinking their own cloud migrations for a reason you may not expect. It turns out that any data stored with a cloud provider based in the USA is legally vulnerable to the Patriot Act, granting American authorities essentially unlimited license to potentially sift and analyze it without ever letting the customer know. Microsoft signed the EU model clauses for its Microsoft Office 365 cloud productivity suite as a way to quell fears, but several analysts have found it to be an insufficient safeguard against that kind of privacy breach. 

The debate over legal issues in the cloud is only going to heat up as we find more questions and fewer answers. And it seems only a matter of time before someone somewhere introduces legislation to try to address these issues.

What do you think?

The cloud’s a busy place these days, and there’s more going on than just five things. Where do you see the cloud going in 2012? Let us know in the comments below.

[Image courtesy of NASA/Flickr]

Sept. 9 - 10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Apache Hadoop-focused management service Cloudera has considerably updated its Cloudera Enterprise software and will begin offering a free version of Cloudera Manager that startups can use, the company announced this morning.

Cloudera makes it possible for companies like Samsung, Aol and Groupon to manage and mine loads of unstructured and structured data. It provides its own spin on Hadoop’s powerful open-source data management software and couples it with IT support and management tools. With the announcement of Cloudera Enterprise 3.7 today, Cloudera is both helping its customers by giving them a larger tool set for managing Hadoop and giving small companies a chance to use Hadoop for free.

“There is no other solution on the market that can support and manage the full operational lifecycle for the full Hadoop stack with software and support integrated together,” Charles Zedlewski, vice president of product at Cloudera, told VentureBeat. “Customers no longer need to cobble together five or six tools and applications to operate a sophisticated Hadoop platform.”

The company identified the four biggest additions to Cloudera Enterprise 3.7 as the following:

• Intelligent Hadoop Log Management: Gathers and scans Hadoop logs for irregularities and proactively create events for the operator.
• Global Time Control: Correlates cluster-wide jobs, activities, logs, system changes, configuration changes and service metrics along a single timeline to dramatically simplify diagnosis.
• Alerts: Alerts on nodes or services in poor health, as well as jobs or activities that are slow or failing; integrates with the central company alerting system.
• Support Integration: Takes a snapshot of the cluster state and automatically sends it to a Cloudera support professional to assist with diagnosis and resolution.

As for the new free version — aptly named Cloudera Manager Free Edition — it supports up to 50 nodes and can be downloaded at cloudera.com/downloads. It allows developers and businesses to configure and perform basic management of a Hadoop cluster.

“The goal with the free version is to simply grow the base number of Hadoop users,” Zedlewski said. “We’re not in the business of making money on 20-person startups.”

Palo Alto-based Cloudera has raised an impressive $76 million to date in four funding rounds. The company last raised $40 million with the round led by Frank Artale of Ignition Partners with participation from Accel Partners, Greylock Partners, Meritech Capital Partners and In-Q-Tel.

Sept. 9 - 10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Venture firm Accel Partners has carved out a $100 million “big data” fund to invest in companies focused on building new IT infrastructure or on applications than run on that new infrastructure.

Accel, based in Palo Alto, Calif., at the heart of Silicon Valley’s venture capital community, has invested in companies like Facebook, Dropbox, Cloudera and Etsy.

As such, the firm has seen how companies like Facebook have been forced to exploit new technologies to store and analyze their huge amounts of data more efficiently. In Facebook’s case, it has used open source project Hadoop to help it process the billions of messages it receives each day. NoSQL database technology is another way companies have become more efficient in storing data.

All big Web companies, including Google, Yahoo and Twitter, and increasingly large enterprise companies, are building applications on these platforms.

Ping Li, the partner at Accel (pictured right) who has led the firm’s investments in in companies such as Cloudera — which commercialized the Hadoop technology — said the new fund will be invested in two types of companies: (1) companies building out the new infrastructure, including in storage, security and management; and (2) companies building applications on top of that infrastructure (spanning, for example, business intelligence, collaboration, mobile and vertical apps).

He said these companies will span just about every sector, from enterprise to gaming — all of which will require new kinds of data-intensive platforms, he said. Investments will be made globally, he added.

Over the last 30 years, legacy data platforms, including relational databases, drove the emergence of significant companies like Oracle, SAP and Symantec, Li said. Likewise, big data will usher in a new era of multi-billion software companies, Li says.

The firm has carved out the $100 million from its existing funds, so this does not represent a fresh dollop of cash, Li said.

Accel also plans to host a “big data” conference in Spring, 2012, to drive discussion on technology trends in the sector, Li said.

Coincidentally, these “big data” technology companies will be a focus at our CloudBeat 2011 conference, which takes place November 30 – December 1 at Hotel Sofitel in Redwood City, CA. We’ll be focusing on 12 of the most disruptive instances of enterprise adoption of the cloud. Join over 500 executives for two days packed with actionable lessons and networking opportunities as we define the key processes and architectures that companies must put in place in order to survive and prosper. Register here. Tickets are limited!

Cloudera provides its own take on Hadoop’s powerful open-source data management software and couples it with IT support and management. Using Hadoop, enterprises can store and process huge amounts of unstructured data. But Hadoop can often be unwieldy and difficult to manage, so Cloudera helps make it possible to manage that data effectively.

“Any company with a data-intensive environment is probably using Hadoop in some form,” Cloudera COO Kirk Dunn told VentureBeat. “Whether those companies have sought out Hadoop training or management solutions, we’re the company people are looking to and the leaders in this space by any metric.”

The new $40 million funding round was led by Frank Artale of Ignition Partners with participation from existing investors Accel Partners, Greylock Partners, Meritech Capital Partners and In-Q-Tel.

“Cloudera is the defining company in the big data industry and the popularity of Hadoop is increasing every day,” said Artale, in a statement. “The Big Data space is moving fast — we looked at many investment opportunities and chose Cloudera due to their clear leadership. The team and its technology are the best in this business and we are excited to be part of it.”

Cloudera also announced today that it has partnered with storage and data solutions company NetApp. The two companies have created the NetApp Open Solution for Hadoop, which combines Cloudera’s wide-ranging Hadoop distribution with a NetApp-constructed RAID architecture. It’s a big deal for enterprise customers seeking to put more power behind Hadoop and customized data processing and management efforts.

“This gives NetApp’s customers the ability to process new data and existing data and that data together, which hasn’t been possible before,” Dunn said.

Cloudera is on a roll when it comes to funding, and its total raised, including today’s round, now stands at an impressive $76 million. It raised a $25 million round a year ago with funding led by Meritech Capital Partners and previous investors Accel and Greylock. Before that, it raised $6 million in June 2009 and $5 million in March 2009.

NoSQL databases such as MongoDB, CouchDB and Cassandra have a number of advantages over traditional relational databases of the type proved by Oracle and Microsoft. Instead of rigid schemas, NoSQL databases have flexible ones which can be updated easily, data access is faster and they often scale better to store very large amounts of data.

Cassandra, which is used by web giants like FaceBook and Twitter, uses a peer-to-peer architecture which has no single point of failure. It’s designed specifically for high-availability applications which cannot afford to lose data even if an entire data centre went down. You wouldn’t be happy if Facebook lost your account data or Twitter your follower list, now would you?

Cassandra can also store different types of data at the same time: structured (similar to that stored in a relational database), semi-structured (data in a format like an XML documents which contains tags or other markers to separate semantic elements) and unstructured data. Cloud or on-site servers can be used to host Cassandra.

DataStax’s CEO Bill Bosworth sees the traditional relational database vendors like Oracle and Microsoft as the main competition. It’s still a big shift, in terms of mindset as much as anything else, for a company to move from a relational database to a NoSQL database. “We want to make it easy and efficient for people to adopt this new technology.” said Bosworth. “It’s a philosophical shift that you have to make but once you make it, you start to see solutions in entirely different ways.”

I asked him about what pushes companies to make that leap.”The top reason would be scale.” said Bosworth. “The second reason would be data type.’I have semistructured and unstructured data I want to work with as well’. The third reason would be change. ‘I want that flexible schema.’” If the existing database uses lots of procedural logic, however, migration will be more complex.

There’s also the cost factor. Bosworth gave the example of a customer called Constant Contact which had been considering using a relational database for a new project and estimated that it would take 9 months and cost $2.5 million. With Cassandra, they were able to get up and running in 3 months at a cost of $250,000. DataStax currently has around 100 customers.

DataStax also announced that it will release a new enterprise product in Q4, 2011 which will include the existing OpsCentre web-based monitoring product and Hadoop and MapReduce (distributed computation of large data sets stored across a cluster) functionality on top of Cassandra. DataStax will charge a per node subscription fee for the product. The new funding will mainly go towards the development of that product and expanding sales efforts.

DataStax was founded in 2010, has 30 employees, is based in the Bay area and received $3 million in from Lightspeed Venture Partners prior to the current round.

Sept. 9 - 10, 2013
San Francisco, CA

Early Bird Tickets on Sale

Platfora, a data management software provider based on Hadoop, announced on Thursday it has raised $5.7 million from Andreessen-Horowitz just a few months after the company was founded.

Hadoop is an open-source data-management software framework. It’s useful for companies that store enormous amounts of data and have to regularly index it. That can include financial services companies that have to track previous prices and old transactions or companies like Yahoo that need to regularly access search information. Platfora aims to add a more manageable user interface to access all that and make the data easier to digest for everyday users and business professionals.

“GoodData (and other big data companies) have a very cool system, but they are focused on a different scale of data,” Platfora chief executive Ben Werther told VentureBeat. “We’re focused on the particular challenges that emerge when you have terabytes, petabytes and beyond and need to get work done for the business rather than doing science projects.”

Platfora takes all that information pulled in from Hadoop’s data management software and delivers it through dashboards and other easy-to-digest user interface elements. There are lots of distributions of the open-source service, which anyone can access and “fork” into their own version and distribute. Werther said Platfora is focused on making sure that its software will work on top of any distribution of Hadoop.

“The last thing everyone needs is another Hadoop distribution, so we’re going to focus up the stack and remain agnostic to the Hadoop distribution below,” Werther said. “And our product, while not open-source, is really intended to be useful to, and accelerate adoption of these open source technologies by allowing much wider groups of users to get value from them.”

Yahoo, Google and Facebook all also use Hadoop to power some of their services. A recent shake-up at Yahoo (the firing of chief executive Carol Bartz for failing to meet the board’s expectations), one of the largest proponents of Hadoop, could mean a change of direction, but that shouldn’t concern open source and Hadoop developers, since many other companies have since begun actively developing and deploying Hadoop, including Cloudera, MapR and HortonWorks. Tech giants like EMC, IBM and Dell are also investing in Hadoop and pre-loading it on some of their server architecture, Werther said.

“Yahoo was a really important supporter of Hadoop in its early days, but the good news is that it has a lot of momentum and innovation from a variety of camps,” Werther said. “In fact the opposite problem is more of a concern — fragmentation due to all the tracks of innovation, but there hasn’t been any evidence of that to date.”

Platfora is based in Palo Alto, Calif., and is on track to have 20 employees six months after launch.

Hadoop is open source software that allows companies to store data on clusters of cheaper servers and run software very quickly on top of those clusters. It organizes and prioritizes data, so often-used data is pushed to the top of the stack for easy access. Along with the organization, Hadoop uses a graphical user interface and heat maps to help data managers monitor usage and locate problems. The software also offers analytics for companies that want to watch their data to spot emerging trends.

“It’ll take a cheap commodity hardware which has storage and CPU power and it will allow you to store data and run operations in parallel,” explained vice president of marketing Jack Norris, in an interview with VentureBeat.

Hadoop is part of the Apache Software Foundation, which focuses on the development of open source software that allows anyone to run or change however they please.

Hadoop is currently used by Yahoo, Google and Facebook, but it’s complicated enough that companies with fewer technical resources than these giants might have difficulty implementing it. That’s where MapR hopes to help.

MapR is a distribution of Hadoop: It’s an edition of the software tweaked by MapR for ease of use.

MapR’s software takes Hadoop and makes it safer and more efficient for enterprises to use. It does this providing support for Linux HA, support for random read/write storage (or the ability to overwrite data) and a native network framework system. The NFS was created by running Hadoop on top of MapR’s storage system, as opposed to Linux.

“[Hadoop] relies on the underlying Linux file system,” said Norris. “We’ve replaced that with the MapR storage system. We’ve rearchitected how files function.”

In addition to its storage system, MapR protects the data by providing snapshots of your data center operations to help with troubleshooting. It also mirror-saves all data, so that if one server goes down, your information is recoverable.

MapR plans to use its funding to scale its operations after seeing recent growth. The round was led by Redpoint Ventures along with existing investors Lightspeed Venture Partners and New Enterprise Associates.

The company was founded in 2009 and has since collected a total of $29 million in funding from the same investors. MapR is located in San Jose, Calif. and has 40 employees.

Hadoop is an open-source data-management software framework. But Cloudera provides a version that is specifically for enterprise users, coupled with IT support and management. The software provides analytics and storage options for large-scale networks that have thousands of nodes and process petabytes (that’s 1,000 terabytes) in any given second. Yahoo, Google and Facebook all also use Hadoop to power some of their services.

The new software, called Enterprise 3.5, automates a lot of the maintenance and administrative proceses like installing security software onto computing clusters running Hadoop-based software. It also features an activity monitor that will relay information about data usage and creation in real time to system administrators, and lets them interact with the computing nodes in real time. For example, any time an anomaly occurs in the system, administrators can quickly lock down and secure the compromised computing node to prevent any further leaks.

The Cloudera software actually consists of 11 different open-source Hadoop projects lashed together into a single enterprise software service. It’s similar to how some companies operate with Linux. Its main software offering for the enterprise, Cloudera Enterprise, also provides users with resource management and real-time monitoring of applications.

The Palo Alto, Calif.-based company was founded in 2008 and has so far raised $36 million. It has around 40 employees, and its clients include RapLeaf and Bank of America. Its most recent round of funding was led by Meritech Capital Partners, though existing investors Accel Partners and Greylock Partners also participated in the round.

Hadoop is an open-source data-management software framework. But Cloudera provides a version that is specifically for enterprise users, coupled with IT support and management. The software provides analytics and storage options for large-scale networks that have thousands of nodes and process petabytes (that’s 1,000 terabytes) at any given second.

The Cloudera software actually consists of 11 different open-source Hadoop projects lashed together into a single enterprise software service. It’s similar to how some companies operate with Linux — the software is open source. Its main software offering for the enterprise, Cloudera Enterprise, also provides users with resource management and real-time monitoring of applications.

The funding is pegged for further research and development and marketing of the Hadoop software architecture for enterprise usage.

The Palo Alto, Calif.-based company was founded in 2008 and has so far raised $36 million. It has around 40 employees, and its clients include RapLeaf and Bank of America. Its most recent round of funding was led by Meritech Capital Partners, though existing investors Accel Partners and Greylock Partners also participated in the round. Yahoo, Google and Facebook all also use Hadoop to power some of their services.

[Photo: mfrascella]

Six Apart’s headaches — The blogging software market is highly competitive, so small differences in quality can make a difference in user adoption. Six Apart, a Silicon Valley start-up that offers several blogging software platforms, including Movable Type, has released a product after acknowledging internally it could make developers mad. [Six Apart's Anil Dash has since responded in comments, saying the company took time to fix the bugs for the release. The references in an internal memo before the release that caused concern were the following: The need for "PR to stay ahead of the curve" with people who say "we rushed the release" and to developers who "will be very mad" for not having the resources to upgrade their plugins.] Moreover, chief architect Brad Fitzpatrick has left the company. We asked Six Apart for comment, but they did not respond. Frankly, the various bugs are one reason VentureBeat moved to embrace WordPress, dropping Movable Type. WordPress, owned by Automattic, also seemed swifter and more flexible. Separately, WordPress’ hosted version (WordPress.com) is more secure than Six Apart’s hosted versions. WordPress has three different data centers, so you wouldn’t see it crash like Six Apart did when an SF data center melted down several days ago. WordPress is run by a Swiss named Toni Schneider, who is obsessed with scaling the company without hitch or server meltdown. Lead developer Matt Mullenweg goes to bed thinking about scaling problems, Schneider adds. Related: I cited Toni in a piece I just did for Forbes about “when to hire an IT guy.”

Clicktale records what happens when a user hits your Web site — Today, Clicktale launched to show things like the number of mouse hovers over a link (in other words, showing how a link may be attractive, but not enough for people to click), how many mouse hovers eventually convert to mouse clicks and other interesting behavior measures. Techcrunch has a good review here. The Israeli start-up has competitors, including RobotReplay and TapeFailure.

Zhanzuo.com, a Facebook clone in China, has acquired Yoolin.com, a campus social networking site targeted at Chinese students abroad — Yoolin was founded in June 2006 by Chinese students from Stanford, UC Berkeley, Harvard and MIT, but according to Alexa data cited by this blog (which reported the acquisition), their traffic didn’t grow much.

Another news site, Newser.com — Journalist Michael Wolff has started a news Web site called newser.com that aggregates news articles for convenient reading. New York Times has the story. We don’t understand the company’s model. The cater-to-all destination site is a dead horse fairly beaten.

Non-profit music industry agent conflicted? — Wired reports that Sound Exchange, a nonprofit that administers copyright licensing and license-fee collection is funding a group called musicFirst, which is lobbying for the enforcement of extra broadcasting fees on terrestrial radio stations. According to Wired: “Whether or not SoundExchange’s lobbying efforts prove to be illegal, its presence as an advocate in this debate undercuts its role as neutral administrator of royalty fees set and approved by the Copyright Royalty Board.”

GoFish’s acquisition of Bolt, dead in water — Details here.

Microsoft testing ad supported Microsoft Works — Details at Ars Technica.

Advertisers in UK yank ads from Facebook when they realize the ads are posted next to the group page of a far-right-wing political party — Details here. Slowly but surely, advertisers are beginning to realize how dangerous it is to run campaigns in social networking sites. Tod M. Sacerdoti, founder of BrightRoll, which inserts advertising into videos for clients, told us recently he has all but abandoned serving social network sites, after seeing multiple examples of advertising networks exposing major advertising brands to lewd, quasi-porn video content.

Answers.com loses 28 percent of its viewers due to a change by Google’s algorithm — GigaOm points to the story.

IAC bags Google, chooses Microsoft – It will use Microsoft’s aQuantive’s ad network over Google’s Doubleclick.

Thomson Financial finally releases VC data — Like the data released last week by VentureOne, it shows venture capitalists are investing at the highest levels since 2001. VCs poured $7.1 billion in 977 deals in the second quarter of 2007 – the largest number deals since the third since Q3 2001. A Thomson spokeswoman said the week’s delay in the survey was caused by a server crash, and then a week’s worth of verification with its partners in the quarterly MoneyTree Survey: PricewaterhouseCoopers and the National Venture Capital Association. We asked whether a reported 61 layoffs at Thomson in recent months had anything to do with the snafu, since some of the VC reporting department has been replaced by outsourced labor in places like the Philippines. Or perhaps caused by distraction caused by Thomson’s pending merger with Reuters? A spokeswoman did not address the layoffs, but said the Reuters deal hasn’t closed yet, and so that played no role.

Vibrator maker Jimmy Jane might get real VC? — Individuals such as Tim Draper have backed Jimmy Jane, the sex-toy company, with about $1 million, but now the New York Times’ Matt Richtel says the company is about to get a real VC round. We contacted Jimmy Jane for comment, but no response thus far. Draper, for his part, suggested something is coming: “I don’t think I am allowed to answer that,” he said, when asked about a pending round. “We don’t make comments on financings until they are done.”

Yahoo advised to… go after social networking — An analyst report by Bear Stearns recommends that Yahoo more aggressively pursue social networking, saying it is a high growth opportunity, and noting that Facebook could be worth $6 billion or so. Yahoo had reportedly sought to buy Facebook last year for $1 billion.

Yahoo advised to… pursue open source — Tim O’Reilly says Yahoo is grasping open source as a competitive advantage and commends it, writing off the news Yahoo is now supporting something called Hadoop.

WiMax notebook computers coming by late next year — So says Intel, a leading provider of the technology, which will operate many times faster than WiFi technology used by most laptops. (Mercury News story)

Silicon Valley’s WiFi network project shifts from free, to paid — A Mercury News story shows the Silicon Valley Joint Venture Wireless Project looks shakier than ever.