Big banks have been keen early-adopters and patrons of the “big data” movement.

One early proponent, Citigroup famously hired Watson, IBM’s number-crunching supercomputer, to help it decide what new products and services to offer customers. Big data technologies are already being used to clamp down on fraud and determine which customers are credit-worthy.

But similar technologies are also bolstering the efforts of competitors to traditional banks that are providing small-dollar loans to people in need. One such firm is ZestFinance, the brainchild of a former engineering lead at Google, which gives loans to people with bad credit histories. ZestFinance crunches far more data than a Main Street bank — it looks at thousands of indicators to determine which customers are most likely to repay a loan in full. These indicators might include the rate of answering emails or the level of interaction with potential employers on LinkedIn.

Likewise, Kabbage offers loans to online merchants that have little or no access to capital (editors’ note: Kabbage won the Innovation Showdown at VentureBeat’s cloud computing conference, CloudBeat). Kabbage looks at a range of external data, including social media streams. ”If a small-business owner is interacting with customers on Facebook and Twitter, that’s an important sign for us,” said Rob Frohwein, the company’s CEO in a recent interview.

LendUp, a startup that Frohwin admires, recently raised a fund to dole out small dollar loans to give to people in need. The team of engineers developed a “big data” analytics engine to get a better picture of the borrower. In addition to demographic information and credit history, the algorithm also pulls in social data from Twitter, Facebook, LinkedIn, and other data sources such as email.

LendUp’s goal is to provide relief to a deserving market. It’s a nice idea in theory, but is the company’s algorithm advanced enough to truly pick out the borrowers that won’t default on a loan? As Wired’s Marcus Wohlsen points out, payday loan experts aren’t convinced that it is. Critics say that the tech community’s efforts to provide loan alternatives to people traditional banks turn away are misguided and that tech entrepreneurs are not going to overturn cycles of crippling debt anytime soon.

To better gauge the arguments on both sides of this question, we asked financial-technology investor Jon Soberg to duke it out with big data entrepreneur, Roman Stanek. Can big data play a role in alleviating poverty? And if so, what type of data is most useful to this new wave of lenders? Here’s what they had to say.

Related: Read our inaugural debate, “Can Big Data solve America’s gun problem?”

Yes. Especially in countries where there is no credit bureau

Jon Soberg, fin-tech investor, Blumberg Capital (disclosure: Soberg is an investor in Lendo and Kreditech) 

I almost feel like the question should be, “How can we justify not using big data to help alleviate poverty?” I say absolutely that big data can, will, and should help alleviate poverty; we are just starting to scratch the surface.

In many countries, there is no credit bureau. Credit is simply not accessible for many people; this is not only true for individuals, it is also true for firms. Nobel Peace Prize-winning Professor Muhammad Yunus, founder of the Grameen Bank, believes that credit is a human right, and the Grameen Bank is at the forefront of making this happen in the most impoverished nations through micro-finance.

We are also seeing a new generation of profit-seeking lenders who are making credit more accessible to both firms and individuals though big data technology.

In countries lacking basic financial data that we take for granted in the US, companies such as Lenddo are using alternative data sources, particularly social media data, to underwrite loans. Lenddo is operating very successfully in the Philippines, catering to the emerging middle class, and offering access to credit that has not existed before in a profitable and non-predatory way. Using social media, they are able to understand enough about a person to assess their ability and willingness to pay and even influence payment using micro-finance-type approaches.

When I worked at Ditech.com in the mortgage industry, underwriting was the critical component of the lending process. I always found it perplexing that the data used in underwriting essentially didn’t change. A FICO score was the key determinant between a prime and subprime customer but was not necessarily highly correlated to the customer’s viability. Today we are seeing a new breed of entrepreneurs going into the financial services industry with a refreshingly new approach.

Did you know that people running a specific version of Windows in Poland are least likely to pay back a loan? Kreditech and ZestCash use literally thousands of data points to assess credit worthiness, and Kreditech can get cash to someone typically in 15 minutes.

One of the key pushbacks when I discuss this new breed of lenders is privacy. Clearly, giving a financial institution carte blanche access to social media is akin to giving away your first born. However, the depth of data necessary to make better underwriting decisions does not need to be as personal as some might assume. Paying attention to the particular schools you attended, where you live, and the employment levels of your friends can be extremely meaningful in a financial decision, and that data is usually publicly available.

I’m not going to argue for a complete lack of oversight or for no regulation over the use of data, but I am saying that there is huge upside for everyone in using some of this data. It is even more pronounced in places where little financial infrastructure exists, and for people who have little or no financial history.

The bottom line for me is quite simple: Big data will provide the ability to make credit available to a much larger percentage of the population and to do it more efficiently than other methods. Using big data responsibly, we will make the world a better place.

No. Using social media data is unethical

Roman Stanek, CEO, GoodData

I see it all the time: the notion that data mining by itself can cut risk — to the point where banks decide whether or not to issue loans. How? By evaluating and analyzing revealing social media data pertaining to a person or small business requesting funds.

I’m obviously a proponent of using data to help make money, whether that’s by increasing revenue, boosting profits, or minimizing risk. But I have to draw a line against the latest concept being currently discussed in the banking industry: capitalizing on the barrage of social media, such as Tweets and Facebook posts, to decide who does (and does not) deserve of a loan.

To me it’s a question of ethics. We need to set boundaries so that companies aren’t mining intimate information we deliberately or unknowingly reveal about ourselves. Is it fair, for example, to hold a college grad’s photo of celebratory drinking against her when she asks for a small-business loan years later? To me, the answer is an obvious no. Society should not let someone’s moment of innocent indiscretion affect her life. If we are to be held accountable for information that defines us, let it at least be something over which we have control, such as our credit score.

As a society we sometimes forget about ethics as we strive to find new ways to build wealth. We’re still living with the ramifications of that ethical lapse when banks fell over themselves to issue subprime loans to people who could never afford them and other banks sold unsafe mortgages to unsophisticated buyers. Without an ethical compass, the financial industry lost its way and dragged down the global economy.

Don’t get me wrong. I’m not suggesting data has no role in a bank’s risk analysis. Collecting the appropriate data, processing it, and presenting it in a way that enables business people to glean insight will become even more vital to the banking industry. I am saying that the decision process needs to incorporate the right data, seasoned with a healthy dose of ethical consideration.

Girl with cash image //  jcjgphotography, Shutterstock

This is a guest post by entrepreneur Jason Garoutte 

It’s hard to be a marketer these days. Even when we carefully craft a message and follow best practices, 99 percent of our attempts can still fall on deaf ears. Are there any other professions where 1 percent is considered good?

I talk to Sales and Marketing leaders every day. Nobody ever says they’ve got enough pipeline. But where can they go for more? Email marketing and pay-per-click both show declining ROI, and many marketers have already invested to the point of diminishing returns. This year the answer seems to be Marketing Automation, and indeed this can add a flow of leads over time. But what if you don’t have time to wait? Today’s marketers need shortcuts through the funnel.

The good news is that shortcuts are possible, and it all comes down to smart use of data. The key skill for marketers to develop over the next two years is the ability to synthesize data. Thanks in part to the rise of social networks, companies and people share more about themselves than ever before.

In this publicly available data, there are gems that can determine targeting, messaging, and personalization. Using data like this is no different from what a good sales rep does, except that a sales rep searches manually using sources like websites and LinkedIn. Marketers need to scale this research for an audience of thousands or millions.

No matter how a marketer sources leads, a little data can propel leads through the funnel. Even if you’re not ready for “big data” tools and predictive analytics, you can still find impressive shortcuts with a bit of intuition and elbow grease.

My top tip to marketers: don’t boil the ocean. Very few marketers have access to data scientists for data mining. So don’t seek to create your own giant database — it will just go stale before you benefit. Instead, be pragmatic and look for quick wins.

Here are three ideas that work from my recent experience.

• Use data to find a specific business need

Just a few years ago, the state of the art in targeting was to select four dimensions: industry, size, geography and title. I don’t know about you, but for my company, there is no SIC code that buckets our prospects. Every business is unique, and your prospects have unique needs. So why not search for those unique needs? Forget SIC codes and look at employees titles. You can find these in job openings on websites and job boards. You can look at titles on LinkedIn and Jigsaw.

• Tailor content for “micro-segments”

Once marketers have a list of targets, it helps to sub-divide that list into “micro-segments”, or groups with a unique perspective on the product being sold. Here’s an example. Some of your prospects may already have a competing technology in use. Often there is evidence of that technology, such as the expertise of employee biographies or required skills on job postings. Some technologies, like web analytics tools, can be seen directly in JavaScript.

If you write a different story for prospects that already use your competitor, you can emphasize the benefits of swapping. It’s s a great way to establish immediacy and relevance. It also shows that you’ve done your homework. I’ve seen campaigns double response rates by taking an existing email template and tweaking it for the customers of different competitors.

• Personalize the script with relevant data

Starting an email with the recipient’s first name hardly qualifies as personalization. Real personalization shows that you know something meaningful. If the recipient doesn’t feel special, then you haven’t personalized.

A marketing automation tool, like Marketo or Eloqua, offers dynamic fields in an email to plug in information and start a true product conversation. With this precision, you can create personalized campaigns that are very relevant to each recipient. In summary, these are exciting times for marketers.

Jason Garoutte is the CMO and GM of Mintigo, which operates a demand generation platform that uses predictive analytics to help marketers find and engage prospects. Prior to joining Mintigo, he worked as a vice president of product marketing at Salesforce.com. 

Top image via Shutterstock

Social data startup DataSift has raised $15 million in its second round of funding, cash that will help it continue to swiftly grow and brew better social data technology.

DataSift made our recent list of 10 startups leading the way in ‘big data’ because its tech can analyze data across the social web, which helps companies get a handle on public opinion and better shape marketing messages. The company said it has doubled revenue every quarter since its 2011 launch and has accumulated “hundreds of corporate customers.”

“We have the world’s best infrastructure in real-time, as well as historical processing of social data, and through our Series B funding, we now have the ability to take that infrastructure to the next level,” DataSift founder and CTO Nick Halstead said in a statement. “The promise of the DataSift platform lies in enabling our customers to bring together their private and public data and applying our powerful processing to that unstructured data. The ultimate goal of big data is drawing actionable insights from a combination of public and private data.”

While Halstead seems enthusiastic about big data, we found it refreshing that DataSift CEO Rob Bailey recently told us he didn’t care for the term.

“I call bullshit on the term ‘big data.’ First of all, the big data problem has been around since before there were computers,” Bailey told VentureBeat a month ago. “Too many companies focus on managing big volumes of big data without focusing on end solutions and providing answers for customers.”

The new funding was led by Scale Venture Partners, with participation from Northgate Capital and Daher Capital. This brings DataSift’s total haul to about $30 million. Rory O’Driscoll, managing director with Scale Venture Partners will join DataSift’s board.

San Francisco-based DataSift was founded in 2010 and has more than 60 employees.

Social media image via sargsyan/DeviantArt

EverySignal is a new way to find the handful of important announcements from your social networks – all your social networks — in just one daily email digest.

The digest brings clarity, organization, and best of all, brevity to an information-gathering process that could take hours, should you choose to forage around the web for important events in the lives of friends, family, and colleagues. At a glance, you can see who moved to a new city, who’s sick, who had a baby — and you can respond with congratulations, encouragement, or advice of your own.

Yes, Facebook, LinkedIn, and Twitter all offer their own homepages, discovery tabs, and algorithms to sift out the noise and deliver the signal, but EverySignal’s new product brings a few distinct advantages, founder Derek Merrill said in a phone chat with VentureBeat yesterday.

“We aggregate across all of the networks,” he said. “We have access to the full breadth of a person’s networks.”

In addition to bringing more data into one place, he said, EverySignal also makes better use of that data without requiring users to take additional steps.

“Facebook, they’re gonna get better at it, but they’re kind of using Life Events as tags … but we know users don’t do that. They have to change their behavior. Most people just want to write what they want to write, and then technology should categorize it for them.”

Merrill said he got into the idea of social data sifting in the most normal way possible. Like most human beings (read: unlike most Silicon Valley wannapreneurs), he doesn’t live and breathe Facebook, so he has a more natural perspective on what’s wrong and frustrating about social media.

“I never lived inside Facebook,” he told us. “Because of that, I really was missing most of the things happening, or if I found out, I was the last person to know.” This applied to life events such as marriages or pregnancies as well as professional events such as promotions or lateral company moves.

“From my point of view, this is bringing me back into [social media] in a natural way. I don’t really care about a picture of someone’s burrito, but I do care when someone starts a business, or when you go on an exotic vacation, or when your dad is hospitalized.”

And for normal people who don’t exist with one eye on a smartphone around the clock, email is still a fine way to get that information across. In early testing, Merrill said, users opened around 70 percent of daily digest emails.

And the EverySignal team strove to find a familiar, safe way to display data for users — something that would convey all the information needed without overwhelming.

“It’s a tough problem,” Merrill said. “If you think about this from an engineering perspective, every user’s interface is different. That has been an enormous challenge. … We decided, let’s just play off normal user familiarity. We’re trying our best to make things feel comfortable. We don’t want to copy the Facebook feed, but that’s what people feel familiar with.”

EverySignal’s business plan involves using the same kind of social data for more commercial purposes, both for power users and for businesses, brands, and agencies.

“We have potential to build a very large business by offering a free product everyone can enjoy and then offering professional features we can charge for … things like getting more real-time updates through a mobile applications or SMS alerts, or access to more information within a network,” said Merrill.

EverySignal launched today from the Science incubator in Los Angeles. Science is about one year old and was founded by former MySpace CEO Mike Jones and Color co-founder Peter Pham.

Top image courtesy of Peshkova, Shutterstock

Avant-garde brands using Tumblr to engage fans can now turn to Union Metrics to dive into the data behind the activities of the startup’s eclectic audience.

Today, social media analytics startup Union Metrics announced that Tumbler named it the preferred analytics provider for brands and marketers on its service.

The San Francisco-based company, which secured access to the entire fire hose of Tumblr data through Gnip, now provides invited clients with analysis and tracking tools for identifying top influencers and curators and dissecting blog posts and topics.

Specifically, Union Metrics for Tumblr supports each of the following:

• Filtering capabilities that facilitate comprehensive tracking of any blog or topic.
• Summary analytics showing overall engagement levels and trends over time.
• Identification of influential contributors and curators.
• Analysis of posts and tags to surface most popular content.
• Individual post-engagement analysis, including interactions over time, amplification trends, and a reblog tree visualization, which clearly shows how posts spread through Tumblr.

Five-year-old Tumblr is home to more than 76 million blogs and nearly 33 billion blog posts. The New York-based company, which has raised $125 million in funding to date, only recently started selling advertisements to companies with big budgets.

The Union Metrics Tumblr tool is currently available to invited parties but will be available to the public at a later date. Interested brands can put their name down on the list for consideration.

---

Big Data and Predictive/Real-time Analytics startups: Are you looking to jumpstart development & accelerate market traction? Sign up for the SAP Startup Focus program to receive technology, support, resources and community to help you develop new applications on SAP HANA, a cutting edge database platform. Get started here, and enter promo code “VB2013″ on the form.

---

Birst raises $26M for big data

Big data startup Birst has raised $26 million in its fourth round of funding, with a goal accelerating growth and product development. The company provides business intelligence software and competes with QlikTech, SAP, IBM, Oracle, GoodData, and Microstrategy. The new funding round was led by big dog VC firm Sequoia Capital with additional participation by prior investors Hummer Winblad and DAG Ventures.

Engrade moves to the front of the class with funding to keep teachers organized

Classroom management software company Engrade has closed a $3 million funding round. The company’s software helps teachers, administrators, and school boards keep track of students’ grades and progress reports. Rethink Education led the round, with NewSchools Venture Fund and private investors Greg Gunn, Zac Zeitlin, and Richard Chino.

Flingo grabs another investment, mere months after its last one

Smart TV app maker Flingo has received another $1 million in funding from entrepreneur Mark Cuban, bringing its total first round to $8 million. Flingo has developed a technology, dubbed SyncApps, that can detect what you’re watching in just a few seconds and communicate that information to your television, smartphone, or computer.

Datasift secures $7.2M investment

DataSift, a social data platform that extracts large amounts of data from a number of online sources, raised $7.2 million. Essentially its services help understand the impact of social media marketing techniques employed by business. Existing investors IA Ventures and GRP Partners led the round.

App security startup Appthority gets a $6.25M boost

App security company Appthority has raised $6.25 million. The company works to prevent potential apps with malware, location tracking, and keystroke recording  from ending up on your smartphone. US Venture Partners and Venrock led the funding.

Vungle Raises $2M seed round

Mobile advertising company Vungle has raised an impressive $2 million seed investment. The company creates ads in the form of video trailers for mobile applications. Big name VCs and angel investors Google Ventures, AOL Ventures, Crosslink Capital, Ron Conway, Dave McClure, Maynard Webb, Scott McNealy, and Tim Draper participated in the round.

McCann WorldGroup invests $4M in Israeli incubator

Media startup incubator Thetime has raised $4 million from investor McCann WorldGroup. Thetime is an Israel-based incubator for media companies looking to get guidance, funding, and new business ideas before they obtain a seed round. After seed funding, Thetime offers additional resources to help startups receive institutional funding. McCann WorldGroup gets a 15 percent stake in Thetime for its investment.

Collaboration software startup Blueprint raises $15.9M

Blueprint has raised $15.9 million to help with software development project management. The company helps business people and software developers, working for the same companies, set goals for software and app development projects. It then manages the work for each project, helping each department communicate effectively. New investor Tandem Expansion Fund led the round, with the participation of existing investors from BDC IT Venture Fund, Walsingham Partners Fund, and several private investors.

Reading child against numbers background image via Shutterstock and Flickr

Klout

Last week, Klout altered the way it determines a person’s “Klout Score” — the measure of a person’s activity on a variety of social networks such as Twitter, Facebook LinkedIn Google+ and others. Based on an individual’s interaction within those social networks, Klout calculates the true reach of that person’s communications and issues them a 1 to 100 Klout score.

The new way of measuring factored in profiles from people who haven’t even signed up for Klout. Those accounts, which don’t require an actual person to opt in, are dubbed “observers”. For instance, a second Twitter account that you occasionally use and/or a public Facebook profile that isn’t connected to a Klout account will pop up in Klout as an observer profile.

Much in the same way that people have criticized services like Pipl and Spokeo, some Klout users reacted negatively to the observer profiles. One user, Tonia Ries, questioned the ethics of the startup’s actions after finding an observer profile for her young son.

“When I logged into my Klout page this morning, I was very surprised to see that Klout now lists my son as one of the people I influence. Anyone who is a parent of a young adult will know that nothing is more unlikely,” Ries wrote in an article for The Realtime Report. “Knowing that my son is not on Twitter, and has always been very careful about managing his privacy on the Internet, [I wondered] how did Klout get the information to create a profile on my son?”

Because of concerns like this (as well as Klout’s policy of not allowing anyone under the age of 13 to use the service), the company has decided to provide a way for anyone to delete their observer profile information. Anyone who wishes to do so can visit Klout’s Opt Out page, which requires you to enter in Twitter or Facebook credentials to verify that the account you’d like to delete is actually yours. “Note that we request only the bare minimum information provided by Twitter or Facebook for authentication. We will not store any of the data, and will never share your information publicly,” Klout states.

The decision to allow people to delete their profile info is definitely the right move for Klout — especially since it uses personal information in its business dealings. However, I suspect that many people will still react negatively, since it does require you to opt-out of the service.

And despite the potential backlash, the practice of adding observer profiles to Klout only enhances its ability to accurately measure a person’s true influence online. I’m sure the company weighed these two factors before deciding to go forward.

“Ultimately, we’re only using publicly available information about people that plenty of other sites are already taking advantage of,” Klout CEO Joe Fernandez told VentureBeat. He added that anyone who is fearful of personal information gathered on a Klout observer profile should re-examine the privacy setting on Facebook and Twitter. “The only information that shows up on a (observer) profile is stuff that those users have decided to allow the world to see.”

Founded in August 2008, the San Francisco-based company previously raised an initial $1.5 million round, followed by a second $8.5 million round in January 2011. The company has a total of $11 million in funding to date from Kleiner, Greycroft Partners, ff Asset Management and others. Klout is rumored to be raising a new $30 million fund that would value the startup at an estimated $200 million.

[via LockerGnome]

Finding a good list of the best this or that can be a hard task. This is because they are often hard to make. In order to round up a great list of products, the reviewer must first test each product, find and test competitors, categorize the products and finally write the review. There’s time cost, locating and acquiring the product cost and cost to pay someone to write the review. That all adds up. Top10 uses the cheaper favorite, crowdsourcing, to develop these lists almost instantly.

For Top10 to do this, users must first create their own top ten lists. The company then curates a list based on users’ predefined preferences found on their personal, topic-specific lists. For instance, a list of “Top 10 Pixar Movies” was aggregated from 67 different users who all created their own best of pixar lists.

Top10 was founded in 2011, to much concern that the product wouldn’t be able to hold up next to competitors. Dave McClure of 500 Startups showed his hesitance at the Launch Conference where Top10 was unveiled by saying that top 10 lists aren’t new and in order to succeed a company like Top10 will need great game mechanics.

Sonali De Rycker, a partner at Accel Partners, however, describes Top10 as solving a major pain point for consumers.

The company plans to use the funding to expand on its product, along with marketing and new hires.

Top10 company is headquartered in London. Other investors involved in this round include Founder Collective, Idealab, Forward Venture Partners and Shakil Khan.

Delicious is perhaps the premiere popular website bookmarking service. It allows people to save links from around the web and organize them using like-minded tags. The service saw little improvement since its purchase by  Yahoo in 2005 — something both Hurley and Chen aim to change.

The duo said they will be overhauling the site’s design as well as the systems used to tag and organize links. The site will also be refocusing to be a place where users can go to see the most recent links shared around topical events, reviews and tips, the Times reports.

However, the service’s core focus will remain true to the original version of Delicious, according to the report.

Chen and Hurley said the new version of Delicious, which will roll out to the public later this year, will eventually feature sharply focused advertising using the social data collected by link sharing.

Blippy’s “pivot” (to use the popular Silicon Valley term for “Well, that idea didn’t work,”) is outlined in detail in a post on TechCrunch, and the company’s chief executive Ashvin Kumar emailed me to confirm that the story is correct. The big message: After a few early spikes due to media coverage, Blippy’s traffic never took off, so it refocused on reviews, and now the company is refocusing again by working on some unidentified social commerce products that it’s not ready to talk about yet.

This is a pretty big change from the excitement around this area last year. Not only did the idea get the attention of Philip Kaplan (who co-founded ad network AdBrite and tech blog Fucked Company), who joined as a co-founder, but Blippy also raised funding from Sequoia Capital and Charles River Ventures, two of the top venture firms around. Meanwhile, TellMe co-founder Angus Davis launched a similar (if slightly more private) startup called Swipely, which had the backing of Index Ventures, First Round Capital, and Greylock Partners.

The lingering question about both companies, of course, was whether anyone outside a small audience of tech-savvy oversharers actually wanted to tell their friends about what they were buying. These doubts were exacerbated when the credit card data of some Blippy users leaked onto Google. So far, at least, the service hasn’t garnered much interest. In trying to find a new way to appeal to users, Swipely also announced a new mission in March to help shoppers save money at local businesses.

“We don’t think people want to share their purchases,” Davis said at the time. “Period.”