TC50: CrowdFlower offers rich analytics for Mechanical Turks

September 15, 2009 | Kim-Mai Cutler | Comments |
|

crowdflowerCrowdFlower, a company which launched today at the TechCrunch50 conference in San Francisco, is a service that helps businesses find workers for menial tasks and rate them based on performance.

Mechnical Turks, as they’re sometimes called, are people who do tasks that require human intelligence and that a software program can’t replicate — like tagging objects in photos or checking videos for pornography. They do this for fun, or for pennies for every task they complete. No one has ever satisfactorily explained why, but that doesn’t stop them.

CrowdFlower helps businesses find mechanical Turks to finish whatever they need. An aspiring taskmaster can go to the site, list a task and add, for example, the data they need sorted from an Internet feed, or in an Excel file. They can build forms to specify certain responses, or make the task more like a quiz.

CrowdFlower then finds workers off Amazon’s Mechanical Turk, gaming site Gambit, Samasource (which finds workers in developing countries) and other services. CrowdFlower comes out of Dolores Labs, famous on the Internet for its porn-sorting work.

What makes CrowdFlower a bit different is that it feeds back rich analytics on the tasks. If you’re create a task, it shows you how likely results are to be true, depending on how many people agree with the result. You can zoom in to find out why certain items have lots of disagreements. You can go back and look at what the individual users have done to see whether they’re historical trustworthy.

Judges at the conference provided some feedback to the company.

Tim O’Reilly: Mechanical Turk is gaining stream as a fundamental Internet service. There’s still a lot of ignorance about this as a business. Seems to be more of a marketing challenge, not a technical challenge.

Click here for more startup news coming out of the TechCrunch50 conference.

Next Story:
Previous Story:

Email
Print
Bookmark
Google
Delicious
Facebook
Twitter
StumbleUpon
MySpace
Digg
Live
More…

Tags: ,

Photo of Kim-Mai Cutler

About the Author, Kim-Mai Cutler

Kim-Mai was born and raised a stone's throw from Apple headquarters in Cupertino by a devout Hewlett-Packard family. After attending UC Berkeley, Kim-Mai worked for Bloomberg, The Wall Street Journal and Dow Jones Newswires in New York, Los Angeles, London and Buenos Aires. Follow her on Twitter at @kimmaicutler, and follow VentureBeat on Twitter at @venturebeat.

  • Actually, CrowdFlower is a service offered by Dolores Labs.
  • Thanks, I'll just make the wording more explicit then.
  • The core technology behind CrowdFlower is the ability to continuously assess the accuracy of the work coming in and filter out the noise. The greatest difficulty in crowdsourcing work is the large variation in worker accuracies. CrowdFlower automatically infers trust for each individual on the task at hand to gather high accuracy results from any worker, from any labor pool (eg Facebook, Mechanical Turk, Samasource, Livework).
  • Now you can control large group of workers remotely and monitor their quality and improve where needed.
  • Samasource has refugees using CrowdFlower to do work (and earn income) from Kenya. We're really excited about the power of CrowdFlower to transform complex workflows into manageable tasks that can be done by workers anywhere in the world.
blog comments powered by Disqus