Kaggle launches competition to help Microsoft Kinect learn new gestures

What if it was as easy to add a new gesture to Microsoft Kinect as it is to create a keyboard shortcut?

Big data startup Kaggle just launched a machine learning competition to develop an algorithm that Kinect can use to learn to recognize a new gesture from a single example. There are cash prizes of $10,000 for the top algorithms, and Microsoft may pay up to $200,000 to license them.

Let’s say you want to use a new gesture to control video editing software. Show Kinect a chopping gesture once and using the new algorithm it should recognize that gesture in future. You can then map the new gesture to, for example, the “cut” feature. Adding new gestures currently involves a considerable amount of manual work.

Kaggle runs Big Data challenges, which involve developing new machine learning algorithms or finding useful patterns in large data sets. “Gesture technology is not a mature area,” said Kaggle’s president and chief scientist Jeremy Howard. While learning from a single example, or “one-shot-learning”, is not an new problem in the machine learning world, “It’s not solved well enough,” Howard explained.

The competition is not specifically for gaming applications. The algorithm could be used for tasks like interpreting sign language, controlling robots or appliances or recognizing when a patient in a hospital room is in distress. Each application has its own vocabulary of gestures, which is what makes it important that Kinect can learn them rather than have to be programmed explicitly for each gesture.

Competitors get a set of videos clips showing examples of gestures (one example of each gesture) that they use to develop their algorithm. The gestures come from different vocabularies such as diving signals and American Sign Language. During development, the algorithm can be tested on a set of 1-5 examples of each learned gesture to see how well it performs at recognizing them. A final set of data, the validation set, can be used by competitors to evaluate their progress and compare their work with others. A real-time leaderboard shows participants their current standing based on their validation set predictions.

“Winners are generally not domain experts but machine learning experts,” said Howard. He should know. Before he became its president, Howard was Kaggle’s most successful competitor. Kaggle is popular with data scientists not only as a way of proving their skill but also because Kaggle gives them access to new data sets that are otherwise hard to find. Kaggle’s community of data scientists comprises thousands of PhDs from quantitative fields such as computer science, statistics, econometrics, maths and physics. “We often have people working on 2-3 competitions at the same time,” said Howard.

Kaggle has already had a couple of big wins. A competition run by NASA and the European Space Agency resulted in a 350 percent improvement on the existing algorithms for mapping dark matter. Insurance company Allstate improved 340 percent on the best available algorithm for evaluating how likely it is that someone will be killed or injured in an accident involving a particular car model.

The biggest competition currently running is the $3 million Heritage Health prize, which aims to predict and prevent unnecessary hospitalizations, thereby saving the U.S. up to $30 billion per year in unnecessary hospitalizations. “This is one of the few times that health data has been anonymized and made public,” Howard told me.

Let the games begin.

Kaggle was founded in 2010, is based in San Francisco and has raised $11 million in funding to date.