VentureBeat

Inspired by the Nintendo Wii’s clever wand-like game controller, gesture-recognition start-ups are coming out of the woodwork. Israel’s 3DV Systems is one of the contenders and it is showing more of its cards.

The Yokne’am, Israel company said it plans to launch a low-cost gesture-recognition camera for game purposes in 2009. And it has hired a well-known video game veteran as its general manager for the North American market. Charles Bellfield, the new hire, has worked at a variety of video game companies in the past 15 years.

3DV is one of a number of companies that want to make it much easier to control games. It has refined a sophisticated camera, the ZCam, that can detect the gestures a person is making and translates those gestures into controls for a video game. You can thus stand in front of a game machine with the camera, wave your arms about, and control everything in the game.

The idea riffs off the motion sensor in the Nintendo Wii but takes it to a much more precise control through simple gestures. You could, for instance, turn up the volume in a game by making a thumbs-up sign. Or you could drive in a racing game by holding out your hands as if you were gripping a steering wheel. I demoed an early version where I was able to fly a plane in a game by putting my hand up in the air and using it as an imaginary joystick.

The 11-year-old 3DV has been making the cameras for years and still sells a $250,000 version for broadcasters who use it insert images into TV shows in real time. The company refined the technology and took out the costs so that it can apply it to the consumer market. By the end of this year, the company will be mass producing the consumer ZCam cameras and it will launch with games in 2009, Bellfield said in an interview.

I knew Bellfield as the die-hard spokesman for Sega, as that company’s Dreamcast video game console business was sinking. Bellfield didn’t tolerate anyone feeling sorry for poor old Sega. More recently, Bellfield did stints as a marketing executive for game companies Capcom and Codemasters. Think of him as the P.T. Barnum of video games. He will be joined by 3DV’s vice president of interactive entertainment, Rich Flier.

Bellfield said 3DV can come in as a differentiator for video game companies. Innovative ideas such as musical-instrument games “Guitar Hero” and “Rock Band” allow their makers to charge $99 and $169 respectively for their unique games. He said that at a time when low-end Flash games are becoming a commodity and high-end games are becoming increasingly expensive to make, developers should turn to novel approaches to gaming to set themselves apart.

Bellfield said the device will run across a variety of platforms and plugs into a universal serial bus (USB) port on any computer or game device. It will be about the same size as a webcam. Bellfield said the company is talking to a variety of developers and publishers. Those game makers are thinking of either enhancing existing games, such as making shooting games better, or creating new categories of games.

Traditionally, gaming peripherals don’t sell that well. But the Guitar Hero, Rock Band and the Wii itself have turned that notion upside down.

“It’s hard to launch something new, but what we have here is a more disruptive technology,” Bellfield said. “You touch nothing, but you can control everything.”

Demonstrations like those by 3DV have spurred the imagination, but they have also drawn out the competition. Another Israeli competitor claiming to have the best 3-D gesture control system is Prime Sense in Tel Aviv. Other rivals include Softkinetic, XTR, Oblong Industries, Canesta, ThinkOptics and GestureTek. We’ve written about 3DV and these others here and here.

Clearly, not all of these companies are going to survive and they’re going to have live through a long gestation before their products see the light of day. The ultimate blessing would be if a game console maker adopts the newfangled controllers, but the next console systems aren’t going to be out until 2010. Meanwhile, there is a rumor that Microsoft will join the motion-sensor control party with a new controller next week at the E3 show.

While we’ve written several times recently about the progress of next-generation, camera-based game control technologies, including the hefty funding received by Prime Sense and an earlier update on several competing companies, there’s one detail we’ve edged around: When you’ll get to use them for yourself.

That’s because most of the companies developing gesture recognition technology aren’t sure, themselves. The firms developing the 3D cameras that make motion-sensing gaming possible have to work through intermediaries to get their products to the consumer market, and in the Byzantine world of game development, no small company can predict which way giants like Sony and Electronic Arts will lean. However, a company called SoftKinetic recently stepped up to tell me that it thinks the moment may be close.

Here’s the basic idea behind 3D gaming: A camera mounted on your TV or computer captures your movements, and with a combination of sophisticated hardware and software, extrapolates them into three dimensions (the exact details vary by company). The effect is as if you had Wii controllers strapped to your body. So in a boxing game, for example, your punches, as well as the ducking and weaving of your head, will be represented in the game world; or, outside of gaming, you could control an operating system like the one in Minority Report.

The important detail in getting this to work isn’t just good cameras. There are two other, equally important factors. One is price — the average consumer won’t pay a huge amount for any controller, no matter how cool. The second is getting console makers and game developers to run with the technology.

The latter is SoftKinetic’s business. The company makes a software development kit (SDK) and API that makes the switch to gesture recognition simple for game makers. Essentially, SoftKinetic takes the data the camera is returning and translates it into straightforward commands for the gaming people. SoftKinetic has partnerships with the four major camera companies, according to CEO Michel Tombroff, and is hammering out agreements with game development companies to get the technology on the market.

Tombroff, who says his company is the only one around acting as the software intermediary between camera makers and game developers, thinks that the breakthrough will come soon, with the first cameras hitting the market within a year. Progress is going “very quickly”, he told me, before hauling me off to a demo to show off how well the technology works.

To be clear, no major games have been developed yet with gesture recognition technology. However, SoftKinetic has been working in the field for years, for military and industrial applications, and has the consumer SDK done. No game developers themselves, they’ve nonetheless cobbled together a few rough applications to show off the goods.

The demo, done using a Prime Sense camera, impressed me with how well the technology works. One quick game involved ducking and contorting into various positions to avoid oncoming obstacles, while another had me flapping my arms to control an avian character. A third was a kung-fu demo, in which I could kick around blocks and break boards (an example of that is in the embedded YouTube video, while the others are here).

Easily the best demo, though, was a modification the SoftKinetic team did for Quake 3, a game that involves running around gunning down baddies. Running and jumping were simple; for the former, I just had to lean forward, and for the latter, of course, I just hopped in place. The rest took an odd turn. To represent my gun, I had to hold my right arm out straight in front of myself. To fire, I had to flap my left arm up and down, chicken-style. Flapping my right arm switched weapons.

Outside of dancing to 80s hits, I usually try not to let myself look quite that stupid in front of other people, but the oddball motion scheme actually made playing the game more enjoyable than using a controller. Even odder than the scheme itself, though, was the fact that it felt natural within about a minute.

That demo proved the concept to me, but also illustrated where today’s technologies will converge with tomorrow’s. Even if a player could get over the ignominy of flapping their arms around to represent gunning down enemies, they would probably rather hold a controller of some sort — either a fake plastic gun, or a controller like the one the Wii has. Exactly what the end result will look like will probably depend on the game. Tombroff isn’t bothering to speculate — designers, he says, will do a far better job at creating sensible design schemes than his company can.

As long as console makers like Sony and Nintendo can provide various technologies like the Wii controller and 3D cameras cheaply enough, they’ll probably end up combining several, not least because players may not always feel like being active. “With the Wii you can play from the couch — you can fake it. Not with this,” says Tombroff. (So no, you won’t really throw away your Wii.)

Unfortunately, even with SoftKinetic acting as middle-man, it still has to hope that big companies will take up the torch and throw their support behind gesture recognition; Tombroff says he’s hammering out a deal with a major company, but the majors are infamous for canceling plans and breaking promises. Even if they do take up gesture recognition, you won’t have a suite of games by this Christmas. At first, the technology will probably show up in arcades and advergaming (think interactive screens in stores), then move into consoles and computer games.

A possible alternate scenario involves the cameras becoming small enough to fit on, or even in, desktop displays and laptop screens. Another company, 3DV Systems, has said it plans on bringing a camera to the PC accessory market by this Fall, providing a possible work-around to the console market.
.