Monday, May 05, 2008

Whole-Body Gaming

New software makes it easier to build games controlled by a user's body position.

By Erica Naone

Solid motion: Softkinetic builds software that helps designers incorporate data from depth-sensing cameras into their applications. One of the system’s functions is to take a cloud of points from a depth map made by the camera and consolidate it into a human-shaped volume, shown above, with identifiable body parts. Developers can then use the volume as the basis for a video avatar, a figure used to represent a player in a 3-D game. Information about the position and velocity of the player’s body parts can be used to control the game. Credit: Softkinetic

Multimedia   See Softkinetic’s system in action.

The massive success of the Nintendo Wii proved the appeal of motion-controlled gaming. Now Softkinetic, a company based in Belgium, is working to let video-game players use a wider range of more-natural movements to control the on-screen action. Softkinetic's software is meant to work with depth-sensing cameras, which can be used to determine a player's body position and motions. "You don't need a controller in your hand," says CEO Michel Tombroff. "You don't need to wear a special outfit. You just come in front of the camera in your living room, and you start playing by moving your entire body."

Attempts to commercialize gestural interfaces date back to at least the late 1980s and the Power Glove, an accessory for the original Nintendo Entertainment System. Many such systems, however, have been defeated by the need for awkward, bulky accessories; others just didn't work that well.

The Wii controller was the field's first success. But the motions it requires can sometimes feel stiff and unnatural, and it's sensitive only to gestures made by the hand in which it's held. Depth-sensing cameras, on the other hand, can pick up gestures made by a variety of body parts, Tombroff says. They can also be tuned to pick up motions more precisely. Designing programs that work with the cameras, however, is difficult: translating depth measurements into a map of a human figure, and determining what motions that figure is making, are computationally daunting tasks. This is where Softkinetic comes in.

Softkinetic's technology started out as research at the University of Brussels, in Belgium, aimed at exploring the user interfaces made possible by stereoscopic cameras, which sense depth by using two input sources, in much the way that the human brain perceives depth by comparing data from two eyes. The group created Softkinetic in mid-2007 and has adapted its research to work with newer depth-sensing cameras as well. Tombroff explains that the newer cameras have better commercial prospects because they've done away with the need for two input sources. As a consequence, they're smaller, with cheaper parts, and easier to incorporate into existing devices such as laptops.

Tombroff says that the new cameras sense depth by using infrared light in one of two ways. First, the camera might send out infrared light and receive the reflections of that light off objects in a room. The sending and receiving information can be compared to determine details of position and depth around the camera. Alternatively, the camera could project a grid of infrared light onto a room, and calculate the positions of objects based on how the grid is distorted.