For each 2D pixel on the image plane, it not only records that pixel's color, i.e., the color of the original 3D point, but also that 3D point's distance along its projection line. There are approaches to estimate that distance for many pixels in an image by using multiple images or good old guesswork, but they have their limitations.Ī 3D camera like a Kinect provides the missing bit of information necessary for 3D reconstruction. While each pixel in a 2D image defines a line from that pixel through the lens back out into 3D space, and while the original 3D point that generated the pixel must lie somewhere on that line, the distance that 3D point 'traveled' along its line is lost in projection. Normally, once 3D objects are projected to a 2D plane that way, it is impossible to go back and reconstruct the original 3D objects.

Kinect vs Regular 2D Camera Any camera, 2D or otherwise, works by projecting 3D objects (or people.), which you can think of as collections of 3D points in 3D space, onto a 2D imaging plane (the picture) along straight lines going through the camera's optical center point (the lens). It is important to understand the difference between 3D cameras like the Kinect on one hand, regular (2D) cameras on the other hand, and so-called '3D cameras' - actually, stereoscopic 2D cameras - on the third hand (ouch). It also contains an active-sensing depth camera using a structured light approach (using what appears to be an infrared LED laser and a micromirror array), which also sends (depth) images of 640*480 pixels 30 times a second (although it appears that not every pixel is sampled on every frame). The Kinect contains a regular color camera, sending images of 640*480 pixels 30 times a second.

The motivation for this project was to convert the Kinect into a 3D camera by combining the depth and color image streams received from the device, and projecting them back out into 3D space in such a way that real 3D objects inside the cameras' field of view are recreated virtually, at their proper sizes (see Figures 1 and 2).įigure 1: Video of a user interacting with a pre-recorded life-size 'holographic' avatar of himself in the KeckCAVES virtual reality environment.įigure 2: Video showing the Augmented Reality Sandbox, which uses a Kinect camera to capture a 3D model of the surface of sand in a sandbox, and a projector to project a dynamic topographic map back onto the sand. The Kinect is intended to be used as a controller-free game controller, tracking the body or bodies of one or more players in its field of view. It contains an array of microphones, an active-sensing depth camera using structured light, and a color camera. The Kinect is an accessory for Microsoft's Xbox game console.

Kinect for xbox 360