I am reading this description here concerning calibration of the Kinect v1. I am trying to get an understanding of how it all works.
From what I understand, the kinect has an near-infrared projector and a near infrared camera. From the description:
The IR camera and the IR projector form a stereo pair with a baseline of approximately 7.5 cm. The IR projector sends out a fixed pattern of light and dark speckles, shown below. The pattern is generated from a set of diffraction gratings, with special care to lessen the effect of zero-order propagation of a center bright dot (see the PrimeSense patent). Depth is calculated by triangulation against a known pattern from the projector. The pattern is memorized at a known depth. For a new image, we want to calculate the depth at each pixel. For each pixel in the IR image, a small correlation window (9x9 or 9x7, see below) is used to compare the local pattern at that pixel with the memorized pattern at that pixel and 64 neighboring pixels in a horizontal window (see below for how we estimate the 64-pixel search). The best match gives an offset from the known depth, in terms of pixels: this is called disparity. The Kinect device performs a further interpolation of the best match to get sub-pixel accuracy of 1/8 pixel (again, see below for how this is estimated). Given the known depth of the memorized plane, and the disparity, an estimated depth for each pixel can be calculated by triangulation
I do not understand the statement: Depth is calculated by triangulation against a known pattern from the projector. The pattern is memorized at a known depth.
What is the known pattern and what does the author mean the pattern is memorized at a known depth? I'm imagining the projector is projecting some kind of pattern to an object, and then the NIR camera looks at the pattern at a given patch, and tries to match it to some patch in the original pattern (as if the original pattern were projected on a flat plane with no distortion). Then I am not sure what mathematical techniques are used to relate the distorted pattern to the original pattern to compute disparity (and hence depth).
Any insights appreciated.