Projector-Kinect Calibration Tutorial

Tutorial contributed by Christian Parsons (http://www.camara-lucida.com.ar/tutorials/calibration).

This tutorial explains how to calibrate a kinect and a projector. The goal of the calibration is to find the extrinsic and intrinsic parameters for both kinect and projector.

After performing the calibration you will end up with some yml files to use in your own software or to make a custom software with the Cámara Lúcida’s openframeworks addon which is prepared to load and parse the files and render using the data. @see https://github.com/chparsons/Camara-Lucida

This tutorial assumes you are using the libfreenect backend. It was tested on Ubuntu and OS X.

1. Calibrate the kinect

You should get a yml file with its intrinsic parameters. See Calibration to do so.

2. Make a board for the kinect-projector calibration

You should print the following chessboard:

Here’s the chessboard to print: https://github.com/chparsons/rgbdemo/blob/master/data/projector_chessboard.svg

The chessboards you see on the corners of the board are printed chessboards. The empty space in the center is there to project another chessboard image with the projector.

You can print a sheet of paper of the whole board size or print and paste 4 patterns on the board corners as far as they are fixed precisely according to the diagram measures above. It should look something like this:

Here’s the pattern you have to project with the projector:

3. Grab a bunch of board poses using RGB Demo? software

As explained in step #2. As you can see in the image above, you have to move the chessboard and grab images like a standard camera calibration. Don’t get too many images since calibration can fail. I usually grab between 20 and 30 poses.

4. Run calibrate_projector binary to perform calibration

In a nutshell, you have to pass these arguments:

  • The folder where the grabbed images are located
  • The kinect calibration file: —kinect, by default it will look for a kinect_calibration.yml file in the same directory.
  • The output calibration yml file name: by default it will output to projector_calibration.yml

e.g. 

build/bin/calibrate_projector grab1 --kinect kinect_calibration.yml --output projector_calibration.yml --projector-width 1024 --projector-height 768

There are other arguments as well, to see a more detailed list of arguments run 

calibrate_projector --help

and you will see this: 

Usage: calibrate_projector [ string] [--kinect string] [--output string] [--projector-width integer] [--projector-height integer] [--pattern-width integer] [--pattern-height integer] [--pattern-pixels integer] [--pattern-size float] [--offsetX_3x3 float] [--offsetY_3x3 float] [--offsetX_3x4 float] [--offsetY_3x4 float] [--offsetX_3x5 float] [--offsetY_3x5 float] [--offsetX_3x6 float] [--offsetY_3x6 float] 

REQUIRED:
                   string  RGBD images directory  ['(null)']

Optional:
            Switch Type    Help [default value] 

          --kinect string  Input YAML filename  ['kinect_calibration.yml']
          --output string  Output YAML filename  ['projector_calibration.yml']
 --projector-width integer Projector width in pixels  [1280]
--projector-height integer Projector height in pixels  [800]
   --pattern-width integer Pattern width (number of inner squares)  [7]
  --pattern-height integer Pattern height (number of inner squares)  [10]
  --pattern-pixels integer Pattern side length (in pixels)  [40]
    --pattern-size float   Square size in used defined scale  [0.04]
     --offsetX_3x3 float   Horizontal position of the 3x3 pattern  [0.66]
     --offsetY_3x3 float   Vertical position of the 3x3 pattern  [0]
     --offsetX_3x4 float   Horizontal position of the 3x4 pattern  [0]
     --offsetY_3x4 float   Vertical position of the 3x4 pattern  [0]
     --offsetX_3x5 float   Horizontal position of the 3x5 pattern  [0]
     --offsetY_3x5 float   Vertical position of the 3x5 pattern  [0.33]
     --offsetX_3x6 float   Horizontal position of the 3x6 pattern  [0.66]
     --offsetY_3x6 float   Vertical position of the 3x6 pattern  [0.29]
            --help bool    Print this message

5. Check the calibration quality

After calibration you will see in the console a line saying “Average pixel reprojection error” This number should be less than 1. If it’s greater than 1 it means the calibration failed for some reason.

You now have a projector_calibration.yml file which has the instrisics of the projector and the [RT] of the projector WRT the kinect. This file should look something like this: 

%YAML:1.0
proj_intrinsics: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ 1.6746276377167364e+03, 0., 5.9718467266920436e+02, 0.,
       1.6685296742861324e+03, 9.8925503657258091e+01, 0., 0., 1. ]
proj_distortion: !!opencv-matrix
   rows: 1
   cols: 5
   dt: d
   data: [ 0., 0., 0., 0., 0. ]
R: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ 9.9997678973133197e-01, 5.9451992963097649e-03,
       -3.3278527531062849e-03, -6.5488107146271920e-03,
       9.7346562929059666e-01, -2.2873954985547745e-01,
       1.8796480636500421e-03, 2.2875603422683657e-01,
       9.7348196887665417e-01 ]
T: !!opencv-matrix
   rows: 3
   cols: 1
   dt: d
   data: [ 1.7321045734712798e-02, 8.4953662904891175e-02,
       4.2560918063563741e-02 ]
proj_size: !!opencv-matrix
   rows: 1
   cols: 2
   dt: i
   data: [ 1280, 800 ]

6. You can now load the file in your software.