Activity 11: Camera Calibration

In this activity, we will try to model the physical processes involved in the geometric aspects of image formation. We will try to calibrate the camera by looking at corresponding world points and image points.

An image is a projection of brightness values from surfaces existing in 3D world space to 2D sensor space. One whole dimension is lost in the projection. Nevertheless, under certain constraints it is possible to recover the lost information. The process involves carefully calibrating the camera by taking several views of a scene.

First, we take a picture of a checkerboard pattern, also known as a Tsai grid, using an ordinary digital camera. We pick an origin and 20-25 more points in the image while taking note of its real world coordinates. The squares are 1x1 inches thick. Using the locate function in Scilab, we get the image coordinates of the same 20-25 points we picked. Next, we set up the matrix Q shown by equation 13 for values of i from 1 to 25. We then solve for a using equation 15 and then plug its elements into equations 11 and 12. For the last two equations, we set a_34 to be equal to 1.


RESULTS:

Shown below is the Tsai grid together with the points (red) we wish to use in the calibration. The origin is set at (0,0,0). The left side of the board is x, the right side is y, and the vertical axis is z.

(xo,yo,zo)
1. (0,0,2)
2. (0,0,5)
3. (0,0,9)
4. (0,0,12)
5. (2,0,1)
6. (5,0,2)
7. (3,0,4)
8. (2,0,7)
9. (7,0,6)
10. (4,0,6)

11. (6,0,9)
12. (2,0,10)
13. (8,0,11)
14. (1,0,3)
15. (0,0,0)
16. (0,6,12)
17. (0,8,10)
18. (0,3,10)
19. (0,2,1)
20. (0,4,3)
21. (0,7,6)
22. (0,5,7)
23. (0,6,2)
24. (0,4,6)
25. (0,2,8)

Using locate, their corresponding 2D image coordinates are as follows:
(yi, zi)
(130.53181, 67.545109)
(130.53181, 114.07882)
(131.00665, 178.18139)
(130.53181, 229.46344)
(106.31529, 45.702754)
(65.954416, 51.40076)
(92.545109, 90.811966)
(105.36562, 143.04368)
(33.665717, 115.50332)
(78.300095, 122.151)
(47.910731, 172.95821)
(104.89079, 194.32574)
(15.147198, 209.52042)
(118.18613, 79.890788)
(130.05698, 37.155745)
(211.25356, 228.51377)
(241.16809, 188.62773)
(168.04368, 192.90123)
(152.849, 45.702754)
(178.96486, 70.868946)
(223.12441, 115.50332)
(194.63438, 136.87085)
(206.03039, 47.127255)
(179.91453, 122.151)
(155.22317, 159.18803)

We set up matrix Q given by equation 13 and solving for a using equation 15 will give us the elements of matrix a.
a =
- 13.945461
8.8516947
- 0.6362905
130.1377
- 4.0454015
- 4.1159857
14.796702
37.040714
- 0.0178636
- 0.0176363
- 0.0052799

Using these values, we plug them into equations 11 and 12 and see whether these new yi's and zi's are close to the values previously obtained.

(new_yi, new_zi)
(130.24043, 67.345268)
(130.39869, 114.03467)
(130.61791, 178.7028)
(130.78881, 229.11771)
(105.9554, 45.617241)
(65.69975, 51.556447)
(92.680325, 90.881065)
(105.45811, 142.91477)
(34.035999, 115.62425)
(78.64952, 122.2471)
(48.193941, 172.64721)
(105.19649, 194.0998)
(14.486398, 209.56016)
(118.26943, 80.084525)
(130.1377, 37.040714)
(211.37148, 228.57471)
(241.39165, 188.6588)
(168.09928, 193.06864)
(153.42664, 45.448497)
(179.10783, 71.109685)
(222.85396, 114.82174)
(194.25085, 137.20802)
(205.9424, 47.461688)
(180.14164, 121.80884)
(154.74534, 159.5493)

We can compute for the accuracy of the values we obtained by finding the difference between the original and new coordinates.

diff =
- 0.2913883, - 0.1998413
- 0.1331196, - 0.0441513
- 0.3887400, 0.5214104
0.2569913, - 0.3457294
- 0.3598862, - 0.0855131
- 0.2546660, 0.1556874
0.1352158, 0.0690990
0.0924834, - 0.1289110
0.3702816, 0.1209236
0.3494252, 0.0961056
0.2832094, - 0.3110003
0.3057051, - 0.2259333
- 0.6608001, 0.0397375
0.0832921, 0.1937365
0.0807186, - 0.1150312
0.1179217, 0.0609362
0.2235590, 0.0310715
0.0556003, 0.1674038
0.5776353, - 0.2542572
0.1429685, 0.2407392
- 0.2704486, - 0.6815880
- 0.3835267, 0.3371792
- 0.0879896, 0.3344329
0.2271150, - 0.3421524
- 0.4778311, 0.3612629

We get a mean of 0.26442074 difference in yi and a mean of 0.218553368 difference in zi. Thus, we can say that our calibration is very accurate since there is only a difference of less than a pixel in the values gathered.
____________________________________________________________________

im = gray_imread('tsai grid.jpg');
imshow(im);
x = locate(25,1);
yi = x(1,:);
zi = x(2,:);

xo = [0 0 0 0 2 5 3 2 7 4 6 2 8 1 0 0 0 0 0 0 0 0 0 0 0];
yo = [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 8 3 2 4 7 5 6 4 2];
zo = [2 5 9 12 1 2 4 7 6 6 9 10 11 3 0 12 10 10 1 3 6 7 2 6 8];

for i = 1:length(xo)
Q((2*i)-1,:) = [xo(i) yo(i) zo(i) 1 0 0 0 0 -(yi(i)*xo(i)) -(yi(i)*yo(i)) -(yi(i)*zo(i))];
Q(2*i,:) = [0 0 0 0 xo(i) yo(i) zo(i) 1 -(zi(i)*xo(i)) -(zi(i)*yo(i)) -(zi(i)*zo(i))];
d((2*i)-1,:) = yi(i);
d(2*i,:) = zi(i);
end

a = inv(Q'*Q)*Q'*d;

for j = 1:length(xo)
yi_new(j,:) = ((a(1)*xo(j))+(a(2)*yo(j))+(a(3)*zo(j))+a(4))/((a(9)*xo(j))+(a(10)*yo(j))+(a(11)*zo(j))+1);
zi_new(j,:) = ((a(5)*xo(j))+(a(6)*yo(j))+(a(7)*zo(j))+a(8))/((a(9)*xo(j))+(a(10)*yo(j))+(a(11)*zo(j))+1);
end
____________________________________________________________________

To verify if the calibration is correct, we predict the image coordinates of some cornerpoints of the checkerboard which were not used in the calibration (yellow). The number of points should be greater than 11. These points are, in world coordinates:

(3,0,8)
(0,1,5)
(0,4,4)
(0,6,5)
(0,3,9)
(0,7,11)
(4,0,12)
(6,0,1)
(8,0,5)
(2,0,3)
(5,0,5)
(7,0,2)

We predict that these points will be transformed into the following by using equations 11 and 12.
(92.030252, 158.4637)
(142.06384, 111.83289)
(179.44842, 87.812934)
(207.50173, 99.481483)
(167.82, 175.48687)
(226.15495, 208.91806)
(77.116722, 229.33722)
(51.635668, 31.057816)
(18.52981, 94.693389)
(105.79334, 77.32756)
(64.717941, 102.67897)
(36.148858, 44.327313)

//prediction code
xo = [3 0 0 0 0 0 4 6 8 2 5 7];
yo = [0 1 4 6 3 7 0 0 0 0 0 0];
zo = [8 5 4 5 9 11 12 1 5 3 5 2];

a = [-13.945461 8.8516947 -0.6362905 130.1377 -4.0454015 -4.1159857 14.796702 37.040714 -0.0178636 -0.0176363 -0.0052799];

for j = 1:length(xo)
yi(j,:) = ((a(1)*xo(j))+(a(2)*yo(j))+(a(3)*zo(j))+a(4))/((a(9)*xo(j))+(a(10)*yo(j))+(a(11)*zo(j))+1);
zi(j,:) = ((a(5)*xo(j))+(a(6)*yo(j))+(a(7)*zo(j))+a(8))/((a(9)*xo(j))+(a(10)*yo(j))+(a(11)*zo(j))+1);
end

After running the original code, these points were converted into:
(91.919444, 158.17825)
(141.77925, 111.69473)
(179.29545, 87.811292)
(207.46461, 99.490062)
(167.2938, 175.12682)
(225.77969, 208.89718)
(76.901337, 228.70553)
(51.993604, 31.260393)
(18.750844, 94.88024)
(105.83348, 77.396374)
(64.879166, 102.73954)
(36.482614, 44.520168)

Again, we get the difference between the two sets of values and find the mean difference for yi and zi.
mean difference for yi = 0.234702
mean difference for zi = 0.179843167
____________________________________________________________________

I give myself 10 points for this activity since the results of the camera calibration was good. The values differed by a value that is less than 1 pixel. Good predictions resulted because of this. My collaborators for this activity were Jeric Tugaff and Julie Dado.