Activity 14: Stereometry

Multiple 2D views allow the computation of depth information. In this activity, we use two identical cameras to capture and reconstruct a 3D object. Images are essentially 3D objects projected onto a 2D frame. From object points at (x,y,z), an image is reduced to (x,y) with z projected as a function of x and y and camera object geometry. In applications such as microscopy or terrain imaging, it is sometimes more informative to preserve the depth information z. In this way, the 3D image may be inspected at different view angles. Stereo imaging is the technique we will use in this experiment and it is inspired by how our two eyes allow us to discriminate depth.

From similar triangles, the following equations are generated. If done on several points on the image, we can reconstruct the object's 3D shape using the z's calculated.

source: M. Soriano, A14 - Stereometry.pdf

Shown below are two images of a simple 3D object that I captured using an Olympus Stylus 770SW digital camera. The vertices of the Rubik's cube will serve as reference points. From figure 1, b was measured to be 30mm using a ruler.
____________________________________________________________________

Next, we calibrate our camera using the algorithm in activity 11 to find the focal length f and the distances from the camera center, x1 and x2. The matrix A was calculated to be,
A = [
-12.892329
8.19365
-1.2825517
121.45776
-5.9037883
-6.0874536
12.008296
80.296042
-0.0214988
-0.0197136
-0.0140380
1]

We now get A(1:3,1:3) and factorize using RQ factorization in order to convert the matrix into the upper diagonal matrix K. We also need to ensure that the factorized A(3,3) element is 1 by dividing the whole matrix by K(3,3).RQ factorization code by Michael Overton
(source: http://www.cs.nyu.edu/faculty/overton/software/uncontrol/rq.m)

function [R,Q]= rq(A)

m = size(A,1);
n = size(A,2);
if n < m
error('RQ requires m<=n')
end

p = mtlb_fliplr(mtlb_eye(m));
Atp = A’*P;
[Q2,R2] = qr(Atp);
bigperm = [P zeros(m,n-m; zeros(n-m,m) mtlb_eye(n-m)];
q = (Q2*bigperm)’;
r = (bigperm*R2*P)’;


K =
[-14.756525, 0.0554842, 121.46413;
0, 14.493566, 80.290264;
0, 0, 1]

We can see that the camera center is located at xo = 121.46413 and yo = 80.290264. The actual size of the image is 256x217. Thus, we can say that the camera center is relatively close to the image center (considering only x since we only need x1 and x2, i.e. 121.5*121.5 = 243, which is close to 256). From this value of xo, we can now solve for x1 and x2 for different points on the cube. Then the height z will follow from the formula. We plot our results in 3D. (see Scilab help on splin2d, example 2)
____________________________________________________________________
Reconstructed images of the Rubik's cube:
Here we have 3 views of the reconstructed image. Although the edges are curved, the shape of the cube is clearly observed. A better reconstruction may be possible by locating more points. For the resulting images, only the 7 visible corner points in the cube were used in finding the correspondence between the 1st and 2nd images.

____________________________________________________________________

I give myself 9 points for this activity since the reconstruction was properly done but the result was not very convincing. The result looked like a cube with smoothened corners. Thanks to JC Nadora, Julie Dado, and Jeric Tugaff for their help in the use of splin2d and interp2d.