Activity 17: Basic Video Processing

Video, whether analog or digital, is a string of still images shown in rapid succession such that an impression of movement is perceived by an observer. Therefore, all image processing techniques we've learned so far may be applied on image frames of a video. The advantage of video is that it allows us access to another physical dimension, time. Thus, the dynamics and kinematics of a system can be extracted from video.

In this activity, we aim to extract some kinematic constants and variables like gravitational constant, position, and velocity upon taking a video of an event like dropping a ball or watching a spring reach equilibrium when initially stretched. These extracted variables will be plotted against time and the kinematic constants will be calculated. The results will be compared against analytical values.

In order to do this with Scilab, we need to install a video processing toolbox. However, the SIVP toolbox cannot read more than two frames from an avi video so we have to use other software to achieve our goal. These are Windows Movie Maker - for capturing the video and saving it directly on the PC, Stoik Video Converter - for converting the .wmv file generated by Movie Maker into an .avi file, and Virtual Dub - to parse the .avi file into image sequences in either .jpg or .bmp formats. Once stored as individual images, the position, shape or area can be extracted using image processing techniques. We loop through the image sequence to get the time variation of the variables of interest.

source: M. Soriano, A17 - Basic Video Processing.pdf

The kinematic event that I used was using a steel ruler as a physical pendulum. From this, I can calculate for some quantities like the center of oscillation, its position after some time, the period of oscillation, etc.
____________________________________________________________________

Physical Pendulum
http://en.wikipedia.org/wiki/Pendulum#Physical_pendulum

The simple pendulum assumes that the rod is massless and that the bob is small, so has negligible angular momentum in itself. A physical pendulum has significant size and mass, and hence a significant moment of inertia.

A physical pendulum behaves like a simple pendulum but the expression for the period is modified.

where:

I is the moment of inertia of the pendulum about the pivot point

L is the distance from the center of mass to the pivot point

m is the mass of the pendulum

For a physical pendulum, since this has the same form as the simple pendulum, it is possible to define the center of oscillation. This is the distance from the pivot point, at which, if all the mass of the pendulum was concentrated (and thus forming a simple pendulum), would give the pendulum the same period.

The moment of inertia in this case is given by



with height h, width w, and mass m (Axis of rotation at the end of the plate)
http://en.wikipedia.org/wiki/List_of_moments_of_inertia
____________________________________________________________________

Before calculating the desired quantities, we first need to process our images. This activity utilizes most of the techniques we have learned so far. Step by step, these are:

1. Separate the video per frame using other software.
2. White balance the images.
3. Normalize the color.
4. Do color segmentation on the object (in our case, the ruler that acted as a physical pendulum).
5. Convert to a B&W image with the proper threshold value.
6. Do morphological operations to close holes and enhance the image.

After these, we can calculate for the centroid proceed in solving for the desired quantities numerically.
____________________________________________________________________

RESULTS
ORIGINAL, WHITE BALANCED, NORMALIZED, SEGMENTED, B&W, MORPHED IMAGES














measured quantities: m = 175 g, length = 0.63 m, width = 0.03 m

Analytic solutions:
Moment of inertia = 175g(0.63m*0.63m)/3 + 175g(0.03m*0.03m)/12 = 23.165625 g m^2
Center of oscillation = I/mL = 23.165625 m/(175*0.315) = 0.4202381 m
Period of oscillation = 2*pi*sqrt(23.165625/(175*9.8*0.315)) = 1.3011116 s

We now proceed in solving these quantities numerically from the data.

Length of ruler = 105 pixels --> 0.63 m
Width of ruler = 6 pixels
Moment of inertia = 175g(0.63m*0.63m)/3 + 175g((6*0.63m/105)*(6*0.63m/105))/12
= 23.1714 g m^2

%error = 0.02493

Center of oscillation = 23.1714 m/(175*(52.5*0.63/105)) = 0.4203429 m

%error = 0.02494

Period of oscillation:
Our video was captured with a fps of 15. The graph of the position vs time of the centroid of the ruler is shown below.
The wave is not a perfect sinusoid due to some air resistance present. The peaks and troughs are relatively equally spaced between each other so the period can be calculated. Peak to peak, the heights for one frame is given from the data:
94, 93, 90, 87, 83, 78, 73.5, 68.5, 65, 64.5, 64.5, 65, 68.5, 73, 77.5, 83.5, 87.5, 90, 92.5, 92.5
Thus, one period was captured in 20 frames. If 15 frames = 1 second, then using ratio and proportion, 20 frames = 1.3333333 s.

%error = 2.47647
____________________________________________________________________

chdir('act17images\');
img = 216; //total no. of frames in video

//open cropped image for color segmentation
im2 = imread('act17crop1.jpg');
wid = zeros(img,1); leng = zeros(1,1);

for i = 1:img
im = imread('images'+string(i)+'.jpg'); //open images
white = im(1:120,120:160,:);

//white balance
r = mean(white(:,:,1));
g = mean(white(:,:,2));
b = mean(white(:,:,3));
im(:,:,1) = im(:,:,1)/r;
im(:,:,2) = im(:,:,2)/g;
im(:,:,3) = im(:,:,3)/b;
im = im/max(im);
m = find(im>1);
im(m) = 1;

//color normalization
I = im(:,:,1)+im(:,:,2)+im(:,:,3);
im(:,:,1) = im(:,:,1)./I;
im(:,:,2) = im(:,:,2)./I;
im(:,:,3) = im(:,:,3)./I;

//histogram backprojection
R = linspace(0,1,32); G = R;
P = zeros(32,32);
[x2,y2] = size(im2);
for j = 1:x2
for k = 1:y2
xr = find(R <= im2(j,k,1));
xg = find(G <= im2(j,k,2));
P(xr(length(xr)),xg(length(xg))) = P(xr(length(xr)),xg(length(xg))) + 1;
end
end

P = P/sum(P);
[x1,y1] = size(im);
bpj = zeros(x1,y1);
for j = 1:x1
for k = 1:y1
xr = find(R <= im(j,k,1));
xg = find(G <= im(j,k,2));
bpj(j,k) = P(xr(length(xr)), xg(length(xg)));
end
end
bpj = bpj/max(bpj);

//convert to B&W
bpj = im2bw(bpj, 0.02);

//morphological operations
strel = ones(4,1);
bpj = erode(bpj,strel); //opening operation
bpj = dilate(bpj,strel);
bpj = dilate(bpj,strel); //closing operation
bpj = erode(bpj,strel);

//get centroid
[p,q] = find(bpj == 1);
wid(i) = (max(q)+min(q))/2;
leng(i) = (max(p)+min(p))/2;
end

//plot position vs time
plot([1:img]/15,wid); //consider fps of video
title('physical pendulum');
xlabel('time (seconds)');
ylabel('relative amplitude');
____________________________________________________________________

I give myself 10 points for this activity since the calculated quantities were close to their analytic values. My collaborators were Jeric Tugaff and Mark Leo Bejemino.