Activity 16: Color Image Segmentation

In image segmentation, a region of interest (ROI) is picked out from the rest of the image such that further processing can be done on it. Selection rules are based on features unique to the ROI. One such feature is color. Color has been used to segment images of skin regions in face and hand recognition, land cover in remote sensing, and cells in microscopy.

In this activity, we aim to do image segmentation using 2 different methods: parametric and non-parametric probability distribution estimation. First, we convert image RGB to normalized chromaticity coordinates (NCC) and then apply the 2 techniques.

We can transform the image RGB values to NCC by,
Since r + b + g = 1, then b = 1 - r - g and it is enough to represent chromaticity by just two coordinates, r and g. When segmenting 3D objects it is better to first transform RGB into rgI. The figure below shows the r-g color space.When r and g are both zero, b = 1. Therefore, the origin corresponds to blue while points corresponding to r=1 and g=1 are points of pure red and green, respectively. Any color therefore can be represented as a point in NCC space. Thus the pixel chromaticities of a red ball will only occupy a small blob in the red region regardless of its shading variation.

source: M.Soriano, A16 - Color Image Segmentation.pdf

The histogram of a green chair is shown below.
____________________________________________________________________

Parametric method:
Original & Probability image

Here, the original object is a green chair. We cropped a region of interest (center part of the backrest) as our color model and tried to see where it matched that of the original image. The pixel values of the resulting image is the probability that a pixel will have the same color as the model. This method segmented the image well and we can infer that the bright parts of the seat are of the same color as that of the backrest.
____________________________________________________________________

Histogram backprojection (non-parametric) method:
Original & Probability image

Histogram backprojection is one technique where based on the color histogram, a pixel location is given a value equal to its histogram value in chromaticity space. We implemented this and the resulting image was generated. Upon comparison with the result from the parametric method, there are more white pixels that correspond to the cropped color model. We can really see where the similar colors lie. The segmentation is better using the 2nd method. This also has the advantage of faster processing because no more computations are needed, just a look-up of histogram values. In the parametric method, the mean and standard deviation of r and g must be found in order to calculate a Gaussian distribution that tests the probability of pixel membership to the ROI.
____________________________________________________________________

stacksize(4e7);

im1 = imread('green chair.jpg');
im2 = imread('green chair crop.jpg');
//imshow(im1);

R1 = im1(:,:,1); G1 = im1(:,:,2); B1 = im1(:,:,3);
I1 = R1 + G1 + B1;
R2 = im2(:,:,1); G2 = im2(:,:,2); B2 = im2(:,:,3);
I2 = R2 + G2 + B2;

r1 = R1./I1; g1 = G1./I1; b1 = B1./I1;
r2 = R2./I2; g2 = G2./I2; b2 = B2./I2;

//parametric estimation
ur = mean(r2); sr = stdev(r2);
ug = mean(g2); sg = stdev(g2);

pr = 1.0*exp(-((r1-ur).^2)/(2*sr^2))/(sr*sqrt(2*%pi));
pg = 1.0*exp(-((g1-ug).^2)/(2*sg^2))/(sg*sqrt(2*%pi));

prob = pr.*pg;
prob = prob/max(prob);
//imshow(prob,[]);

//2d histogram
r = linspace(0,1,32); r = g;
P = zeros(32,32);
[x,y] = size(im2);
for i = 1:x
for j = 1:y
xr = find(r <= im2(i,j,1));
xg = find(g <= im2(i,j,2));
P(xr(length(xr)), xg(length(xg))) = P(xr(length(xr)), xg(length(xg)))+1;
end
end
P = P/sum(P);
//surf(P);

//backprojection
[x,y] = size(im1);
recons = zeros(x,y);
for i = 1:x
for j = 1:y
xr = find(r <= im1(i,j,1));
xg = find(g <= im1(i,j,2));
recons(i,j) = P(xr(length(xr)), xg(length(xg)));
end
end
imshow(recons, []);
____________________________________________________________________

I give myself 10 points for this activity since the colored image was properly segmented using 2 different methods. Histogram backprojection is more efficient since less calculations are needed, only a look-up table. It also produced a better probability image. Thanks to Jeric Tugaff for his help in the histogram backprojection code.