Activity 17 - Video Processing

In this activity, video processing was done. First, the video is ripped to create images representing frames in the video. The images obtained were then processed.
The video used in this activity is a rolling object on an inclined plane. The objective of this activity is to obtain the acceleration seen from the video and compare it to theoretical values given some parameters.

(I cannot upload the video because of large file size)..

The images obtained from the video were converted to binary images. The centroid of the rolling cylinder for all the images were obtained and plot the distance from the origin versus the frame. The plot is given below:




Conversion of values from pixel to millimeter and frame to time were performed through known values in the video (for example, the size of the rolling cylinder and the frame rate). Curve fitting was then performed to the plot to obtain a polynomial equation of distance that depends on the constant acceleration and velocity which is given by:
d = 331.69 t^2 + 21.002 t^2
from the equation, the acceleration is given by 331.69x2 = 663.38 mm/sec^2 (which is the acceleration along the x - axis since the video was viewed from the top).
Theoretical calculation of the acceleration given by the equations below revealed that for that same parameter, the acceleration is 592.71 mm/sec^2. Large difference from theoretical and experimental values maybe due to the centroid finding of the object.

For this activity, I would give myself a grade of 8. I would like to acknowledge the help of Abraham Latimer Camba for the theoretical computation of the acceleration.

Appendix:

//15 frames per second
I = [];
se = [1 1;1 1];
for i =1:15
I = imread("vid" + string(i) + ".jpeg");
I1 = im2bw(I,0.7);
I2 = erode(dilate(I1,se),se);
I3 = erode(I2,se);
imwrite(I3,"videos" + string(i)+".jpeg");
[x1,y1] = find(I3==1);
x(i)=mean(x1);
y(i)=mean(y1);
end
distancey = y - min(y);
distancex = x - min(x);
distance = sqrt(distancex^2 + distancey^2);
for i = 1:15
velocity(1) = distancey(1);
velocity(i+1) = distance(i+1) - distancey(i);
end
for i = 1:15
acceleration(1) = 0;
acceleration(i+1) = velocity(i+1) - velocity(i);
end
scf(0);plot(distance);
scf(1);plot(velocity);
scf(2);plot(acceleration);

Activity 16 - Color Image Segmentation

In this activity, a sample from the whole image is picked out to get the Region Of Interest (ROI) from the image.
Example of this is the human skin recognition.
There are two basic techniques in segmentation; (1) Probability Distribution Estimation and (2) Histogram Backprojection.

The image and the used in this activity is shown below:
Figure 1. The original image
Figure 2. Sample from the original image

Probability Distribution Estimation

In this technique, a sample from the original image is cropped from the original image. Per pixel, the r value and the g of both the original image and the cropped image are normalized by dividing it to the sum of r, g and b values of that pixel. The probability that the the r value and the g value of the original image belong to the ROI is given by the equation:

Equation 1

The resulting ROI is shown below:Figure 3

Histogram Backprojection

In this technique, the 2D histogram of the portion of the original image was obtained. The obtained histogram was normalized to obtain the probability distribution function(PDF). The r and the g values of the original image per pixel were back-projected (or replaced) by finding the r and the g values from the PDF. The resulting image is shown below:

Figure 4

Comparing the two techniques, for me the Probability Distribution Estimation is a better technique for segmentation because of the expected ROI was obtained. In contrast to the Histogram Back-projection, not all the expected ROI was obtained.


For this activity, I will give ,myself a grade of 8 because of the late submission.
I would like to acknowledge Abraham Latimer Camba for helping me in the Histogram Back-projection part.



Appendix:
Source code


//First: Segmentation via probability
I = imread("F:\AP 186\act16\pic.jpg");
I2 = imread("F:\AP 186\act16\small.jpg");
n1 = size(I);
scf(1);imshow(I);
r1 = I(:,:,1)./(I(:,:,1)+I(:,:,2)+I(:,:,3));
g1 = I(:,:,2)./(I(:,:,1)+I(:,:,2)+I(:,:,3));
b1 = I(:,:,3)./(I(:,:,1)+I(:,:,2)+I(:,:,3));
r2 = I2(:,:,1)./(I2(:,:,1)+I2(:,:,2)+I2(:,:,3));
g2 = I2(:,:,2)./(I2(:,:,1)+I2(:,:,2)+I2(:,:,3));
b2 = I2(:,:,3)./(I2(:,:,1)+I2(:,:,2)+I2(:,:,3));
pr = (1/(stdev(r2)*sqrt(2*%pi)))*exp(-1*((r1-mean(r2)).^2/(2*stdev(r2))));
pg = (1/(stdev(g2)*sqrt(2*%pi)))*exp(-1*((g1-mean(g2)).^2/(2*stdev(g2))));
new = (pr.*pg);
scf(3);imshow((new),[]);

//Second: Segmentation via Histogram


r2_1 = floor(r2*255);
g2_1 = floor(g2*255);
n = size(r2);
Hist = zeros(256,256);
for i = 1:n(1)
for j =1:n(2)
x = r2_1(i,j)+1;
y = g2_1(i,j)+1;
Hist(x,y) = Hist(x,y) +1;
end
end
scf(5);plot3d(0:255,0:255,Hist);
Hist = Hist/max(Hist);

scf(2);imshow(log(Hist+0.0000000001),[]);
r1 = round(r1*255);
g1 = round(g1*255);
for i = 1:n1(1)
for j = 1:n1(2)
T(i,j) = Hist(r1(i,j)+1,g1(i,j)+1);
end
end
scf(4);imshow(T,[]);
imwrite(new/max(new),"F:\AP 186\act16\1_1.jpg");
imwrite(T,"F:\AP 186\act16\2_2.jpg");

Activity 15 - Color Image Processing

In this activity, images having unbalanced colored images were enhanced by;

(1) White Balancing,
(2) Gray World Balancing

White Balancing:

White balancing technique uses a known white object from the image for balancing. The red, green and blue of the RGB values of the known white object in the image is used as "NORMALIZING" or as divider for all the RGB values of all the pixels in the image.

Gray World Balancing:

Gray balancing technique uses the mean of all the red, green and blue values present in the image as the divider for all the RGB values of the pixels present in the image.

Here are some examples of images that were white balanced and gray balanced:

For this activity, I will give myself a grade of 7 out of 10 because of the late submission.
I would like to acknowledge Mark Leo for helping me debug errors on my program.

Appendix:

Source code for White Balancing created in Scilab:

I = imread("C:\Documents and Settings\AP186user15\Desktop\act15\outside - daylight.jpg");
imshow(I);
n = size(I);
RGB = round(locate(1,flag=1));
r = I((RGB(1)),(RGB(2)),1);
g = I((RGB(1)),(RGB(2)),2);
b = I((RGB(1)),(RGB(2)),3);
Ibal(:,:,1) = I(:,:,1)/r;
Ibal(:,:,2) = I(:,:,2)/g;
Ibal(:,:,3) = I(:,:,3)/b;

index=find(Ibal>1.0);
Ibal(index)=1.0;
//Inew(:,:,1) = Ibal(:,:,1)/max(I(:,:,1));
//Inew(:,:,2) = Ibal(:,:,2)/max(I(:,:,2));
//Inew(:,:,3) = Ibal(:,:,3)/max(I(:,:,3));
imwrite(Ibal,"C:\Documents and Settings\AP186user15\Desktop\act15\outside - daylight_bal.jpg");

Source code for Gray World Balancing:
I = imread("C:\Documents and Settings\AP186user15\Desktop\act15\o- incandescent.jpg");
//imshow(I);
//n = size(I);
//RGB = round(locate(1,flag=1));
r = mean(I(:,:,1));
g = mean(I(:,:,2));
b = mean(I(:,:,3));
Ibal(:,:,1) = I(:,:,1)/r;
Ibal(:,:,2) = I(:,:,2)/g;
Ibal(:,:,3) = I(:,:,3)/b;

index=find(Ibal>1.0);
Ibal(index)=1.0;
Ibal = 0.75*Ibal;
//Inew(:,:,1) = Ibal(:,:,1)/max(I(:,:,1));
//Inew(:,:,2) = Ibal(:,:,2)/max(I(:,:,2));
//Inew(:,:,3) = Ibal(:,:,3)/max(I(:,:,3));
imwrite(Ibal,"C:\Documents and Settings\AP186user15\Desktop\act15\o- incandescent_bal_gray.jpg");

Activity 14 - Streometry

A13 – Photometric Stereo

In this activity, an object illuminated by same point source with 4 different location in reference to the object. The resulting images have different shadings. The objective of this activity is to predict the original shape of the image.
Letting the matrix V be the three dimensional position of the point source for different positions given and matrix I be the resulting image depending on the position of the light source. For each image with size (NxN), the images are then reshaped to (N^2 x 1) to form a column matrix. All of the reshaped images are then combined to form a single matrix I.
Matrix g contains all the information about the original shape about object and can be solved using Equation 1:

Equation 1

Getting then normal vector of the g matrix, I used the Equation 2:

Equation 2

To know the shape of the object, we get the function f first by partial differentiation of normal vectors (x and y) with respect to z using Equation 3:Equation 3

Finally, function f can be obtained using Equation 4:

Equation 4

The result of the shape of the image is shown below:

Figure 1

For this activity, I will give myself a grade of 10 because all of the objectives of the activity were met.

Appendix (Source code in Scilab):


loadmatfile ('C:\Documents and Settings\AP186user15\Desktop\ap18657activity13\Photos.mat');
browsevar();
I(1,:) = (I1(:))';
I(2,:) = (I2(:))';
I(3,:) = (I3(:))';
I(4,:) = (I4(:))';

V(1,:) = [0.085832 0.17365 0.98106];
V(2,:) = [0.085832 -0.17365 0.98106];
V(3,:) = [0.17365 0 0.98481];
V(4,:) = [0.16318 -0.34202 0.92542];

g = (inv(V'*V)*(V'))*I;
N = size(g);
gmag = [];
for i = 1:N(2)
gmag(i) = sqrt(g(1,i)**2 + g(2,i)**2 + g(3,i)**3)+0.0000000001;
end
n(1,:) = g(1,:)./gmag(1,:);
n(2,:) = g(2,:)./gmag(1,:);
n(3,:) = g(3,:)./gmag(1,:)+0.0000000001;
Fx = -n(1,:)./n(3,:);
Fy = -n(2,:)./n(3,:);
f = cumsum(Fx,1)+cumsum(Fy,2);
newf = matrix(f,[128,128]);
plot3d(1:128,1:128,newf)

Activity 11 - Camera Calibration

In this activity, we captured an image of a checker board with 1 inch by 1 inch dimension of each square. These 3D object is then transform it into a 2D image by the camera. The objective of this activity is to compute the Transformation Matrix present into the camera that transforms from world to camera coordinates given by the simple equation:

Equation 1

where G is the transformation matrix, xi and yi are the image coordinates and xw, yw and zw are the world coordinates.

25 differents spots on the image (xi's and yi's) and the corresponding world coordinates of the spots (xw, yw and zw) were taken. Plugging in it into the equation given by:

Equation 2

Note: xo, yo and zo are the same as the xw, yw and zw.

Figure 1 . Image used in calibration

Note: The white dots are the points used for the calibration process and the red cross marks are the points that were predicted

The column matrix with components a is the transformation matrix.The values of a can easily be calculated by the equation:

Equation 3

where Q matrix is the leftmost matrix in Equation 2 and d matrix is the rightmost matrix in Equation 2.

The values of the transformation matrix are:

- 11.158755
16.994741
- 0.3046630
81.972117
- 4.4211357
- 3.0139446
- 19.251234
271.84956
- 0.0126526
- 0.0067189
0.0023006

To verify if the transformation matrix is correct, 12 different spots(which were not used in the calibration) from the image were then taken and predicted the location of the spots using the transformation matrix given by the equation:

Equation 4

Note: a34 is set to 1.
The results of the predicted location with comparison with the actual location using locate function in scilab is shown below:


In this activity, I will give myself a grade of 10 because the % errors I got are considerably low. I also acknowledge Abraham Latimer Camba for lending me the picture I used in this activity and Rafael Jaculbia for teaching how to use the locate function in scilab.

Appendix:
Source code in scilab


Image_mat1 = imread("F:\AP 186\activity 11\new.jpg");
gray = im2gray(Image_mat1);
imshow(gray);
Image_mat = locate(25,flag=1)';
Object_mat = fscanfMat("F:\AP 186\activity 11\Object.txt");
No = size(Object_mat);
Ni = size(Image_mat);
No2 = No(1)*2;
O = [];
for i = 1:No2
if modulo(i,2) == 1
k = ((i+1)/2);
O(i,1) = Object_mat((i+1)/2,1);
O(i,2) = Object_mat((i+1)/2,2);
O(i,3) = Object_mat((i+1)/2,3);
O(i,4) = 1;
O(i,5:8) = 0;
O(i,9) = -1*(Object_mat(k,1)*Image_mat(k,1));
O(i,10) = -1*(Object_mat(k,2)*Image_mat(k,1));
O(i,11) = -1*(Object_mat(k,3)*Image_mat(k,1));
end
if modulo(i,2)==0
O(i,5) = Object_mat(i*0.5,1);
O(i,6) = Object_mat(i*0.5,2);
O(i,7) = Object_mat(i*0.5,3);
O(i,8) = 1;
O(i,1:4) = 0;
O(i,9) = -1*(Object_mat(i*0.5,1)*Image_mat((i+1)/2,2));
O(i,10) = -1*(Object_mat(i*0.5,2)*Image_mat((i+1)/2,2));
O(i,11) = -1*(Object_mat(i*0.5,3)*Image_mat((i+1)/2,2));
end
end
d = [];
for i = 1:No2
if modulo(i,2)==1
d(i) = Image_mat(((i+1)/2),1);
end
if modulo(i,2) == 0
d(i) = Image_mat(i/2,2);
end
end

a = (inv(O'*O)*O')*d;
New_o = fscanfMat("F:\AP 186\activity 11\Object2.txt");
w = size(New_o);
y = [];
z = [];
for i = 1:w(1)
y(i) = (a(1,1)*New_o(i,1) +a(2,1)*New_o(i,2)+a(3,1)*New_o(i,3)+a(4,1))/(a(9,1)*New_o(i,1) +a(10,1)*New_o(i,2)+a(11,1)*New_o(i,3)+1);
z(i) = (a(5,1)*New_o(i,1) +a(6,1)*New_o(i,2)+a(7,1)*New_o(i,3)+a(8,1))/(a(9,1)*New_o(i,1) +a(10,1)*New_o(i,2)+a(11,1)*New_o(i,3)+1);
end

imshow(gray);
f = locate(12,flag=1);


Source:
[1] Activity manual and lectures provided by Dr. Maricor Soriano on her Applied Physics 186 class

A10 – Preprocessing Handwritten Text

In this activity, we are to remove an image cropped from a larger image which have horizontal lines by filtering. The original image is shown in Figure 1.

Figure 1. Original Image

After filtering the image in its Fourier Space, I binarized the image and did some closing morphological operation using a 2x2 square image as the structuring element. Finally, I labelled the image and used the hotcolormap function in scilab to separate the labelled parts of the image.

Figure 2. Result

I will give myself a grade of 9 because some of the horizontal lines were not removed.


Appendix:

Source Code in Scilab:

a= imread("G:\AP 186\Activity 10\image.jpg");
b = im2gray(a);
g = imread("G:\AP 186\Activity 10\filter3.bmp");
h = im2gray(g);
filter1 = fftshift(h);
c= (fft2(b));
scf(1);imshow(b,[]);
d = (fft2(filter1.*c));
I = abs(fft2(fft2(d)));
scf(2);imshow(I,[]);
w = I/max(I);
q = floor(255*w);
mat = size(q);
newpict = q;
for i = 1:mat(1) // Number of rows
for j = 1:mat(2) // Number of columns
if newpict(i,j) <>
newpict(i,j) = 0;
else
newpict(i,j) = 1;
end
end
end
SE_sq = [];
for i = 1:2
for j =1:2
SE_sq(i,j) = 1;
end
end
//
//SE_cross(1:3,1:3)=0;
//SE_cross(2,1:3) = 1;
//SE_cross(1:3,2) = 1;
//////////////////////////
p = abs(newpict-1);
Image_close1 = erode(dilate(p,SE_sq),SE_sq);//Closing
//Image_open1 = dilate(erode(Image_close1,SE_cross),SE_cross);//Opening
Invert = abs(Image_close1 - 1);
scf(3);imshow(Invert,[]);
o = bwlabel(Image_close1);
scf(4); imshow(o,[]);xset("colormap",hotcolormap(255));
scf(5); imshow(Invert,[]);