Activity 11 - Camera Calibration

In this activity, we captured an image of a checker board with 1 inch by 1 inch dimension of each square. These 3D object is then transform it into a 2D image by the camera. The objective of this activity is to compute the Transformation Matrix present into the camera that transforms from world to camera coordinates given by the simple equation:

Equation 1

where G is the transformation matrix, xi and yi are the image coordinates and xw, yw and zw are the world coordinates.

25 differents spots on the image (xi's and yi's) and the corresponding world coordinates of the spots (xw, yw and zw) were taken. Plugging in it into the equation given by:

Equation 2

Note: xo, yo and zo are the same as the xw, yw and zw.

Figure 1 . Image used in calibration

Note: The white dots are the points used for the calibration process and the red cross marks are the points that were predicted

The column matrix with components a is the transformation matrix.The values of a can easily be calculated by the equation:

Equation 3

where Q matrix is the leftmost matrix in Equation 2 and d matrix is the rightmost matrix in Equation 2.

The values of the transformation matrix are:

- 11.158755
16.994741
- 0.3046630
81.972117
- 4.4211357
- 3.0139446
- 19.251234
271.84956
- 0.0126526
- 0.0067189
0.0023006

To verify if the transformation matrix is correct, 12 different spots(which were not used in the calibration) from the image were then taken and predicted the location of the spots using the transformation matrix given by the equation:

Equation 4

Note: a34 is set to 1.
The results of the predicted location with comparison with the actual location using locate function in scilab is shown below:


In this activity, I will give myself a grade of 10 because the % errors I got are considerably low. I also acknowledge Abraham Latimer Camba for lending me the picture I used in this activity and Rafael Jaculbia for teaching how to use the locate function in scilab.

Appendix:
Source code in scilab


Image_mat1 = imread("F:\AP 186\activity 11\new.jpg");
gray = im2gray(Image_mat1);
imshow(gray);
Image_mat = locate(25,flag=1)';
Object_mat = fscanfMat("F:\AP 186\activity 11\Object.txt");
No = size(Object_mat);
Ni = size(Image_mat);
No2 = No(1)*2;
O = [];
for i = 1:No2
if modulo(i,2) == 1
k = ((i+1)/2);
O(i,1) = Object_mat((i+1)/2,1);
O(i,2) = Object_mat((i+1)/2,2);
O(i,3) = Object_mat((i+1)/2,3);
O(i,4) = 1;
O(i,5:8) = 0;
O(i,9) = -1*(Object_mat(k,1)*Image_mat(k,1));
O(i,10) = -1*(Object_mat(k,2)*Image_mat(k,1));
O(i,11) = -1*(Object_mat(k,3)*Image_mat(k,1));
end
if modulo(i,2)==0
O(i,5) = Object_mat(i*0.5,1);
O(i,6) = Object_mat(i*0.5,2);
O(i,7) = Object_mat(i*0.5,3);
O(i,8) = 1;
O(i,1:4) = 0;
O(i,9) = -1*(Object_mat(i*0.5,1)*Image_mat((i+1)/2,2));
O(i,10) = -1*(Object_mat(i*0.5,2)*Image_mat((i+1)/2,2));
O(i,11) = -1*(Object_mat(i*0.5,3)*Image_mat((i+1)/2,2));
end
end
d = [];
for i = 1:No2
if modulo(i,2)==1
d(i) = Image_mat(((i+1)/2),1);
end
if modulo(i,2) == 0
d(i) = Image_mat(i/2,2);
end
end

a = (inv(O'*O)*O')*d;
New_o = fscanfMat("F:\AP 186\activity 11\Object2.txt");
w = size(New_o);
y = [];
z = [];
for i = 1:w(1)
y(i) = (a(1,1)*New_o(i,1) +a(2,1)*New_o(i,2)+a(3,1)*New_o(i,3)+a(4,1))/(a(9,1)*New_o(i,1) +a(10,1)*New_o(i,2)+a(11,1)*New_o(i,3)+1);
z(i) = (a(5,1)*New_o(i,1) +a(6,1)*New_o(i,2)+a(7,1)*New_o(i,3)+a(8,1))/(a(9,1)*New_o(i,1) +a(10,1)*New_o(i,2)+a(11,1)*New_o(i,3)+1);
end

imshow(gray);
f = locate(12,flag=1);


Source:
[1] Activity manual and lectures provided by Dr. Maricor Soriano on her Applied Physics 186 class

A10 – Preprocessing Handwritten Text

In this activity, we are to remove an image cropped from a larger image which have horizontal lines by filtering. The original image is shown in Figure 1.

Figure 1. Original Image

After filtering the image in its Fourier Space, I binarized the image and did some closing morphological operation using a 2x2 square image as the structuring element. Finally, I labelled the image and used the hotcolormap function in scilab to separate the labelled parts of the image.

Figure 2. Result

I will give myself a grade of 9 because some of the horizontal lines were not removed.


Appendix:

Source Code in Scilab:

a= imread("G:\AP 186\Activity 10\image.jpg");
b = im2gray(a);
g = imread("G:\AP 186\Activity 10\filter3.bmp");
h = im2gray(g);
filter1 = fftshift(h);
c= (fft2(b));
scf(1);imshow(b,[]);
d = (fft2(filter1.*c));
I = abs(fft2(fft2(d)));
scf(2);imshow(I,[]);
w = I/max(I);
q = floor(255*w);
mat = size(q);
newpict = q;
for i = 1:mat(1) // Number of rows
for j = 1:mat(2) // Number of columns
if newpict(i,j) <>
newpict(i,j) = 0;
else
newpict(i,j) = 1;
end
end
end
SE_sq = [];
for i = 1:2
for j =1:2
SE_sq(i,j) = 1;
end
end
//
//SE_cross(1:3,1:3)=0;
//SE_cross(2,1:3) = 1;
//SE_cross(1:3,2) = 1;
//////////////////////////
p = abs(newpict-1);
Image_close1 = erode(dilate(p,SE_sq),SE_sq);//Closing
//Image_open1 = dilate(erode(Image_close1,SE_cross),SE_cross);//Opening
Invert = abs(Image_close1 - 1);
scf(3);imshow(Invert,[]);
o = bwlabel(Image_close1);
scf(4); imshow(o,[]);xset("colormap",hotcolormap(255));
scf(5); imshow(Invert,[]);

Activity 9: Binary Operations

In image-based measurements, it is often desired that the region of interest (ROI) is made well segmented from the background either by edge detection (defining contours) or by specifying the ROI as a blob. Raw images must generally be preprocessed to make them suitable for further analysis.
Binarizing an image simplifies the separation of background from ROI. However, the optimum threshold must be found by examing the histogram.
In this activity, paper cut outs (with equal areas) were scanned in a flat bed scanner. The image was cropped in different positions and saved in a different image file. Each cropped image file was then processed by separating it from the background and binarizing the image file. After the binarization process the image file, morphological operations were done to eliminate pixels which were not to be the region of interest the ; the closing operation and the opening operation, which are just combinations of erosion and dilation operation. The structuring element used in the program was a 4x4 square image. The original image file is shown in Figure 1.

Figure 1. Original scanned image of cut outs

Problem encountered in approximating the area was the overlapping of circles which cannot be separated by my program. These overlaps gave me large area which obviously I can simply eliminate in my data. I limit my data on areas from 400 pixels to 600 pixels because greater or less than this range were considered to be noise or overlapping. Histogram of the areas is shown on Figure 2.Figure 2. Histogram of approximate area

The mean of the histogram is 541.4821 pixels and the standard deviation is 17.3184.

In this activity, I acknowledge the help of Rafael Jaculbia on how to get the histogram of my data in Microsoft Office Excel 2007. I will give myself a grade of 10 because I believe that I did all my best in this activity.


Appendix:

Source code in Scilab:



pict = imread("E:\AP 186\Act9\cropped11.jpg");
pict2 = im2gray(pict);
mat = size(pict);
newpict = pict2;
for i = 1:mat(1) // Number of rows
for j = 1:mat(2) // Number of columns
if newpict(i,j) < se_sq =" [];" i =" 1:4" j ="1:4" image_close1 =" erode(dilate(newpict,SE_sq),SE_sq);//Closing" image_open1 =" dilate(erode(Image_close1,SE_sq),SE_sq);//Opening" image_close2 =" erode(dilate(Image_open1,SE_sq),SE_sq);//Closing" image_open2 =" dilate(erode(Image_close2,SE_sq),SE_sq);//Opening" labelled =" bwlabel(Image_open2);//" count =" []" i =" 1:max(Labelled)" w =" find(Labelled" k =" size(w);">

Source:

[1] Activity 9 manual provided by Dr. Maricor Soriano

Activity 8: Morphological Operations

Mathematical morphology is a set-theoretical approach to multi-dimensional digital signal or image analysis, based on shape. The signals are locally compared with so-called structuring elements S of arbitrary shape with a reference point R.

The eroded image of an object O with respect to a structuring element S with a reference point R, , is the set of all reference points for which S is completely contained in O.

The dilated image of an object O with respect to a structuring element S with a reference point R, , is the set of all reference points for which O and S have at least one common point.

Here are some examples of images that are eroded and dilated:

I would like to acknowledge Rafael Jaculbia for lending me his image templates I used in getting the erosion and dilation. I also acknowledge Billy Narag and Abraham Latimer Camba for helping understand idea of dilation and erosion.

For this activity, I would give myself a grade of 9 for the reason that I am not sure of some of my results because I did not correspond on my guess image.

Reference:
http://ikpe1101.ikp.kfa-juelich.de/briefbook_data_analysis/node178.html

Enhancement in the Frequency Dom

Part 1:(FFT of 2D sine function)

Increasing the frequency of the 2D sine function increases the separation of the peak frequency in the FFT space and by rotating the sine function by some angle also rotates the peak frequency in the FFT space.
Multiplication of two sine function create a four peaks in the FFT space. The spacing of the peaks depends on the frequency of the of each sine. If the two sine functions have different frequencies, the peaks will not form a perfect square.

Part 2: (Finger Print)
Source code for finger print improvement:
a = imread("E:\AP 186\activity 7\finger ko.bmp");//image read
b = im2gray(a);
IM=fft(b,-1);
h1=mkfftfilter(b,'butterworth1',5);
h2=mkfftfilter(b,'butterworth2',30);
h=(h1-h2)*255;//Create filter
c = fft2(b);
apergrayshift = fftshift(h);
FR = apergrayshift.*(c);
IRA = (fft2(FR));
scf(1)
imshow(abs(fft2(fft2(IRA))),[]);
scf(2)
imshow(b,[]);
Part 3: (Space image improvement)

There is a given a image having vertical lines with uniform spacing, we can eliminate these vertical lines by taking the 2D Fourier Transform. By creating a filter which blocks the cross line on the Fourier Space of the original image, we can eliminate the vertical line present on the image. The FFT, the filter and the resulting image is shown below:

Collaborators:
Rafael Jaculbia, Elizabeth Anne Prieto

In this activity,I will give myself myself a grade of 7 because I believe that I did not obtain the right result for the second part.

Fourier Transform Model of Image

Part A: Fourier Transform of an Image

The Fourier Transform (FT) is one among the many classes of linear transforms that recast information into another set of linear basis. In particular, FT converts a signal with dimension X into a signal with dimension 1/X. If the signal has dimensions of space, a Fourier Transform of the signal will be inverse space or spatial frequency. The FT of an image f(x,y) is given by:

F ( f x , f y )=∫∫ f ( x , y) exp(−i 2pi( f x x f y y )) dx dy

This part of the activity discusses mainly on getting the fourier transform of the image and returning to its original image (meaning getting the fourier transform twice) . Getting the fourier transform twice will give an inverted original image. The simple source code is written below (at the appendix).

Figure 1. Fourier transform of 2d(image)

Part B: Convolution

The convolution between two 2-dimensional functions f and g is given by
h  x , y =∫∫ f  x ' , y '  g  x – x ' , y – y ' dx ' dy' .
or simply h = f*g.
The convolution is a linear operation which means that if f and g are recast by linear transformations, such as the Laplace or Fourier transform, they obey the convolution theorem:

H=FG

where H , F and G are the transforms of h, f and g, respectively. The convolution is a “smearing” of one function against another such that the resulting function h looks a little like both f and g.
In this part of the activity, a circular image with different sizes were convolved to an image (VIP). As I decrease the size of the circular aperture, the resulting image after getting the convolution blurs more. Meaning, the smaller the diameter of the circle, the less the resolution of the resulting image.
Figure 2. Convolution of image to a circular image

Part C: Template Matching by Correlation

The correlation p measures the degree of similarity between two functions f and g.
The more identical they are at a certain position x,y, the higher their correlation
value. Therefore, the correlation function is used mostly in template matching or
pattern recognition.
For this part of the activity, I made an image with written words,"THE RAIN IN SPAIN STAYS MAINLY IN THE PLAIN". I also made an image template written,"A" with the same font size as of the phrase. The two image has exactly the same size. After getting their correlation and returning the original image by fourier transform, the resulting image highlights all A's.

Figure 3. Matching the template to the original image

Part D: Edge detection

In this part of the activity, correlation was utilized (same as Part C) but the template used here are different nxn matrices wherein the total sum of the matrix is equal to zero. The convolved images for different patterns of matrices detect the edge of the image and the direction of the edges depends mainly on the pattern of the matrix. For example, for the first matrix (Pattern 1), all same values in the matrix are in the horizontal direction. The edge of the resulting image also in the horizontal direction. But for the third pattern, the direction of the numbers with the same values are both in the horizontal and vertical direction thus forming edges both vertical and horizontal (including the diagonal direction).

Figure 4. Matrix pattern for the edges and the resulting image after convolutiion

For this activity, I proudly dive myself a grade of 10 because I got all my results properly.

Collaborator:
Billy Narag

Appendix:
Source Codes:
2D Fourier Transform;

a = imread("C:\Documents and Settings\AP186user15\Desktop\A.bmp");//File open
b = im2gray(a);//set to grayscale
Fb = fft2(b);//Fast Fourier Transform
Fb2 = (abs(fft2(fft2(a))));//Twice Fourier Transform
scf(1)
imshow(abs(Fb),[]);
scf(2)
imshow(fftshift(abs(Fb)),[]);
scf(3)
imshow(Fb2,[]);

Convolution;

a = imread("C:\Documents and Settings\AP186user15\Desktop\circle.bmp"); //open file
c = imread("C:\Documents and Settings\AP186user15\Desktop\VIP.bmp"); // open file
b = im2gray(a); // converts image to grayscale
d = im2gray(c); // converts image to grayscale
Fshifta = fftshift(b); // Shift the FFT
Fc = (fft2(a));//Fourier Transform of the image
FCA = Fc.*(Fshifta);// Convolution
InFFT = fft2(FCA);//Fourier transforming the image back
Fimage = abs(InFFT); // getting the absolute value
imshow(Fimage,[]);


Correlation;

a = imread("E:\AP 186\Activity 6\A2.bmp");//open image file
c = imread("E:\AP 186\Activity 6\The rain.bmp");//open image file
b = im2gray(a);// converts the image file into grayscale
d = im2gray(c);//
Fa = fft2(b);// fast fourier transform
Fc = fft2(d);
Fc_conj = conj(Fc); // getting the complex conjugate
FCA = Fa.*(Fc_conj);//convolution
InFFT = fft2(FCA);
Fimage = abs(fftshift(InFFT));
imshow(Fimage,[]);


Edge Detection;

a = imread("C:\Documents and Settings\AP186user15\Desktop\Activity 6\VIP.bmp");//open image file
pattern = [-1 -1 -1; -1 8 -1; -1 -1 -1,];//matrix pattern
b = im2gray(a);//converts the image into grayscale
c = imcorrcoef(b,pattern);//convolution of the two matrices with different sizes
imshow(c,[]);


Source:

[1]Activity manual provided by Dr. Jing Soriano

Physical Measurements from Discrete

2D Discrete Fourier transform can be obtained by the formula given by:

The function(image) has a size of NxM. Using the function fft2, it performs the fourier transform along the rows and ftt performs the fourier transform along the columns.


Answers to questions:

a) In a light bulb a Fmax of the llight bub is 120 Hz. To get the threshold sampling interval, use the equation T= 1/(Fmax*2) which is equal to the value 4.16ms.


b) Increasing the total number of samples N increases the height because you increase the number of samples.

Figure1. The left image has lower N and the right image has higher N

c) Decreasing sampling interval Δt decreases the frequency of the signal (if you decrease the total time T) thus shifting the peak to a lower value and increasing the peak (if you increase the total number of samples N).

Figure 2. The left image has the lower dt

d) Fixing time interval T but increasing the N increases the height of the peak (same as the answer to question b).

In this activity, I will give myself a grade of 8.5 for the reason that few of my answers came from my collaborator.

Collaborator:

Rafael Jaculbia