cancelled

Solutions are requested for six problems about image processing. The problems are beginner level, probably not more than 1-2 hours of work for anyone with some experience in the area. While the assignment document talks about a lot of Matlab, the solutions can also be coded in **C#**. One C# console application that loads the input files, does the required processing and gives the required output files will suffice. ## Deliverables Following are the problems that are asked to be solved. (the full pdf that includes the text below + the figures referred are can be found in files) ** Problem 1: Basic Image Reconstruction** An interlaced camera transmits a full-resolution image by first transmitting the odd lines in a frame, then transmits the even lines of the image in the second consecutive frame. The full resolution image can be recovered by combining odd and even image frames accordingly: In this problem, write code to assemble a full resolution image based on consecutive digitized odd and even image frames captured by an interlaced camera. In addition, it is known that the camera is faulty. The defect only effects the even image frames. In the transmission process, the pixel rows of the even frames appear shifted by the mean value of the row (subtraction of the row's mean from the original row pixel intensities). Your program must correct this disturbance so that the recovered full resolution image doesn't show any disturbances. The following odd (top) and even (bottom) image frames are stored in two “mat?? images, or “dat?? images: even corrupted frame ([login to view URL]); odd frame ([login to view URL]). To load the frames, either use Matlab's "load" function (see 'help load'), or if you are using C, read the data as raw floats ([login to view URL] and [login to view URL]) of size (220x560pixels). After you load the data, check to see if you can view similar images to the ones below (brain). i. Reconstruct the full resolution frame (complete image) by assuming both even and odd fields are non-faulty. Display and print the result. ii. A simple way to correct the image is to use the odd rows mean values and add them to the adjacent even row. Once you are done with the function and have tested it, display and print your result, and answer the following. iii. Compare your results in (i) and (ii) and discuss why does the idea in (ii) works? ** Problem 2: Color camera with a single sensor** Electronic photosensitive sensors are monochromatic. To acquire color images, a color filter array (Bayer) mosaic is often combined with a single photosensitive sensor (e.g., CCD) to create a color camera. Figure 1 visualizes the raw sensor data of such a color camera. Color information is stored in the image according to the utilized Bayer filter mosaic (Fig. 2). In this assignment, you are to write a Matlab m-file script to generate a color image (RGB color space) from the raw sensor data ([login to view URL]). Missing information in the three color channels must be generated by using an appropriate interpolation method. The upper left corner of the Bayer filter mosaic used in combination with the sensor is shown in Fig. 2. Discuss the effect of the Bayer filter mosaic on image resolution: both spatial resolution and color resolution. ** Problem 3: Spatial Filtering** Implement both of the following filters, and apply them on the following three images: [login to view URL] (has Gaussian noise), [login to view URL] (has impulsive noise), and [login to view URL] (has a combination of both noises). For this homework, do not use built-in filtering related functions(either matlab, opencv or such) since here the goal is for you to get practice with spatial neighborhood operations. i. a mean filter ii. Display/print your results for the three images iii. a median filter iv. Display/print your results for the three images v. a mean-median filter with a blending parameter alpha. Play with alpha to obtain the visually most pleasing/denoised (noise removed) image. vi. Display/print your results for the three images. Print your alpha parameter to the titles. vii. Which filter is more appropriate for which image? Explain/discuss why. ** Problem 4: 2D image Interpolation and a 2D Geometric Morph** In this problem, you will write your own functions for interpolation of your images with the two techniques we learned in class (do not use any matlab or opencv built-in functions or alike), and compare the results. Load the [login to view URL] image. 1. To test your routines interpNN and interpBilin, use the following function (this is matlab, but you can easily rewrite in C) to create a 2D displacement field (Ux,Uy) (you can try to create your own different morphs too, but the morph should be smooth, if there are discontinuities in the vector field the result doesn’t look good). % a is your input image % (Ux,Uy): 2D vector field function [Ux,Uy] = createDeformationField2D(a) [ysize,xsize] = size(a); % initialize a 2D vector field/displacement field for morphing Ux=zeros(xsize,ysize); Uy=zeros(xsize,ysize); y0=ysize/2; x0=xsize/2; % pick some center point from the image % create the vector field for y=1:ysize, for x=1:xsize yy=y-y0; xx=x-x0; % design a crazy 2d vector field % surface normal of a fictitious sphere/ellipse nx=xx/5; ny=yy/5; theta = atan2(yy,xx); Ux(x,y) = 2*(abs(cos(theta))) * nx ; Uy(x,y) = 2*(sin(theta)) * ny ; end, end 2. Write 2D Nearest Neighbor interpolation (name your function/routine interpNN). Use interpNN to add the vector field (Ux,Uy) to your image. Display/print the resulting image. 3. Bilinear Interpolation (name your function/routine interpBilin). Use interpBilin to add the vector field (Ux,Uy) to your image. Display/print the resulting image. 4. Compare the resulting morphed Lena images. Which one is visually more pleasing? Did you expect this result? Comment on which regions you observe the differences over the image. I show here a sample portion from the Lena image I morphed with the displacement field above. ** Problem 5: Geometric Transformations** In the next two problems, you will continue to experiment with geometric image transformations: the Rotation and the Affine (as opposed to a general/non parametric smooth morph in Problem 4). For this homework, you are not allowed to use matlab’s maketform or imtransform functions or similarly any OpenCV or other C packaged-ready functions, but write your own code. You will use your interpolation routine from 'bilinear' gray-value interpolation method (Problem 4) to generate the transformed images in the next two problems. Read Problem 6 at the same time, since that has some more explanations on the geometric transformations. 2D Rotation: Load the original image ‘[login to view URL]’. Load the second image ‘[login to view URL]’, which is the original image rotated by an unknown angle. i. Simple 2D rotation estimation method: Use the corners of the rectangle in both images to do this. Calculate the rotation angle teta from each pointwise correspondence between the two images. In this way, you will get 4 different teta estimates using the coordinate point pairs of the rectangle corners. Write down your calculated teta values, and explain the slight difference in teta estimates. ii. Use the average of the 4 rotations, and obtain a single teta value tetaNew. Transform your original image ‘[login to view URL]’ by a 2D rotation with this new angle teta tetaNew. Display and print your result. **Problem 6: Geometric Transformations: Affine Transformation** Imagine that you have been given images of the same scene, taken with different cameras. Suppose that the second camera delivers a geometrically distorted image ([login to view URL]), and that the resulting image can't be directly overlaid on the image generated with the first camera ('[login to view URL]'). In order to perform any sort of analysis of the images, you must first ensure that the images line up; this can be accomplished using geometric image transformations. In this assignment, you are to write a Matlab script/or C routines that will transform image lenaAffine line up with the image '[login to view URL]'. You goal is to try to find the optimal geometric transformation between these two images. First determine appropriate point mappings (corresponding points) between each pair of images. Note that impixelinfo( ) might be useful in this context; impixelinfo( ) can be called after imshow( ), allowing you to use the cursor to get pixel coordinates. i. Estimate the 2D affine transformation matrix parameters (4 of them) by using for instance 5 corresponding features from both of the images. For instance, you can choose the following five features: 1. Lena's hat at the top, 2. Lena’s hat’s rim on the right, 3. the intersection of mirror with black vertical bar at the background, 4-5. pupils of her eyes. In matlab you can use ginput to capture mouse click operations. In C/C++, for instance, you can use OnLButtonDown type events to catch the point locations. ii. Using these 5 point correspondences, set up a simple Linear Least Squares Estimation problem to solve for a11, a12, a21, a22 (Affine matrix A’s elements). Write down the derivations on how you obtained the Ax=b type LSE problem as we did in the class. iii. Solve the problem via programming. iv. Compare your result with the lenaAffine image to check if they look similar, i.e. you are checking visually whether the affine transform you estimated is correct. iv. Use the estimated A matrix to transform the original image. How well does it line up with lenaAffine? Try to analyze your transformation results. What are the possible sources of error? Try to optimize the selection of the corresponding points. From which regions it is preferrable to select points?