Panorama Stitching

Introduction

Sometimes, when capturing, we can only get a partial image of the object, especially when the size of the object is extremely large. However, if the two partial images have some content overlapping, we can make use of CV algorithms to "stitch" them together to get a panorama of the scene. This is exactly the purpose of this project.

Requirements

In this project, you are required to construct a panorama stitching system. Having two (or multiple) images with overlapping regions at hand, your system can "merge" them together to get a bigger picture, which contains the "union" contents of the two separate images. At the same time, some photometric defects aroused by merging can be considered and dealt with, to make the resulting panorama more vivid and realistic [4].  In addition, since I have provided sample codes for this project, you can follow my framework; however, some components need to be modified by you. Specifically, please use the SIFT method to detect interest points and to construct local descriptors.

In this project, you can use the local features based image alignment method, mentioned in our lecture, to align the two images, and then to merge them together. In such a method, the basic procedures include:

1> Given two images im1 and im2, detect local interest points sets on each of them, respectively;

2> For each extracted interest point, a descriptor (SIFT) should be constructed based on its neighborhood;

3> The correspondence relationship between the interest points on two images should be established;

4> The geometric transformation matrix (usually called as homography matrix) between im1 and im2 could be estimated based on the correspondence relationship established in step 3); usually at this step, RANSAC algorithm can be used;

5> Based on the obtained homography matrix, images can be aligned and stitched. Besides, different lighting conditions of the two images need to be considered to make the "stitching" more natural.

Following is an example (actually, two examples are provided in the source code). The source code and testing images for this example can be found here.

Figure 1

Figure 2

Figure1 and Figure2 are the pictures of our SSE building, taken by myself on Sep. 18, 2011. They have different contents and different view points. Now, I want to stitch them together.

STEP1: interest points detection

I simply use the Harris corner detector here.

STEP2: local descriptor for each point.

I simply use a fixed neighborhood as the descriptor and use the normalized correlation as the matching criterion to find point correspondences between two images.

STEP3: using RANSAC algorithm to find the inliers of the correspondence pairs and to estimate the homography matrix

Above figure shows the inliers of correspondence pairs.

STEP4: using the estimated homography matrix to transform img1 to the coordinate system of img2.

STEP5: merge transformed img1 with img2 together, as shown in the following figure.

Most Related References

1. http://en.wikipedia.org/wiki/Image_stitching

2. R. Szeliski, Image alignment and stitching: a tutorial, 2006 (More information about image stitching than our project requirements are described in this tutorial.)

Created on: Mar. 10, 2014

Last updated on: Mar. 10, 2014