全景成像技術是近年來十分重要的議題，到目前為止已有許多全景成像的技術發展在各式各樣的產品上。從最早期開發在個人電腦中以及近年來開發在數位相機與可攜式裝置中都可以看到這類技術的運用。在傳統的全景成像技術中，使用者可以使用一連串的影像或是一小段影片來建立同一場景的全景圖。但在傳統的全景成像技術中，如果影像或影片內有移動的物件時，所建立出來的全景圖背景會保留其物件，同時當全景圖的素材中有較為明顯的人物時也容易造成人物結構上的模糊。因此本論文針對此問題提出一個運用人臉辨識、物件切割、動量背景修補以及有效的全景合成技術的演算法。針對背景有移動人物的場景，將其人物與背景先進行分離，在此部份本論文先使用著名的物件切割方法-GrabCut演算法來切割物件並且將其改良為自動化程序，讓使用者可以不用經過手工的標記物件即可達到物件切割的效果。接著使用動量背景修補方法將分離出來的區域填補成原先可能的背景，在此部分本論文根據傳統的影像修補方法並加入動量的參考依據讓背景也跟著在移動的情況下也可以順利地得到正確的修補結構。最後再將所有的素材合成為一張全景影像，本論文所提出的全景圖製作方法加入了影像能量圖與影像縫線的概念，讓接合的縫線落在重疊區域內較不明顯的結構中，如此一來可以有效地解決人物鬼影和因為重疊位置的關係所形成的結構模糊的問題。而人臉辨識的運用除了定義出人物的大概區域外，也可以用來判斷針測到的人物是否是使用者認識的人，如果認定是陌生人時即可利用上述的方法進行移除的動作，進一步達到過濾陌生人的功能。本論文所提出的方法除了人物區域定義是可以進行手動設定，其他的步驟皆為自動化程序，如此可有效減少使用者操作上的負擔。

Panoramic photography is becoming very popular within the general users, skilled photographers and in many useful computer and Internet based application domains like 3D virtual reality. With the introduction of panoramic photography support in the general purpose digital cameras and smart phones, users and applications that use the panoramic photos are also increasing. In traditional panoramic photography, moving objects or as referred in this thesis - the strangers, in the background should be eliminated since those strangers obscure the scenery that we want to retain in our photograph. This thesis discusses a novel method to remove the strangers (moving objects) whose face data is not available in the face database of camera) from the background of the focused area and compose a panoramic image. In the proposed system the object segmentation is automation and based on GrabCut algorithm. The method of motion inpainting of background can be repair background on moving background, effectively. The method of panorama creation is using concept of energy map and image seam that avoided ghost problem in panorama and maintained the structure of human. The proposed of panorama creation system is fully automatic except that the user required marking the unidentified moving objects in the object segmentation phase.

Table of Content
Chapter 1 Introduction	1
1.1	Motivation	1
1.2	Overview of Panorama	2
1.3	Organization of this Dissertation	2
Chapter 2	 Related Works	5
2.1	Feature and Object Extraction	5
2.2	Panorama Creation	7
2.3	Seam Carving	10
Chapter 3	 Object Segmentation	13
3.1	Face Detection	13
3.2	Object Location Definition	15
3.3	Object Segmentation	17
3.4	The Dynamic Background Inpainting	20
3.4.1	Motion Estimation	21
3.4.2	Background Structure Inpainting	24
Chapter 4 Proposed Panorama Creation	28
4.1	Feature Matching	28
4.2	Overlapping Inpainting	32
4.3	Optimal Seam and Combination	37
4.3.1	Image calibration	37
4.3.2	Find Optimal Seam	41
Chapter 5 Experimental Results	48
Chapter 6 Conclusion and Future Work	66
Reference	68

Table of Figure
Figure 1. The flowchart of proposed approach	3
Figure 2. The flowchart of proposed method	4
Figure 3. The result of SIFT and ASIFT algorithms	6
Figure 4. The example of panoramic creation	7
Figure 5. Results obtained using the Autostitch	9
Figure 6. The result of image retargeting via seam carving	12
Figure 7. The result of face detection	14
Figure 8. The result of Object Location Definition	16
Figure 9. The example of GrabCut algorithm	17
Figure 10. The result of Object Segmentation	19
Figure 11. The sample of video inpainting [32]	20
Figure 12. Direction definition of motion estimation	21
Figure 13. The schematic diagram of step 1	22
Figure 14. The schematic diagram of step 2	22
Figure 15. The schematic diagram of step 3	23
Figure 16. The schematic diagram of step 4	24
Figure 17. The schematic diagram of background structure inpainting	25
Figure 18. The inpainting result of each frame	26
Figure 19. The final result of background structure inpainting	27
Figure 20. The human result of SIFT and ASIFT algorithm	29
Figure 21. The results of image combination	30
Figure 22. The poor result of human in panoramic creation	32
Figure 23. The disarray structure of human	33
Figure 24. The integrity structure of human	35
Figure 25. The schematic diagram of overlapping inpainting	36
Figure 26. The cylindrical projection in different frames	38
Figure 27. The result of easy combination	38
Figure 28. The schematic diagram of optimal seam	43
Figure 29. The example of shortest path	44
Figure 30. The energy map of optimal seam	46
Figure 31. Image combination (top: original images, bottom: combined photo with overlapped region is marked in red)	47
Figure 32. Main interface of the system	49
Figure 33. Steps of the tool and result	50
Figure 34. Examples of main structure maintain	55
Figure 35. Performance chart of table 1	57
Figure 36. Experimental results (left: one frame from source video, right: resultant panorama)	65

[1].	M. Brown, D. G. Lowe (2007) Automatic Panoramic Image Stitching using Invariant Features. International Journal of Computer Vision. 74(1):59-73.
[2].	Saeid Fazli, Hamed Moradi Pour, Hamed Bouzari, (2009) Particle Filter based Object Tracking with Sift and Color Feature. International Conference on Machine Vision. pp: 89-93.
[3].	H. Bay, T. Tuytelarrs, L. J. V. Gool, (2006) SURF: Speeded Up Robust Features. European Conference on Computer Vision. 3951:404-417.
[4].	H. Bay, A. Ess, T. Tuytelarrs, L. J. V. Gool, (2008) Speeded-Up Robust Features (SURF). Computer Vision and Image Understanding. 110:346-359.
[5].	Kekre, Hemant B., Sudeep D. Thepade. (2008) Rotation Invariant Fusion of Partial Image Parts in Vista Creation using Missing View Regeneration. WASET International Journal of Electrical Computer and Systems Engineering (IJECSE) 47: 660.
[6].	Helmut Dersch (2007) Panorama Tools. Open Source Software for Immersive Imaging International VR Photography Conference, 2007.
[7].	Wang Meng (2009) Panorama Painting: With a Bare Digital Camera. In Image and Graphics, 2009. ICIG'09. Fifth International Conference on. pp: 82-86.
[8].	Song, Baosen, Yongqing Fu, Jinlin Wang (2011) Automatic panorama creation using multi-row images. Information Technology Journal 10:1977-1982.
[9].	Yingen Xiong, Kari Pulli (2010) Fast image stitching and editing for panorama painting on mobile phones. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). pp: 47-52.
[10].	Wen-Yan Lin, Siying Liu, Yasuyuki Matsushita, Tian-Tsong Ng, Loong-Fah Cheong. (2011) Smoothly varying affine stitching. Computer Vision and Pattern Recognition (CVPR) 2011 IEEE Conference on. pp: 345-352.
[11].	M. Brown, D. G. Lowe (2003) Recognising Panoramas. In Proceedings of the 9th International Conference on Computer Vision (ICCV2003). pp: 1218-1225.
[12].	I. Laurent, K. Christof, and N. Ernstr (1998) A Model of Saliency Based Visual Attention for Rapid Scene Analysis, IEEE transactions on pattern analysis and machine intelligence. 20(11): 1254-1259.
[13].	F. Liu and M. Gleicher (2005) Automatic image retargeting with fisheye-view warping. In Proc. ACM Symposium on User Interface Software and Technology. pp: 153-162.
[14].	Shai Avidan, Ariel Shamir (2007) Seam Carving for Content-Aware Image Resizing. ACM Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH. 26(3):Article 10.
[15].	Y. S. Wang, C. L. Tai, O. Sorkine and T. Y. Lee (2008). Optimized scale-and-stretch for image resizing. ACM Transactions on Graphics (TOG)-Proceedings of ACM SIGGRAP Asia 2008. 27(5): Article 118.
[16].	S. Battiato (2012). Content-based image resizing on mobile devices. In International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP). pp: 87-90.
[17].	Chenjun Tao, Jiaya Jia and Hanqiu Sun (2007). Active window oriented dynamic video retargeting. In Proceedings of the Workshop on Dynamical Vision. pp: 1-12.
[18].	Sunghyun Cho, Hanul Choi, Matsushita, Y. and Seungyong Lee (2009). Image retargeting using importance diffusion. Image Processing (ICIP), 2009 16th IEEE International Conference on. pp: 977-980.
[19].	Li-Qun Chen, Xing Xie, Xin Fan, Wei-Ying Ma, Hong-Jiang Zhang and He-Qin Zhou (2003). A visual attention model for adapting images on small displays. Multimedia System. 9(4):353-364.
[20].	Xin Fan, Xing Xie, He-Qin Zhou and Wei-Ying Ma (2003). Looking into video frames on small displays. In Proc. Eleventh ACM Int. Conf. Multimedia. pp: 247-250.
[21].	Jianping Xiao, Xuecheng Zou, Zhenglin Liu and Xu Guo (2007). A novel adaptive interpolation algorithm for image resizing. International Journal of Innovative Computing, Information and Control. 3(6):1335-1345.
[22].	Anthony Santella, Maneesh Agrawala, Doug DeCarlo, David Salesin and Michael Cohen (2006). Gaze-based interaction for semi-automatic photo cropping. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp: 771-780.
[23].	Yunfeng Zhang, Shanshan Gao, Caiming Zhang and Jing Chi (2009). Application of a bivariate rational interpolation in image zooming. , International Journal of Innovative Computing, Information and Control. 5(11):4299-4307.
[24].	Jian Yao, Jean-Marc Odobez (2008) Fast human detection from videos using covariance features. Presented at: The Eighth International Workshop on Visual Surveillance (VS2008). http://hal.inria.fr/inria-00325628/. Accessed 29 Septembre 2008.
[25].	Yu-Ting Chen, Chu-Song Chen (2008) Fast Human Detection Using a Novel Boosted Cascading Structure With Meta Stages. IEEE Transactions on Image Processing. 17(8):1452-1464.
[26].	C. Rother, V. Kolmogorov, A. Blake (2004) "GrabCut": interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23:309-314.
[27].	S. Vicente, V. Kolmogorov, C. Rother (2008) Graph cut based image segmentation with connectivity priors. In IEEE Conference on Computer Vision and Pattern Recognition. pp: 23-28.
[28].	V. Vezhnevets, V. Konouchine (2005) Grow-Cut - Interactive Multi-Label N-D Image Segmentation. Proc. Graphicon. pp: 150-156.
[29].	Yuri Y. Boykov, Marie-Pierre Jolly (2001) Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D Images. Eighth International Conference on Computer Vision (ICCV'01). 1:105-112.
[30].	Patwardhan KA, Sapiro G, Bertalmio M. (2007) Video Inpainting Under Constrained Camera Motion. IEEE Transactions on Image Processing. pp: 545-553.
[31].	Y. Wexler, E. Shechtman, M. Irani (2007) Space-Time Completion of Video. IEEE Trans. on Pattern Analysis and Machine Intelligence. 29:463-476.
[32].	Yu-Ting Chen, Chu-Song Chen (2008) Fast Human Detection Using a Novel Boosted Cascading Structure With Meta Stages. IEEE Transactions on Image Processing. 17(8):1452-1464.
[33].	C. Kokaram, S. J. Godsill (1997) Joint Detection, Interpolation, Motion and Parameter Estimation for Image Sequences with Missing Data. In International Conference on Image Processing. pp: 191-194.
[34].	L. -M. Po, W. C. Ma (1996) A Novel Four-Step Search Algorithm for Fast Block Motion Estimation. IEEE Trans. Circuits Syst. Video Technol. 6(3):313-317.
[35].	R. Li, B. Zeng, M. L. Liou, (1994) A New Three-Step Search Algorithm for Block Motion Estimation. IEEE Trans. Circuits Syst. Video Technol. 4(4):438-442.
[36].	C. H. Cheung, L. M. Po (2005) Novel cross-diamond-hexagonal search algorithms for 	fast block motion estimation. IEEE Transactions on Multimedia. 7(1):16-22.
[37].	S. Vassiliadis, E. A. Hakkennes, J. S. S. M. Wong, G. G. Pechanek (1998) The Sum-Absolute-Difference Motion Estimation Accelerator. In the 24th. EUROMICRO Conference. pp: 559-566.
[38].	Bruno Postle (2011) Panorama Tools. Open Source Software for Immersive Imaging. http://panotools.sourceforge.net/. Accessed 22 February 2011.
[39].	Wan-Lei Zhao, Chong-Wah Ngo (2009) Scale-Rotation Invariant Pattern Entropy for Keypoint-Based Near-Duplicate Detection. Image Processing of IEEE Transactions. 18(2):412-423.
[40].	Yifan Lu, Lei Wang, Hartley, R.,Hongdong Li, Chunhua Shen (2008) Multi-view Human Motion Capture with an Improved Deformation Skin Model. Computing: Techniques and Applications (DICTA) Digital Image. pp: 420-427.
[41].	J. M. Morel, G. Yu (2009) ASIFT: A New Framework for Fully Affine Invariant Image Comparison. SIAM Journal on Imaging Sciences. 2(2):438-469.
[42].	A. Criminisi, P. Perez, K. Toyama (2004) Region Filling and Object Removal by Exemplar-Based Image Inpainting. IEEE Trans. On Image Processing. 13:1200-1212.
[43].	A Criminisi, I Reid, A Zisserman (1999) A plane measuring device. Image and Vision Computing. 17(8):625-634.
[44].	B. Heigl, R. Koch, M. Pollefeys, J. Denzler, L. Van Gool (1999) Plenoptic modeling and rendering from image sequences taken by hand-held camera. In Mustererkennung. pp: 94-101.