近年來，深度攝影機的價格越趨近便宜，令更多的研究者能夠使用深度攝影機。相對原本的二維影像，增加深度資訊對於電腦視覺的應用有很大的幫助，但是目前從深度攝影機得到原始的深度影像都有破洞、邊緣不完整、雜訊等問題。因此本論文提出了一個深度影像的修補方法，並將修補後的影像以3D重建的方式重現。首先，對破洞的部分進行偵測，並使用破洞周圍有效的深度資訊以及背景的深度資訊進行破洞的修補。再來，在複雜背景下，彩色資訊的邊緣不明顯，我們使用形態學和中值濾波器對邊緣做修正。在簡單背景下，使用彩色圖像的邊緣對深度圖像進行校正。最後，將得到無破洞且邊緣較完整的深度影像以Kinect Fusion的方法重建3D模型。

In recent years, the price of depth camera became low, so that researchers can use depth camera to do more application. For computer vision, depth images can provide more useful information. However, generally there are some problems in depth image, such as holes, incomplete edge, and temporal random fluctuations. Therefore, this paper proposes a depth image inpainting method applying on 3D object reconstruction. First, in the holes filling part we use No-measured pixels detection to locate the position of no-measured pixels, and then use the pixels which is near to nmd-pixels to estimate the depth information. Secondly, in complex background the edge of the color image is not obvious, so that we use erosion and dilation operation and median filter to smooth the edge. In simple background, we use our edge modified method to correct the edge of depth image depend on color image. Finally, we use 3D reconstruction method, Kinect Fusion, proposed by Microsoft to modeling the object.

目錄
致謝	I
中文摘要	II
英文摘要	III
目錄	IV
圖目錄	VI
表目錄	VIII
第一章 緒論	1
1.1 研究動機	1
1.2 研究方法	2
1.3 論文架構	2
第二章 相關研究與背景知識	3
2.1 相關研究	3
2.1.1 雙眼立體視覺攝影機深度修補	3
2.1.2 結構光攝影機深度修補	5
2.2 相關技術	9
2.2.1 色彩空間：HSV	9
2.2.2 邊緣偵測	11
2.2.3 物件標記(Labeling)	16
2.2.4 Kinect Fusion	18
第三章 3D建模系統	21
3.1 系統架構	21
3.2 系統流程	21
3.2.1 深度圖像修補	21
3.2.2 邊緣修正	26
3.2.3 3D Reconstruction	34
第四章 實驗結果	36
4.1 實驗環境	36
4.2 複雜背景建模	37
4.3 單純背景前景建模	40
4.4 實驗比較	43
4.4.1 時間比較	43
4.4.2 建模效果比較	43
第五章 結論與未來展望	45
5.1 結論	45
5.2 未來展望	46
參考文獻	47
 
圖目錄
圖2. 1 Stereo camera	4
圖2. 2 Kinect	5
圖2. 3 M. Camplani[3]所提出之方法	6
圖2. 4 F. Qi[4]所提出之方法	7
圖2. 5 M. Schmeing [5] 提出之方法	8
圖2. 6 HSV色彩空間	10
圖2. 7 Roberts邊緣偵測	12
圖2. 8 Sobel邊緣偵測	13
圖2. 9 Laplace邊緣偵測	14
圖2. 10 Canny邊緣偵測	15
圖2. 11四連通遮罩	17
圖2. 12 Kinect Fusion流程圖	18
圖3. 1系統流程圖	21
圖3. 2深度圖像修補流程圖	21
圖3. 3 Nmd-pixels	22
圖3. 4前景破洞	23
圖3. 5前景破洞修補	23
圖3. 6“＊”字型遮罩	25
圖3. 7修補後的深度影像	26
圖3. 8複雜背景邊緣修正	27
圖3. 9邊緣修正流程圖	27
圖3. 10背景濾除	28
圖3. 11邊緣強化	28
圖3. 12初始種子選取遮罩	29
圖3. 13標籤後的圖像	30
圖3. 14種子區域	30
圖3. 15 HSV色彩空間	31
圖3. 16彩色標籤後的圖像	32
圖3. 17深度填補是意圖	32
圖3. 18邊緣修正深度影像	33
圖3. 19複雜背景建模	35
圖3. 20前景建模	35
圖4. 1 Kinect	36
圖4. 2雜背景建模	38
圖4. 3複雜背景建模	39
圖4. 4單純背景建模	40
圖4. 5單純背景建模	41
圖4. 6單純背景建模	42
圖4. 7建模效果比較圖	44
 
表目錄
表4. 1 Kinect規格	37
表4. 2計算時間比較	43

參考文獻
[1] C. Fehn,”Depth-Image-Based Rendering(DIBR), Compression and Transmission for a New Approach on 3D-TV”, Proceedings of SPIE, Vol .5291, 2004, pp.93-104.
[2] C. Cheng, S. Lin, S. Lai, and J. Yang,” Improved Novel View Synthesis from Depth Image with Large Baseline,” in Proc. of the International Conference on Pattern Recognition (ICPR) , 2008, pp.1-4.
[3] M. Camplani and L. Salgado, “Efficient spatio-temporal hole filling strategy for Kinect depth maps, ”Proceedings of SPIE, Vol. 8290, 2012.
[4] F. Qi, J. Han, P. Wang, G. Shi, and Fu Li, “Structure guided fusion for depth map inpainting,”Pattern Recognition Letters, Vol. 34, 2013, pp.70-76.
[5] M. Schmeing and X. Jiang, “Edge-Aware Depth Image Filtering using color segmentation, ”Pattern Recognition Letters, Available online, 2014.
[6] 賴韋名，盲人避障輔助系統之設計，淡江大學電機工程學系碩士論文，民國一百零二年。
[7] LS. Davis, "A survey of edge detection techniques", Computer Graphics and Image Processing, Volume 4, no. 3, 1975, pp. 248-260.
[8] Irvin Sobel,” Neighborhood coding of binary images for fast contour following and general binary array processing,” Computer Graphics and Image Processing, Volume 8, Issue 1, 1978, pp.127-135.
[9] Peter J. Burt and Edward H. Adelson, “The Laplacian Pyramid as a Compact Image Code,” in Proc. of IEEE Transactions on Communications, Vol. 31, No. 4, 1983, pp.532-540.
[10] John Canny, “A Computational Approach to Edge Detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 8, No. 6, 1986, pp. 679-698.
[11] L. Chen, H. Wei, and James M. Ferryman, “A Survey of Human Motion Analysis using Depth Imagery,” Pattern Recognition Letters, Vol. 34, No. 15, 2013, pp. 1995-2006.
[12] R. Adams, and L. Bischof, “Seeded region growing,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 6,1994, pp. 641-647.
[13] S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. Davison, and A. Fitzgibbon, “KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera,” Proceedings of the 24th annual ACM symposium on User interface software and technology, 2011, pp.559-568.
[14]R. A. Newcomne, S. Izadi, O. Hiliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohli, J. Shotton, S. Hodges, and A. Fitzgibbon, “Real-time dense surface mapping and tracking,” Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality, 2011, pp. 127-136.
[15]L. Zhang and W. J. Tam, “Stereoscopic Image Generation Based on Depth Images for 3DTV,” IEEE Trans. Broadcast., vol. 51, 2005, pp.191-199.
[16]W. Y. Chen, Y. L. Chang, S. F. Lin, L. F. Ding, and L. G. Chen, “Efficient Depth Image Based Rendering with Edge Dependent Depth Filter and Interpolation,” in Proc., IEEE International Conference Multimedia and Expo, 2005, pp. 1314-1317.
[17]K. Khoshelham and S. O. Elberink, “Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Application” Sensor, 2012, pp.1437-1454.