透過Depth image based rendering (DIBR)的技術，我們可以將一張2D的彩色影像，搭配相對應的深度圖，合成出一張虛擬的3D影像出來。然而，透過DIBR所合成出來虛擬的3D影像會在新影像上產生破洞(Hole) ，降低3D影像的品質。一般來說，為了避免新合成的3D影像產生破洞，常見的做法是先對整張深度圖做平滑處理，目的在縮小相鄰兩點的位移差距，減小破洞的大小。但這種做法會在新影像上產生另一種幾何失真的問題。因此，在本篇研究中，我們提出一個基於紋理方向偵測的修補方式。針對可能產生破洞的地方先做紋理方向的判斷，判斷出紋理方向後破洞依該方向做修補。實驗結果顯示，我們的做法能保有完整的深度資訊，可以避免大部分的幾何失真，減少計算的時間。

The Depth Image Based Rendering (DIBR) technology is a common approach to create a virtual 3D image from one single 2D image together with the corresponding depth image. However, holes caused by disocclusion in the warped left/right images become a problem. To reduce the sizes and the number of holes, smoothing the depth image is often adopted. But smoothing also results geometric distortions and degrades the depth image quality. In this study, a hole filling method based on the edge texture direction is proposed. Texture directional information is first probed in the background pixels where holes will take place after warping. Then, in the warped image, holes are filled according to their directions. Experimental results showed that this algorithm preserves the complete depth information and reduces the amount of geometric distortion as expected.

目錄
第一章 緒論 1
1.1 研究動機與目的 1
1.2 論文架構 3
第二章 相關文獻回顧 4
2.1 立體視覺基本原理 4
2.2 Depth Image Based Rendering (DIBR) 概要 8
第三章 影像破洞之修補系統 19
3.1 視差值修正 22
3.2 破洞偵測 25
3.3 紋理特徵偵測 26
3.4 垂直紋理特徵強化 30
3.5 破洞修補 32
第四章 實驗結果與分析 35
第五章 結論與未來研究 45
參考文獻 46
附錄：英文論文 50

圖目錄
圖1、視差示意圖[19] 4
圖2、紅綠眼鏡及立體影像 6
圖3、偏光眼鏡及原理示意圖 7
圖4、視差屏障示意圖 7
圖5、深度影像及虛擬影像範例 8
圖6、Disocclusion & Occlusion示意圖 9
圖7、DIBR流程圖 10
圖8、對稱式高斯平滑實驗結果圖 12
圖9、非對稱高斯平滑實驗結果圖 13
圖10、Edge dependent filtering of Depth Map實驗結果圖 14
圖11、Distance map實驗結果圖 15
圖12、Cheng et al. [7]實驗結果圖 16
圖13、攝影機架攝與立體影像合成之關係圖 17
圖14、破洞修補系統之流程圖 21
圖15、深度資訊錯誤之區域示意圖 22
圖16、視差值修正之流程圖 24
圖17、視差值修正後之深度影像 25
圖18、破洞位置分析圖 25
圖19、Sobel value取樣方式示意圖 27
圖20、紋理特徵搜尋方向示意圖 28
圖21、紋理特徵搜尋&判斷演算法流程圖 29
圖22、Texture 判斷演算法 30
圖23、垂直紋理特徵強化之流程 31
圖24、破洞修補之流程圖 32
圖25、水平鏡射修補 34
圖26、Interview測試影像及實驗結果圖 38
圖27、Cones測試影像及實驗結果圖 40
圖28、Art測試影像及實驗結果圖 42
圖29、Art實驗結果局部放大圖 42

表目錄
表1、PSNR值比較 44

[1] A. Redert, M. Op de Beeck, C. Fehn, W. Ijsselsteijn, M. Pollefeys, L. Van Gool, E. Ofek, I. Sexton, and P. Surman, “ATTEST –Advanced Three-Dimensional Television System Techniques”, Proceedings of 3DPVT’ 02, pp. 313-319, Jun. 2002.
[2] J. Flack, P. Harman, and S. Fox, “Low Bandwidth Stereoscopic Image Encoding and Transmission”, Proceedings of SPIE, Vol. 5006, pp. 206-214, 2003.
[3] C. Fehn, K. Hopf, and Q. Quante, “Key Technologies for an Advanced 3D-TV System ” In Proceedings of SPIE Three-Dimensional TV, Video and Display III, , pp. 66-80, Oct. 2004.
[4] C. Fehn, "Depth-Image-Based Rendering (DIBR), Compression and Transmission for a New Approach on 3D-TV”, Proceedings of SPIE, Vol. 5291, pp. 93-104, 2004.
[5] Q. H. Nguyen, M. N. Do, and S. J. Patel, “Depth Image-Based Rendering from Multiple Cameras with 3D Propagation Algorithm,” in IMMERSCOM’09; Proceedings of the 2nd International Conference on Immersive Telecommunications. ICST, pp. 1–6, 2009.
[6] C. Vzquez, W. J. Tam, and F. Speranza, “Stereoscopic Imaging: Filling Disoccluded Areas in Depth Image-Based Rendering,”
Proceedings of SPIE, Vol.6392, 2006.
[7] C.-M. Cheng, S.-J. Lin, S.-H. Lai, and J.-C. Yang, “Improved Novel View Synthesis from Depth Image with Large Baseline,” in Proc. of the 19th International Conference on Pattern Recognition (ICPR), pp. 1–4, 2008.
[8] W. J. Tam, G. Alain, L. Zhang, T. Martin, and R. Renaud, “Smoothing Depth Maps for Improved Stereoscopic Image Quality,” in Proc. Inter-national Society for Optical Engineering Conf. Three-Dimensional TV, Video, and Display III, vol. 5599, pp.162–172, Oct. 2004.
[9] L. Zhang and W. J. Tam, “Stereoscopic Image Generation Based on Depth Images for 3DTV,” IEEE Trans. Broadcast., vol. 51, pp. 191–199, Jun. 2005.
[10] W. J. Tam and L. Zhang, “Non-Uniform Smoothing of Depth Maps before Image-Based Rendering,” in Proc. of ITCOM’04, vol. 5599, pp. 173-183, Oct. 2004.
[11] W. Y. Chen, Y. L. Chang, S. F. Lin, L. F. Ding, and L. G. Chen, “Efficient Depth Image Based Rendering with Edge Dependent Depth Flter and Interpolation,” in Proc. IEEE Int. Conf. Multimedia and Expo, pp. 1314–1317, Jul. 2005,
[12] Y. K. Park, K. Jung, Y. Oh, S. Lee, J. K. Kim, G. Lee, H. Lee, K. Yun, N. Hur, and J. Kim, “Depth-Image-Based Rendering for 3DTV Service over T-DMB,” Signal Process.: Image Commun., vol. 24, pp. 122–136, Jan. 2009.
[13] S.-B. Lee and Y.-S. Ho, “Discontinuity-Adaptive Depth Map Filtering for 3D View Generation,” in IMMERSCOM ’09: Proceedings of the 2nd International Conference on Immersive Telecommunications, ICST, pp. 1–6, 2009.
[14] P. Lee and Effendi, “Nongeometric Distortion Smoothing Approach for Depth Map Preprocessing,” IEEE Transactions on Multimedia, vol. 13, No. 2, pp. 246-254, 2011.
[15] I. Daribo, C. Tillier, and B. Pesquet-Popescu, “Distance Dependent Depth Filtering in 3D Warping for 3DTV,” in Proc. of Multimedia Signal Processing, pp. 312-315, Oct. 2007.
[16] S. Curti, D. Sirtori, and F. Vella, “3D Effect Generation from Monocular View”, In IEEE Proc. of 3DPVT’02 International Symposium on 3D Data Processing Visualization and Transmission, pp. 550 –553, 2002.
[17] Damiel Scharstein. (July, 2007). Middlebury stereo datasets [Online].
Available: http://vision.middlebury.edu/stereo/data.
[18] Index of /3dav/3DAV_Test_Data/EE4/FHG_HHI. [Online]. Available: https://www.3dtv-research.org/3dav/3DAV_Test_Data/EE4/FHG_HHI
[19] http://gnn.gamer.com.tw/6/43706.htm