物體辨識與姿態估測是機器視覺應用中重要的處理工作。近年來，由於RGB-D攝影機的廣泛使用，三維物體辨識技術也越來越受到重視，因其在雜亂的場景中，不但具有較高的強健性與辨識率，且能夠更加容易且精準的估測物體三維姿態。因此，本論文設計一個基於RGB-D攝影機的三維物體辨識與姿態估測系統。所設計之系統首先經由RGB-D攝影機擷取色彩點雲影像，並提取點雲影像特徵點。接著使用紋理以及形狀特徵以建立CSHOT特徵描述子，並將特徵點資訊進行匹配以找出對應點。利用對應點資訊輸入霍夫投票演算法，濾除錯誤的特徵匹配點，以輸出物體初始姿態資訊。最後經由RANSAC演算法優化初始姿態，再利用假設驗證步驟將物體辨識中錯誤的物體假設去除，以取得最佳的物體辨識以及姿態估測結果。在實驗結果中，證實本論文所設計之系統不僅能夠成功辨識場景中物體，且對於物體的平移以及旋轉都能精準估測姿態。

Object recognition and pose estimation are important functions in applications of computer vision. In recent years, RGB-D cameras become more and more popular and 3D object recognition technology has got more and more attention as it not only has a higher object recognition rate in a complex environement, but also is able to accurately estimate the 3D pose information of the object in the workspace. Hence, this thesis presents a novel design of a RGB-D camera based 3D object recognition and pose estimation system. First of all, The proposed system takes colored point cloud data and extracts keypoints of the scene from the RGB-D camera. Then, the existing Color Signature of Histograms of Orientations (CSHOT) description algorithm is employed to build descriptors of the detected keypoints based on texture and shape information. Given the extractd keypoint descriptors, a matching process is performed to find correspondences between the scene and a colored point cloud model of an object. Next, a Hough voting algorithm is adopted to filter out matching errors in the correspondence set and estimate the initial 3D pose of the object. Finally, the pose estimation stage employs RANSAC and hypothesis verification algorithms to refine the initial pose and filter out poor estimation results with error hypothesis. Experimental results show that the proposed system not only successfully recognizes object in a complex scene, but also is able to accurately estimate the 3D pose information of the object with respect to the camera.

目錄
中文摘要：	I
Abstract:	II
目錄	III
圖目錄	VIII
表目錄	X
第一章 序論	1
1.1	研究背景	1
1.2	研究動機與目的	3
1.3	論文架構	4
第二章 相關研究	5
2.1  基於三維特徵點之物體偵測演算法	5
2.1.1  三維特徵偵測演算法	5
2.1.1.1  LSP (Local Surface Patches)演算法	6
2.1.1.2  ISS(Intrinsic Shape Signatures)演算法	7
2.1.1.3  KPQ (KeyPoint Quality)演算法	7
2.1.2  三維特徵描述子演算法	8
2.1.2.1  PS(Point Signatures)演算法	10
2.1.2.2  KP(KeyPoint)演算法	10
2.1.2.3  3D-SURF(3D Speeded-Up Robust Features)演算法	10
2.1.2.4  3DSC(3D Shape Context)演算法	10
2.1.2.5  PFH(Point Feature Histogram)演算法	11
2.1.2.6  FPFH(Fast Point Feature Histogram)演算法	11
2.1.2.7  MeshHoG(Mesh Histogram of Gradients)演算法	11
2.1.2.8  SHOT(Signature of Histograms of Orientations)演算法	12
2.2  基於點雲之物體姿態估測演算法	12
2.2.1  ICP(Iterative Closest Point)演算法	13
2.2.2  RANSAC(RANdom SAmple Consensus)演算法	13
2.3  系統架構	14
第三章 三維特徵偵測與物體辨識	16
3.1  場景分割處理	16
3.1.1 連通元件(connected component)演算法	16
3.1.2 平面分割	18
3.1.3 物體模型分割	20
3.2  特徵偵測	21
3.2.1 特徵點提取	21
3.2.2 特徵描述子建立	22
3.2.2.1  SHOT區域參考座標系(local reference frame)	23
3.2.2.2  CSHOT(Color SHOT)特徵描述子	26
3.3  物體辨識處理	30
3.3.1 特徵點匹配	30
3.3.2 物體辨識演算法	31
3.3.2.1  BOARD區域參考座標系(local reference frame)	32
3.3.2.2  霍夫投票(Hough voting)演算法	34
第四章 三維物體姿態估測	38
4.1  物體姿態估測演算法	38
4.1.1 RANSAC演算法	39
4.1.2 本論文所使用之RANSAC演算法	39
4.2  物體姿態驗證	41
4.2.1  GO假設演算法	41
第五章 實驗結果與分析	44
5.1  軟硬體介紹	44
5.2  物體姿態估測實驗	47
5.3  平移實驗	48
5.3.1 X軸平移	49
5.3.2 Y軸平移	49
5.3.3 Z軸平移	50
5.3.4 平移實驗結果分析	51
5.4  旋轉實驗	52
5.4.1 X軸旋轉	52
5.4.2 Y軸旋轉	53
5.4.3 Z軸旋轉	54
5.4.4 旋轉實驗結果分析	56
5.5  隨機姿態實驗	57
5.5.1 隨機姿態	58
5.5.2 隨機姿態實驗結果分析	58
第六章 結論與未來展望	60
參考文獻	61
附錄一-X軸平移實驗結果圖	64
附錄二-Y軸平移實驗結果圖	67
附錄三-Z軸平移實驗結果圖	69
附錄四-X軸旋轉實驗結果圖	71
附錄五-Y軸旋轉實驗結果圖	74
附錄六-Z軸旋轉實驗結果圖	77
附錄七-隨機姿態實驗結果圖	80

圖目錄
圖1.1、特徵描述方法示意圖：(a)區域特徵；(b)全域特徵。	2
圖2.1、簽名與直方圖示意圖。	9
圖2.2、系統架構圖。	15
圖3.1、標籤合併情況示意圖。	18
圖3.2、特徵偵測流程圖。	21
圖3.3、Mian區域參考座標系示意圖。	25
圖3.4、SHOT區域參考座標系距離權重示意圖。	25
圖3.5、特徵向量方向歧異問題示意圖。	26
圖3.6、特徵向量方向劃分示意圖。	26
圖3.7、SHOT簽名結構示意圖。	29
圖3.8、CSHOT描述子示意圖。	29
圖3.9、物體辨識處理流程圖。	29
圖3.10、方向軸決定示意圖：(a)z軸；(b)x軸。	34
圖3.11、霍夫投票方法動作流程。	36
圖3.12、使用區域參考座標系進行霍夫轉換。	36
圖3.13、以霍夫投票濾除錯誤匹配點。	36
圖5.1、Microsoft Kinect v1攝影機。	45
圖5.2、Pan-Tilt二軸馬達控制平台。	46
圖5.3、UR5機器手臂。	46
圖5.4、實驗用之參考點雲模型：(a) A物體；(b) B物體。	47

表目錄
表2.1、三維特徵描述子分類表。	9
表5.1、電腦規格表。	45
表5.2、攝影機規格表。	45
表5.3、使用軟體版本。	45
表5.4、Pan-Tilt二軸馬達控制平台規格表。	46
表5.5、UR5機器手臂硬體規格表。	47
表5.6、A物體之X軸平移數據。	48
表5.7、B物體之X軸平移數據。	49
表5.8、A物體之Y軸平移數據。	49
表5.9、B物體之Y軸平移數據。	50
表5.10、A物體之Z軸平移數據。	50
表5.11、B物體之Z軸平移數據。	50
表5.12、A物體之X,Y,Z三軸之平移誤差數據。	51
表5.13、B物體之X,Y,Z三軸之平移誤差數據。	52
表5.14、A物體之X軸旋轉數據。	53
表5.15、B物體之X軸旋轉數據。	53
表5.16、A物體之Y軸旋轉數據。	54
表5.17、B物體之Y軸旋轉數據。	54
表5.18、A物體之Z軸旋轉數據。	55
表5.19、B物體之Z軸旋轉數據。	55
表5.20、A物體之X,Y,Z三軸之旋轉誤差數據。	56
表5.21、B物體之X,Y,Z三軸之旋轉誤差數據。	56
表5.22、隨機姿態實驗數據。	58
表5.23、隨機姿態誤差數據。	59

[1]A. Frome, D. Huber, R. Kolluri, T. Bulow, J. Malik, “Recognizing objects in range data using regional point descriptors,” Proceedings of the European Conference on Computer Vision, vol. 3023, pp. 224–237, 2004.
[2]H. Y. Kuo, H. R. Su, S. H. Lai, C. C. Wu, “3D object detection and pose estimation from depth image for robotic bin picking,” IEEE International Conference on Automation Science and Engineering, pp. 1264–1269, 2014.
[3]M. Beetz, Z. C. Marton, D. Pangercic, R. B. Rusu, A. Holzbach, “Hierarchical object geometric categorization and appearance classification for mobile manipulation,” IEEE/RAS International Conference on Humanoid Robots, pp. 365–370, 2010.
[4]R. B. Rusu, G. Bradski, R. Thibaux, J. Hsu, “Fast 3d recognition and pose using the viewpoint feature histogram,” International Conference on Intelligent Robots and Systems, pp. 2155–2162, 2010.
[5]A. Aldoma, N. Blodow, D. Gossow, S. Gedikli, R. Rusu, M. Vincze, G. Bradski, “CAD-model recognition and 6 DOF pose estimation using 3D cues,” International Conference on Computer Vision, pp. 585–592, 2011.
[6]M. Zhu, K. Derpanis, Y. Yang, S. Brahmbhatt, M. Zhang, C. Phillips, M. Lecce, K. Daniilidis, “Single image 3D object detection and pose estimation for grasping,” IEEE International Conference on Robotics and Automation, pp. 3936–3943, 2014.
[7]H. Chen, B. Bhanu, “3D free-form object recognition in range images using local surface patches,” Pattern Recognition Letters, vol. 28, no. 10, pp. 1252–1262, 2007.
[8]C. Dorai, A. Jain, “COSMOS—A representation scheme for 3D free-form objects,” IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 19 , no. 10, pp. 1115–1130, 1997.
[9]Y. Zhong, “Intrinsic shape signatures: a shape descriptor for 3D object recognition,” International Conference on Computer Vision – 3D Representation for Recognition Workshop, pp. 689–696, 2009.
[10]A. S. Mian, M. Bennamoun, R. A. Owens, “On the repeatability and quality of keypoints for local feature-based 3D object retrieval from cluttered scenes,” International Journal of Computer Vision, vol. 89, no. 2-3, pp. 348–361, 2010.
[11]C. S. Chua, R. Jarvis, “Point signatures: a new representation for 3D object recognition,” International Journal of Computer Vision, vol. 25, no. 1, pp. 63–85, 1997.
[12]J. Knopp, M. Prasad, G. Willems, R. Timofte, L. V. Gool, “Hough transform and 3D SURF for robust three dimensional classification,” European Conference on Computer Vision, pp. 589–602, 2010.
[13]H. Bay, A. Ess, T. Tuytelaars, L. V. Gool, “Speeded-Up Robust Features (SURF),” Computer Vision and Image Understanding, vol. 110, no. 3, pp. 346–359, 2008.
[14]A. Frome, D. Huber, R. Kolluri, T. Bülow, J. Malik, “Recognizing objects in range data using regional point descriptors,” European Conference on Computer Vision, vol. 3, pp. 224–237, 2004.
[15]R. Rusu, N. Blodow, Z. Marton, M. Beetz, “Aligning point cloud views using persistent feature histograms,” International Conference on Intelligent Robots and Systems, pp. 3384–3391, 2008.
[16]R. Rusu, N. Blodow, M. Beetz, “Fast point feature histograms (fpfh) for 3d registration,” International Conference on Robotics and Automation, pp. 3212–3217, 2009.
[17]D. G. Lowe, “Object recognition from local scale-invariant features,” International Conference on Computer Vision, pp. 1150–1157, 1999.
[18]A. Zaharescu, E. Boyer, R. Horaud, “Keypoints and local descriptors of scalar functions on 2d manifolds,” International Journal of Computer Vision, vol. 100, no. 1, pp. 78–98, 2012.
[19]F. Tombari, S. Salti, L. D. Stefano, “Unique signatures of histograms for local surface description,” European Conference on Computer Vision, pp. 356–369, 2010.
[20]P. J. Besl, H. D. McKay, “A method for registration of 3D shapes,” International Society for Optics and Photonics, vol. 14, no. 2, pp. 239–256, 1992.
[21]H. Pottmann, S. Leopoldseder, M. Hofer, “Registration without ICP,” Computer Vision and Image Understanding, vol. 95, no. 1, pp. 54–71, 2004.
[22]A. Segal, D. Haehnel, S. Thrun, “Generalized-icp,” Robotics: Science and Systems, vol. 25, pp. 26–27, 2009.
[23]M. Fischler, R. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395, 1981.
[24]A. J. B. Trevor, S. Gedikli, R. B. Rusu, H. I. Christensen, “Efficient organized point cloud segmentation with connected components,” 3rd Workshop on Semantic Perception Mapping and Exploration, 2013.
[25]L. Shapiro, G. C. Stockman, Computer Vision, chapter 3, pp. 69–75. Prentice Hall, 2001.
[26]S. Holzer, R. B. Rusu, M. Dixon, S. Gedikli, N. Navab, “Adaptive neighborhood selection for real-time surface normal estimation from organized point cloud data using integral images,” IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2684–2689, 2012.
[27]F. Tombari, S. Salti, L. D. Stefano, “A combined texture-shape descriptor for enhanced 3D feature matching,” International Conference on Image Processing, pp. 809–812, 2011.
[28]R. Bro, E. Acar, T. Kolda, “Resolving the sign ambiguity in the singular value decomposition,” Journal of Chemometrics, vol. 22, pp. 135–140, 2008.
[29]F. Tombari, L. D. Stefano, “Hough voting for 3d object recognition under occlusion and clutter,” IPSJ Transactions on Computer Vision and Application, vol. 4, pp. 20–29, 2012.
[30]A. Petrelli, L. D. Stefano, “On the repeatability of the local reference frame for partial shape matching,” IEEE International Conference on Computer Vision, pp. 2244–2251, 2011.
[31]A. G. Buch, D. Kraft, J. K. Kamarainen, H. G. Petersen, N. Kruger, “Pose estimation using local structure-specific shape and appearance context,”  International Conference on Robotics and Automation, pp. 2080–2087, 2013.
[32]A. Aldoma, F. Tombari, L. D. Stefano, M. Vincze, “A global hypotheses verification method for 3D object recognition,” European Conference on Computer Vision, pp. 511–524, 2012.