本論文提出了一個目標物的三維模型建置與視覺追蹤方法，可幫助機器人完成更困難的物體三維姿態估測和強健型物體追蹤。本論文所提出的方法分為兩部份:第一部分為提出一個三維電腦輔助設計(CAD)模型的建置方法，以協助後續的追蹤應用；第二部份為提出基於三維CAD模型的物體姿態估測方法，達到強健三維目標物追蹤的實現。本論文第一部份利用Micorsoft Kinect感測器來擷取物體的點雲資料，並找出物體的平面資訊。接著，利用平面幾何特性得到物體各頂點的資訊，即可將物體三維CAD模型建立出來。本論文第二部分則是將物體三維CAD模型投影在影像平面上的輪廓資訊進行物體追蹤應用，其透過目標物的影像輪廓資訊，利用三維CAD模型的投影輪廓結果計算兩者之間的最佳擬合的姿態，達到物體三維姿態估測及視覺追蹤的目的。

This thesis presents a three-dimensional (3D) model building method and a model-based visual tracking method for an object-of-interest. The proposed methods can help robots to accomplish more difficult tasks, such as 3D pose estimation and robust visual tracking of a rigid object. The contents of this thesis are divided into two parts. The first part presents the design of a 3D computer-aided design (CAD) model building algorithm, which produces a 3D CAD model of a geometrical object to facilitate the subsequent visual tracking task. The second part presents the design of a CAD-model based object pose estimation algorithm to implement the function of robust 3D object tracking. We first used Microsoft Kinect to capture the point cloud data of an object-of-interest for extracting its planar features. Then, its 3D CAD model was produced by vertex information of the object found by using geometrical characteristics of the object planar features. Next, a robust CAD-model based visual tracking method was implemented by fitting the contour of the projection of the 3D CAD model to the object contour on the image plane, which also can provide 3D pose information of the object during the visual tracking task.

目錄
中文摘要…………………………………………………………………I
Abstract………………………………………………………………….II
目錄...........................................................................................................III
圖目錄...................................................................................................... VI
表目錄........................................................................................................X
第一章 序論 .........................................................................................1
1.1 研究背景.....................................................................................1
1.2 研究動機與目的.........................................................................5
1.3 論文架構.....................................................................................6
第二章 實驗系統介紹 .........................................................................8
2.1 硬體介紹.....................................................................................8
2.2 軟體介紹.....................................................................................9
2.2.1 OpenNI........................................................................................9
2.2.2 Point Cloud Library(PCL)........................................................10
第三章 重建物體模型演算法 ...........................................................13
3.1 點雲幾何元件分類..................................................................14
3.2 基本幾何關係、濾除關鍵點與關鍵點匹配 ..........................17
3.2.1 基本幾何關係............................................................................................ 17
3.2.2 濾除錯誤關鍵點....................................................................................... 19
3.2.3 關鍵點匹配................................................................................................. 21
第四章 姿態估測相關演算法 ...........................................................24
4.1 邊緣模型選擇............................................................................................ 25
4.2 投影模型...................................................................................................... 26
4.3 計算誤差...................................................................................................... 27
4.4 姿態更新...................................................................................................... 28
第五章 物體姿態估測演算法 ...........................................................31
5.1 輪廓線偵測...............................................................................32
5.2 最佳化重新投影誤差...............................................................34
5.2.1 尋找真實物體邊界.................................................................................. 34
5.2.2 控制點和邊界線配對............................................................................. 35
5.2.3 最佳化重新投影誤差............................................................................. 36
5.3 系統架構圖...............................................................................39
第六章 實驗結果與分析 ...................................................................42
6.1 重建模型結果.............................................................................................43
6.2 物體姿態估測實驗.................................................................................. 44
6.3 遮擋物姿態估測實驗............................................................................. 52
第七章 結論與未來展望 ...................................................................56
參考文獻...................................................................................................57


圖目錄 
圖 2.1、本論文演算法架構。 .................................................................8
圖 2.2、Kinect 攝影機[1]。.....................................................................9
圖 2.3、OpenNI 架構圖[5]。.................................................................10
圖 2.4、PCL 架構圖[2]。 ......................................................................11
圖 2.5、國內外使用 PCL 函式庫開發的公司[2]。 .............................12
圖 3.1、提出之重建物體模型演算法流程圖。 ...................................14
圖 3.2、物體點雲圖。…………………………………………………14
圖 3. 3、尋找幾何平面: (a)尋找到第一個平面點雲，(b) 尋找到第二
個平面點雲，(c) 尋找到第三個平面點雲，(d) 尋找到第四個平面點
雲，(e) 尋找到第五個平面點雲，(f) 尋找到第六個平面點雲………..15
圖 3.4、平面多邊形: (a)第一個平面多邊形，(b) 第二個平面多邊形，
(c) 第三個平面多邊形，(d) 第四個平面多邊形，(e) 第五個平面多邊
形，(f) 第六個平面多邊形………………………………………..……16
圖 3.5、兩個多邊形平面示意圖。 .......................................................17
圖 3.6、兩平面交線示意圖。 ...............................................................18
圖 3.7、兩線交點示意圖。 ...................................................................19
圖 3.8、非目標物上關鍵點側視圖。 ...................................................20
圖 3.9、關鍵點上線條示意圖。 ...........................................................20
圖 3.10、斜面上四個關鍵點示意圖。 .................................................21
圖 3.11、其中兩點的方向向量關係圖。 .............................................21
圖 3.12、兩個關鍵點連線對應點示意圖。 .........................................22
圖 3.13、所有關鍵點連線對應點示意圖。 .........................................22
圖 3.14、找到關鍵點連線對應點虛擬程式碼。 .................................23
圖 4. 1、姿態估測相關演算法流程圖。 ..............................................24
圖 4. 2、鋒利邊緣示意圖。 ..................................................................25
圖 4. 3、不鋒利邊緣示意圖。 ..............................................................25
圖 4. 4、樣本點和邊緣之間誤差示意圖。 ..........................................28
圖 5. 1、所提出之姿態估測方法流程圖。 ..........................................31
圖 5. 2、未進行輪廓線偵測示意圖。 ..................................................32
圖 5. 3、輪廓線偵測示意圖。 ..............................................................32
圖 5. 4、邊緣圖。 ..................................................................................33
圖 5. 5、輪廓圖。 ..................................................................................33
圖 5. 6、多條霍夫線示意圖。 ..............................................................35
圖 5. 7、濾除霍夫線後示意圖。 ..........................................................35
圖 5. 8、模型邊界線控制點示意圖。 ..................................................36
圖 5. 9、控制點到霍夫線距離示意圖。 ..............................................36
圖 5. 10、本論文演算法系統架構圖。 ................................................40
圖 6. 1、三軸旋轉與三軸方向示意圖。 ..............................................42
圖 6. 2、目標物幾何物體原始影像: (a)較複雜目標物原始影像，(b)簡
單的目標物原始影像。 ..........................................................................43
圖 6. 3、三維 CAD 模型重建: (a)較複雜目標物 CAD 模型，(b)簡單幾
何目標物。…………………………………………………………......43
圖 6. 4、yaw 原始影像：(a)初始姿態，(b)y 軸轉 6 度姿態，(c) y 軸轉
18 度姿態，(d)y 軸轉 24 度姿態。........................................................45
圖 6. 5、yaw 結果影像：(a)初始姿態，(b)y 軸轉 6 度姿態，(c) y 軸轉
18 度姿態，(d)y 軸轉 24 度姿態。........................................................47
圖 6. 6、pitch 姿態原始影像：(a)初始姿態，(b)x 軸轉-6 度姿態，(c) x
軸轉-10 度姿態，(d)x 軸轉-12 度姿態。..............................................48
圖 6. 7、pitch 結果影像：(a)初始姿態，(b)z 軸轉-6 度姿態，(c) z 軸
轉-10 度姿態，(d)z 軸轉-12 度姿態。 ..................................................49
圖 6. 8、roll 姿態原始影像：(a)初始姿態，(b)x 軸轉-6 度姿態，(c) x
軸轉-12 度姿態，(d)x 軸轉-18 度姿態。..............................................51
圖 6. 9、roll 結果影像：(a)初始姿態，(b)z 軸轉 6 度姿態，(c) z 軸轉
12 度姿態，(d)z 軸轉 18 度姿態。........................................................52
圖 6. 10、遮擋物影像：(a)無遮擋物，(b)遮擋物一，(c)遮擋物二，(d)
遮擋物三，(e)遮擋物四。………………………………………………53
圖 6. 11、遮擋物估測結果：(a)無遮擋物估測結果，(b)遮擋物一估測
結果，(c)遮擋物二估測結果，(d)遮擋物三估測結果，(e)遮擋物四估
測結果。...................................................................................................55

表目錄
表 6. 1 yaw 的實驗比較.........................................................................45
表 6. 2 pitch 的實驗比較........................................................................48
表 6. 3 roll 的實驗比較 ..........................................................................50
表 6. 4 遮擋物的實驗比較 .....................................................................54

[1]Kinect攝影機, available at: http://www.xbox.com/zh-TW/Kinect 
[2]Point Cloud Library(PCL), available at: http://pointclouds.org/
[3]王傳崴(2013)，“應用於多重視角點雲合併之增強型3D RANSAC RGB-D 映射演算法”，淡江大學電機系碩士論文。
[4]H. Bat, T. Tuytelaars, and L. V. Gool, “SURF: Speeded Up Robust Features”, Proceedings European Conference Computer Vision, pp.430-443, Graz, Austria, 2006.
[5]C. Y. Tsai, C. W. Wang and W. Y. Wang “Design and Implementation of a RANSAC RGB-D Mapping Algoritm for Multi-View Point Cloud Registration,” IEEE International Automatic Control Conference, pp.367-370, 2013.
[6]Open Natural Interaction (OpenNI), available at: http://www.openni.org/ 
[7]G. Sithole, and G. Vosselman “Automatic Structure Detection in a Point Cloud of an Urban Landscape,” IEEE GRSS/ISPRS Joint Workshop on Data Fusion and Remote Sensing Over Urban Aress,  pp.67-71, 2003.
[8]G. Vosselman, B. G. H. Grote, G. Sithole, and T. Rabbani “Recognizing structure in laser scanner point clouds,” International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 46(8/W2),  pp.33-38, 2004.
[9]R. Schnabel, R. Wahl and R. Klein “Efficient RANSAC for Point-Cloud Shape Detection,” The Eurographics Association and Blackwell Publishing, Vol. 0, No.0, pp. 1-12, 2007. 
[10]L. Kobbelt, S. Campagna, J. Vorsatz, and H. P. Seidel “Interactive Multi-Resolution Modeling on Arbitrary Meshes” Computer Sciences Dapartment (IMMD9).
[11]P. J. Green, and B. W. Silvermant “Constructing the Convex Hull of a set of points in the Plane,” The Computer Journal Vol. 22, No.3, pp. 262-266, 1978.
[12]B. Bascle, P. Bouthemy, R. Deriche and F. Meyer “Tracking Complex Primitives in an Image Sequence,” IEEE IRISA/INRIA, Campus Universitaire de Beaulieu,  pp.426-431, 1994.
[13]M. Isard and A. Blake “Contour Tracking by Stochastic Propagation of Conditional Density,” In Proc. European Conf. Computer Vision, pp.345-356, 1996.
[14]G. Hager and K. Toyama, “The XVision system: A General-purpose Substrate for Portable Real-time Vision Applications,” Yale University Departhent of Computer Science, 1995.
[15]H. Kollning and H. –H. Nagel, “3d Pose Estimation by Fitting Image Gradients Directly to Polyhedral Models,” IEEE Int. Conf. On Computer Vision,  pp.569-574, 1995.
[16]V. Lepetit, P. Lagger and P. Fua, “Randomized Trees for Real-time Keypoint Recognition,” Ecole Polytechnique Federale de Lausanne (EPFL), 2005.
[17]Y. Amit and D. Geman, “Shape Quantization and Recognition with Ramdomized Trees,” Neural Computation, Vol. 9, No. 7, pp.1545-1588, 1997.
[18]C. Choi, and H. I. Christensen “Real-time 3D Model-based Tracking Using Edge and Keypoint Features for Robotic Manipulation,” International Conference on Robotics and Automation Anchorage Convention District IEEE pp.4048-4055, 2010.
[19]Boost, available at: http://www.boost.org/
[20]Eigen, available at: http://eigen.tuxfamily.org/index.php?title=Main_Page
[21]FLANN, available at: http://www.cs.ubc.ca/~mariusm/index.php/FLANN/FLANN
[22]VTK, available at: http://www.vtk.org/
[23]CUDA, available at: https://developer.nvidia.com/technologies/Libraries
[24]Qhull, available at: http://www.qhull.org/
[25]T. Drummond and R. Cipolla, “Real-time Visual Tracking of Complex Structures,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 7, pp.932-946, 2002.
[26]Open Source Computer Vision Library(OpenCV), available at: http://sourceforge.net/projects/opencvlibrary/ 
[27]R. O. Duda, and R. E. Hart, “Use of the Hough Transform to Detect Lines and Curves in Pictures,” CACM(15), Vol. 15, No.1, pp.11-15, 1972.
[28]R. L. PI0, “Euler Angle Transformations,” IEEE Transactions on Automatic Control, Vol. 11, No. 4, pp.707-715, 1966.
[29]S. Sarkka, “Notes on Quaternions,” Internal Technical Document, Helsinki University of Technology, 2007.
[30]Gimbal Lock, available at: http://coazure-code.blogspot.tw/2010/01/gimbal-lock.html