本論文使用物件模型結合行人偵測器，發展機器人即時偵測與追蹤移動物體系統。三個主要研究議題包括結合移動特徵偵測與行人偵測器、改良機率物件模型、與物件模型訓練調整機制等。首先，將移動特徵偵測結合行人偵測器，以限定移動物件特徵的分佈範圍。不同群的移動特徵，將被訓練成不同的移動物件；其次，修改機率物件模型的描述，確保移動物體追蹤的強健性。使用單張影像的特徵取代多張影像的特徵訓練物件模型，以提高運算效率與進行線上建模。最後，利用移動物件辨識與追蹤的回授，調適物件模型訓練的條件。調適機制分別依據定值、專家表與模糊規則等方法設計。
發展的移動物件追蹤系統進一步與視覺式同時定位與建圖系統整合，成為同時定位、建圖與移動物體追蹤系統。使用擴張型卡爾曼過濾器估測系統狀態，以及使用加速強健特徵建立視覺式環境地圖。本研究也規劃多個實驗測試範例，驗證所發展的系統之效能。

This thesis presents an algorithm of robot visual moving-object tracking (MOT) based on the probabilistic object model with the pedestrian detector. Three major topics are investigated in the study including the combination of moving feature detection and pedestrian detection, the improvement of probabilistic object model, and the tuning mechanism of object model training. Firstly, the moving feature detection is integrated with a pedestrian detector to set the location boundaries of image features belonged to an object. Therefore, different groups of moving features will be trained as separated moving objects. Secondly, the representation of the probabilistic object model is modified to ensure the robustness of moving object tracking. Instead of using the image features from multiple image frames, the image features of one frame are used to train the object model for the purposes of computational efficiency as well as on-line training. Finally, the feedback of the recognition and tracking of moving objects is utilized to tune the training condition for the object model. The tuning mechanisms are designed based on fixed values, expert table, and fuzzy rules.
	The developed MOT is further integrated with the visual simultaneous localization and mapping (vSLAM) to form a simultaneous localization, mapping, and moving object tracking (SLAMMOT) system. The extended Kalman filter (EKF) is used to estimate the system states and the speeded-up robust features (SURFs) are employed to represent the visual environment map. Several experiments are carried out in this research to validate the performance of the developed systems.

目錄
中文摘要 I
英文摘要 II
目錄	III
圖目錄	V
表目錄	VII
第1章 序論	1
1.1 研究動機	1
1.2 研究目的	1
1.3 文獻探討	2
1.4 研究範圍	3
第2章 視覺式定位與建立地圖	5
2.1 狀態估測	5
2.1.1 擴張型卡爾曼過濾器	6
2.1.2 運動模型	6
2.1.3 靜態地標與移動物件	7
2.1.4 視覺量測模型	8
2.1.5 特徵初始化	12
2.2 地圖建立與管理	13
2.2.1 資料關聯	14
2.2.2 多組特徵描述向量	15
2.2.3 刪除地圖地標	16
2.2.4 新增地圖地標	16
2.2.5 動態Hessian值調整機制	17
2.2.6 特徵狀態更新	17
第3章 移動物體偵測與追蹤	18
3.1 偵測與追蹤演算法	19
3.1.1 動態特徵偵測	19
3.1.2 行人偵測器	20
3.1.3 物件資料關聯	23
3.2 物件模型	24
3.2.1 機率式物件模型	24
3.2.2 改良物件模型	25
3.3 物件組合	28
3.3.1 同質物件的組合	29
3.3.2 異質物件的排斥	30
3.4 物件辨識與物件追蹤	31
3.4.1 機率式物件辨識與追蹤	31
3.4.2 改良物件辨識與追蹤	32
3.4.3 同質物件辨識與追蹤範例	34
3.4.4 異質物件辨識與追蹤範例	36
第4章 物件模型調適機制	38
4.1 物件辨識與追蹤情況	38
4.2 定義調適條件	39
4.2.1 影響行人偵測器開啟的條件	39
4.2.2 影響物件訓練程序啟動的條件	40
4.2.3 影響動態特徵數量的條件	40
4.3 調適機制設計	41
4.3.1 辨識追蹤成功，行人偵測器未開啟	42
4.3.2辨識追蹤成功，行人偵測器開啟	42
4.3.3辨識追蹤未成功	42
4.3.4 調適機制流程	42
4.4 測試範例	44
4.4.1 範例：辨識追蹤成功，行人偵測器未開啟	44
4.4.2 範例：辨識追蹤成功，行人偵測器開啟	45
4.4.3 範例：未成功辨識追蹤物件	46
4.5 連續物件追蹤問題	47
第5章 修正物件模型調適機制	49
5.1 調適機制專家表	49
5.2 模糊條件	50
5.3 測試範例	52
5.3.1 單人連續辨識與追蹤測試範例	52
5.3.2 雙人連續辨識與追蹤測試範例	53
5.3.3 物件模型完整性測試範例	54
第6章 定位、建圖與移動物體追蹤系統整合	56
6.1 漸層式行人框架	56
6.1.1 漸層式行人框架測試範例	57
6.2 同時定位與建圖	58
6.2.1 實測範例圖示說明	58
6.2.2 SLAM實測範例	58
6.3 同時定位、建圖與行人追蹤	60
6.3.1 SLAM與兩位移動行人追蹤實測範例	60
6.3.2 SLAM與三位移動行人追蹤實測範例	63
第7章 研究成果與討論	68
7.1研究成果	68
7.2未來研究方向	68
參考文獻	70
附錄A  Jacobian矩陣	73

圖目錄
圖1.1 同時定位建圖與移動物體追蹤系統流程圖	4
圖2.1 雙眼攝影機	11
圖2.2 透視投影法示意圖	11
圖2.3 左攝影機與地標位置示意圖	11
圖2.4 左攝影機與地標位置示意圖	12
圖2.5 同時定位建圖與移動物體追蹤系統流程圖	14
圖2.6 地圖地標	15
圖2.7 動態搜尋視窗	15
圖3.1 同時定位建圖與移動物體追蹤系統流程圖	19
圖3.2行人偵測流程圖	22
圖3.3a 物件偵測器應用於追蹤行人	22
圖3.3b第312張影像	22
圖3.3c第313張影像	23
圖3.3d第314張影像	23
圖3.4 機率式物件模型應用於移動物體追蹤	25
圖3.5改良物件模型參數	27
圖3.6 物件模型訓練流程	28
圖3.7.a 物件組合表	29
圖3.7.b物件組合表	29
圖3.7.c物件組合表	29
圖3.8同質物件(左攝影機第198張影像)	30
圖3.9異質物件(左攝影機第344張影像)	31
圖3.10 物件辨識流程	34
圖3.11左攝影機第49張影像與左攝影機第50張影像	34
圖3.12左攝影機第70張影像與左攝影機第85張影像	35
圖3.13左攝影機第196張影像與左攝影機第198張影像	35
圖3.14左攝影機第401張影像與左攝影機第403張影像	35
圖3.15左攝影機第91張影像與左攝影機第92張影像	35
圖3.16左攝影機第360張影像	36
圖3.17左攝影機第365張影像	36
圖3.18左攝影機第371張影像	36
圖3.19左攝影機第374張影像	37
圖4.1 同時定位建圖與移動物體追蹤系統流程圖	38
圖4.2 行人物件模型建立程序	44
圖4.3第416張影像到第418張影像	45
圖4.4第398張影像到第400張影像	46
圖4.5第413張影像到第415張影像	47
圖4.6第183張影像、第184張影像與第243張影像	48
圖5.1 輸入變數pct的歸屬函數	51
圖5.2 輸出自由動態特徵數量歸屬函數	51
圖5.3輸出自由動態特徵數量百分比歸屬函數	51
圖5.4 輸出暫定動態特徵數量歸屬函數	51
圖5.5輸出動態特徵距離歸屬函數	52
圖5.6物件模型定值調適機制	52
圖5.7物件模型專家表調適機制	53
圖5.8物件模型模糊調適機制	53
圖5.9 物件模型定值調適機制	54
圖5.10 物件模型專家表調適機制	54
圖5.11 物件模型模糊調適機制	55
圖6.1 未使用漸層式行人框架測試範例	57
圖6.2使用漸層式行人框架測試範例	58
圖6.3 解說圖	58
圖6.4 0th影像：系統啟動	59
圖6.5 14th影像：系統啟動	59
圖6.6 4279th影像：狀態估測，建立環境地圖	59
圖6.7 757th影像：狀態估測，建立環境地圖	60
圖6.8 0th影像：系統啟動	61
圖6.914th影像：建立地圖	61
圖6.10 210th影像：辨識與追蹤物件	61
圖6.11 559 th影像：辨識與追蹤物件	62
圖6.12 726 th影像：辨識與追蹤物件	62
圖6.13 911th影像：辨識與追蹤物件	62
圖6.14 935 th影像：辨識與追蹤物件	63
圖6.15 963th影像：辨識與追蹤物件	63
圖6.16 1044th影像：辨識與追蹤物件	63
圖6.17 0th影像：系統啟動	64
圖6.18 14th影像：建立地圖	64
圖6.19 145th影像：辨識與追蹤物件	65
圖6.20 267th影像：辨識與追蹤物件	65
圖6.21 435th影像：辨識與追蹤物件	65
圖6.22 545th影像：辨識與追蹤物件	66
圖6.23 827th影像：辨識與追蹤物件	66
圖6.24 848th影像：辨識與追蹤物件	66
圖6.25 876th影像：辨識與追蹤物件	67
圖6.26 1117th影像：辨識與追蹤物件	67
圖6.27 1174th影像：辨識與追蹤物件	67

表目錄
表4.1 調適條件的門檻值定值調動規則	41
表4.2第一種狀況調適條件的門檻值調動	45
表4.3第二種狀況調適條件的門檻值調動	46
表4.4第三種狀況調適條件的門檻值調動	47
表5.1 調適機制專家表	50
表5.2 模糊規則庫	50
表5.3 不同調適機制的辨識與追蹤成功張數(成功率%)	53
表5.4 不同調適機制的雙人辨識與追蹤成功張數(成功率%)	53

[1]	M.W.M.G. Dissanayake, P. Newman, S. Clark, H. Durrant-Whyte, and M. Csorba, “A Solution to the Simultaneous Localization and Map Building (SLAM) Problem,” IEEE Transactions on Robotics and Automation, vol.17, no. 3, pp. 229–241, 2001.
[2]	S. Se, D.G. Lowe, and J.J. Little, “Vision-Based Global Localization and Mapping for Mobile Robots,” IEEE Transactions on Robotics, vol.21, no.3, pp.364-375, 2005.
[3]	A.J. Davison, I.D. Reid, N.D. Molton, and O. Stasse, “MonoSLAM Real Time Single Camera SLAM,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.29, no.6, pp.1052-1067, 2007.
[4]	L.M. Paz, P. Pinies, J.D. Tardos, and J. Neira, “Large-Scale 6-DOF SLAM with Stereo-in-Hand,” IEEE Transactions on Robotics, vol.24, no.5, pp.946-957, 2008.
[5]	Z. Wang, S. Huang, and G. Dissanayake, Simultaneous Localization and Mapping: World Scientific Publishing Co., 2011.
[6]	M. Montemerlo, FastSLAM: a factored solution to the simultaneous localization and mapping problem with unknown data association, Ph.D. dissertation, Carnegie Mellon University, Pittsburgh, PA, 2003.
[7]	S.J. Davey, “Simultaneous Localization and Map Building Using the Probabilistic Multi-Hypothesis Tracker,” IEEE Transactions on Robotics, vol.23, no.2, pp.271-280, 2007.
[8]	R. Sim, P. Elinas and J.J. Little, “A Study of the Rao-Blackwellised Particle Filter for Efficient and Accurate Vision-Based SLAM,” International Journal of Computer Vision, vol.74, no.3, pp.303-318, 2007.
[9]	OpenSLAM, website:http://openslam.org/ (accessed 12/10/2014)
[10]	D. Nister, O. Naroditsky, and J. Bergen, "Visual odometry," in Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, 2004, pp. I-652-I-659 Vol.1.
[11]	D. Nistér, O. Naroditsky, and J. Bergen, "Visual odometry for ground vehicle applications," Journal of Field Robotics, vol. 23, pp. 3-20, 2006.
[12]	D. Scaramuzza and F. Fraundorfer, "Visual Odometry [Tutorial]," Robotics & Automation Magazine, IEEE, vol. 18, pp. 80-92, 2011.
[13]	F. Fraundorfer and D. Scaramuzza, "Visual odometry: Part II: Matching, robustness, optimization, and applications," Robotics & Automation Magazine, IEEE, vol. 19, pp. 78-90, 2012.
[14]	J. Sola, Towards visual localization, mapping and moving objects tracking by a mobile robot: a geometric and probabilistic approach, Ph.D. dissertation, Institut National Polytechnique de Toulouse, 2007.
[15]	S. Wangsiripitak, and D.W Murray, “Avoiding moving outliers in visual SLAM by tracking moving objects,” Proceedings of IEEE International Conference on Robotics and Automation, 2009.
[16]	D. Migliore, R. Rigamonti, D. Marzorati, M. Matteucci, D.G. Sorrenti, “Use a Single Camera for Simultaneous Localization And Mapping with Mobile Object Tracking in dynamic environments,” Proceedings of IEEE International Conference on Robotics and Automation, 2009.
[17]	C.H. Hsiao, and C.C. Wang, "Achieving Undelayed Initialization in Monocular SLAM with Generalized Objects Using Velocity Estimate-based Classification," Proceedings of IEEE International Conference on Robotics and Automation, 2011.
[18]	Y.-T. Wang, C.-H. Sun, and M.-J. Chiou, "Detection of moving objects in image plane for robot navigation using monocular vision," EURASIP Journal on Advances in Signal Processing, vol. 2012, p. 29, 2012.
[19]	W. Choi, C. Pantofaru, and S. Savarese, "A General Framework for Tracking Multiple People from a Moving Camera," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, pp. 1577-1591, 2013.
[20]	N. Dalal and B. Triggs, “Histograms of Oriented Gradients for Human Detection,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[21]	V. Ferrari, M. Marin-Jimenez, and A. Zisserman, “Progressive Search Space Reduction for Human Pose Estimation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[22]	P. Viola and M. Jones, “Robust Real-Time Face Detection,” Int’l J. Computer Vision, vol. 57, no. 2, pp. 137-154, 2003.
[23]	M. Jones and J. Rehg, “Statistical Color Models with Application to Skin Detection,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
[24]	B. Leibe, A. Leonardis, and B. Schiele, "Robust Object Detection with Interleaved Categorization and Segmentation," International Journal of Computer Vision, vol.77, no.1-3, pp. 259-289, 2008.
[25]	B. Leibe, K. Schindler, N. Cornelis, and L. van Gool, "Coupled Detection and Tracking from Static Cameras and Moving Vehicles," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 10, pp. 1683-1698, 2008.
[26]	A. Ess, B. Leibe, K. Schindler, and L. van Gool, "Robust Multiperson Tracking from a Mobile Platform," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.31, no.10, pp.1831-1846, 2009.
[27]	M.D. Breitenstein, F. Reichlin, B. Leibe, E. Koller-Meier, and L. Van Gool, "Online Multiperson Tracking-by-Detection from a Single, Uncalibrated Camera," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no.9, pp.1820-1833, 2011.
[28]	陳國瑋，使用線上稀疏呈現的物件模型進行機器人並行追蹤與建圖，淡江大學機械與機電工程學系碩士論文，2012。
[29]	R. Smith, M. Self, and P. Cheeseman, "Estimating Uncertain Spatial Relationships in Robotics," in Autonomous Robot Vehicles, I.J. Cox and G.T. Wilfong, Eds., Springer-Verlog, pp.167-193, 1990.
[30]	H. Durrant-Whyte and T. Bailey, 2006, Simultaneous Localization and Mapping: Part 1, IEEE Robotics and Automation Magazine.
[31]	T. Bailey and H. Durrant-Whyte, 2006, Simultaneous Localization and Mapping: Part 2, IEEE Robotics and Automation Magazine.
[32]	K. Konolige and M. Agrawal, "FrameSLAM: From bundle adjustment to real-time visual mapping," IEEE Transactions on Robotics, vol. 24, pp. 1066-1077, 2008.
[33]	B. Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon, "Bundle Adjustment — A Modern Synthesis," in Vision Algorithms: Theory and Practice. vol. 1883, B. Triggs, A. Zisserman, and R. Szeliski, Eds., ed: Springer Berlin Heidelberg, 2000, pp. 298-372.
[34]	Y.-T. Wang, C.-T. Chi, and S.-K. Hung, "Robot Visual SLAM in Dynamic Environments," Advanced Science Letters, vol. 8, pp. 229-234, 2012.
[35]	D.H. Ballard, “Generalizing the hough transform to detect arbitrary shapes,” Pattern Recognition, vol.13, no.2, pp.111-122, 1981.
[36]	S. Gillijns and B. De Moor, "Unbiased minimum-variance input and state estimation for linear discrete-time systems," Automatica, vol.43, pp.111–116, 2007.
[37]	H. Blom and Y. Bar-Shalom, "The interacting multiple-model algorithm for systems with Markovian switching coefficients," IEEE Transactions on Automatic Control, vol.33, pp.780-783, 1988.
[38]	H. Bay, A. Ess, T. Tuytelaars and L. Van Gool, “SURF: Speeded-Up Robust Features”, Computer Vision and Image Understanding, vol.110, pp.346-359, 2008.
[39]	C.C. Wang, C. Thorpe, S. Thrun, M. Hebert, and H. Durrant-Whyte, “Simultaneous Localization, Mapping and Moving Object Tracking,” The International Journal of Robotics Research, vol. 26, no. 09, pp. 889-916, 2007.
[40]	M.S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking, IEEE Transaction on Signal Processing, vol.50, no.2, pp.174-188, 2002.
[41]	S. Hutchinson, G.D. Hager, & P.I. Corke, "A Tutorial on Visual Servo Control," IEEE Transactions on Robotics and Automation, vol.12, no.5, pp.651-670, 1996.
[42]	馮盈捷，使用尺度與方向不變特徵建立機器人視覺式SLAM知稀疏與續存性地圖，淡江大學機械與機電工程學系碩士論文，2011。
[43]	林冠瑜，"使用低階攝影機實現機器人視覺式SLAM"，淡江大學機械與機電工程學系碩士論文，2012。
[44]	G. Shakhnarovich, T. Darrell and P. Indyk, Nearest-neighbor methods in learning and vision, The MIT Press, 2005.
[45]	洪敦彥，基於擴張型卡爾曼過濾器的機器人視覺式同時定位、建圖、與移動物體追蹤，淡江大學機械與機電工程學系碩士論文，2010。
[46]	J. MacQueen, “Some Methods for Classification and Analysis of Multivariate Observations”, in Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, pp. 281-297, 1967.
[47]	D.G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, 60, 2, pp. 91-110, 2004.