人機互動的系統不斷在進化中，由早期的打空單點輸入、鍵盤一維輸入、滑鼠的二維輸入。直到現在鍵盤與滑鼠依然是人機互動工具中最常見的工具之一。近年來人機互動輸入介面又有跳躍性的發展，例如觸控及“體感”。尤其是體感，他將人機介面帶入三維的輸入。但是現在還只能用於特定的介面上，例如遊戲機。因此我們希望能將系統建立在與人生活相關的電腦上。本系統利用深度資訊與Camshift追蹤演算法的結合，以擷取最前面的手部資訊，並判斷手的位移，進而控制電腦的軟體。

The HCI(Human–Computer Interaction) system has been advanced continually. Until now, keyboards and mouse are the most common tools for HCI. HCI system has been greatly developed in recent years, such as touch and “motion sensing”. Especially in motion sensing, it brings three-dimensional input into HCI interface which is only used in specific interface, such as video game players. Therefore, we hope the system can be build on computers which are closely related to our life. The system takes advantage in combining stereo vision method and Camshift Algorithm, captures the front information of hands and distinguish movements of hands to control computers’ software.

目錄
中文摘要................................................................................................... .......I
英文摘要.........................................................................................................II
目錄...............................................................................................................III
圖目.................................................................................................................VI
表目.................................................................................................................IX
第1章緒論.................................................................................................1
1.1 研究動機.............................................................................................1
1.2 研究方法.............................................................................................3
1.3 論文架構.............................................................................................5
第2章相關研究簡介.................................................................................6
2.1 相關研究.............................................................................................6
2.2 討論...................................................................................................12
第3章背景知識.......................................................................................13
3.1 彩色模型...........................................................................................13
3.2 相機模型...........................................................................................16
3.2.1 基本投影幾何.........................................................................16
3.2.2 矩陣旋轉與平移向量.............................................................19
3.2.3 Homography.............................................................................20
3.3.4 相機參數.................................................................................21
3.3立體成像............................................................................................25
3.3.1Epipolar Geometry ....................................................................25
3.3.2立體校正..................................................................................27
3.3.3立體匹配..................................................................................30
3.4CamShift..............................................................................................32
第4章基於立體視覺之手部辨識系統.......................................................37
4.1 系統流程...........................................................................................37
4.2輸入影像............................................................................................38
4.3影像校正............................................................................................38
4.4 深度建立...........................................................................................44
4.5 追蹤...................................................................................................45
4.5.1追蹤流程圖..............................................................................46
4.5.2追蹤步驟..................................................................................46
4.6手勢辨識............................................................................................50
第5章實驗結果.......................................................................................52
5.1 實驗環境...........................................................................................52
V
5.2深度資訊與實際距離之關係............................................................53
5.3複雜背景下之系統準確度分析........................................................54
5.4環境干擾下之系統準確度分析........................................................57
5.5光影變化環境下之系統準確度分析................................................60
5.6背景環境變化之系統準確度分析....................................................63
5.7不同手部姿勢對系統之影響............................................................66
5.8使用者不同對系統之影響................................................................69
第6章結論與未來展望...........................................................................71
6.1 結論...................................................................................................71
6.2 未來方向...........................................................................................72
參考文獻..................................................................................................73
圖目
圖1.1 觸控式輸入，(a)為IPhone、 (b)為Window7。......................................1
圖1.2 三維輸入的遊戲主機(a)Wii，(b)PS3，(c)XBox360。........................2
圖1.3 系統示意圖。..........................................................................................4
圖2.1 Young-Joon Chai所提出的手部偵測之方式[7]。.................................7
圖2.2 [8]提出之手勢辨識系統。......................................................................7
圖2.3 猜拳機[9]。.............................................................................................8
圖2.4 利用貝氏網路之手勢辨識[6]。.............................................................9
圖2.5 曾士宏提出之環狀ROI演算法[11]。...............................................10
圖2.6 [12]提出利用手勢控制音樂撥放器。..................................................10
圖2.7 指尖追蹤演算法流程示意圖[5]。.......................................................11
圖3.1 RGB色彩模型。....................................................................................14
圖3.2 HIS色彩模型。......................................................................................15
圖3.3 針孔成相示意圖。................................................................................16
圖3.4 以投影中心為原點之針孔成像。........................................................17
圖3.5 epipole Geometry示意圖。...................................................................26
圖3.6 二維與三維成像系統之關係示意圖。................................................27
圖3.7 左右影像之間之關係圖。....................................................................28
圖3.8 左右掃描線配對示意圖。....................................................................30
圖3.9 取內差點以增強準確度[4]。...............................................................31
圖3.10 匹配之結果[4]。...............................................................................31
圖3.11 (a)維輸入影像， (b)為籃球色彩之直方圖，(c)反投影之圖。.........32
圖3.12 Meanshift示意圖[19]。.......................................................................34
圖4.1 系統流程圖。........................................................................................37
圖4.2 輸入影像。............................................................................................38
圖4.3 8×5之棋盤圖。..................................................................................39
圖4.4 校正影像之角點。................................................................................39
圖4.5 (a)~(t)左webcam之20張棋盤影像。...............................................40
圖4.6 (a)~(t)右webcam之20張棋盤影像。...............................................41
圖4.7圖(a)~(c)為未校正之影像，圖(d)~(f)為校正後影像。.....................43
圖4.8 距離與深度間的關係。........................................................................45
圖4.9 追蹤流程圖。........................................................................................46
圖4.10 (a)膚色之影像，圖4.10 (b)之直方圖影像。...................................47
圖4.11 (a)為原圖， (b)為反投影的結果。....................................................48
圖4.12 將深度影像二值化的結果。............................................................48
圖4.13 結合深度與膚色資訊所得到之結果。..............................................49
圖4.14 追蹤結果。..........................................................................................49
圖4.15 開始區域。..........................................................................................50
圖4.16 系統起始示意圖。..............................................................................50
圖4.17 (a)~(d)四種位移方式。....................................................................51
圖5.1 Minoru雙眼webcam。.........................................................................52
圖5.2 (a)~(d) 本系統於複雜環境下之追蹤結果。.....................................54
圖5.3 (a)~(d) CamShift於複雜環境下之追蹤結果。................................55
圖5.4 (a)~(b) 本系統抓取失敗的情況。....................................................57
圖5.5 (a)為深度二值化之後的值，(b)抓取的目標。....................................57
圖5.6 (a)與(c)為反投影加上深度資訊的影像，(b)與(d)為抓取目標。..58
圖5.7 (a)~(d) 光影變化下系統運作之結果示意圖。.................................60
圖5.8(a)~(d) 光影變下加上背景相減之CamShift系統結果示意圖。......61
圖5.9(a)~(d) 背景變化下本系統之運作。.................................................63
圖5.10(a)~(d) 背景變化加上背景相減之CamShift系統結果圖。..........64
圖5.11(a)~(j)不同手部姿勢。......................................................................67
圖5.12 抓到手臂的位置。.............................................................................68
圖5.13(a)~(x) 不同的使用者。...................................................................69
表目
表5.1 閥值與使用者距離之關係..................................................................53
表5.2 本系統於複雜背景下之追蹤成功率..................................................56
表5.3 CamShift演算法於複雜背景下之追蹤成功率.................................56
表5.4 只用使用深度資訊之追蹤成功率......................................................59
表5.5 結合色彩資訊與深度資訊之追蹤成功率..........................................59
表5.6 光影變化對系統之追蹤成功率..........................................................62
表5.7光影變化對加上背景相減之CamShift系統之追蹤成功率..............62
表5.8本系統於背景變動環境中之準確率..................................................65
表5.9加上背景相減之CamShift系統於背景變動環境中之準確率.........65
表5.10 不同手部姿勢之系統準確率............................................................68

[1] G. Xu and Z. Zhang, Epipolar Geometry in Stereo, Motion and Object Recognition,Dordrecht: Kluwer, 1996
[2] O’Reilly ,“Learning OpenCV Computer Vision with the OpenCV Library”
[3] Gonzalez, Woods“Digital Image Processing”.
[4] Birchfield, S., Tomasi, C. “Depth discontinuities by pixel-to-pixel stereo” Computer Vision, 1998. Sixth International Conference on Digital Object Identifier: 10.1109/ICCV.1998.710850 Publication Year: 1998 , Page(s): 1073 - 1080
[5] S.Malik, “Real-time Hand Tracking and Finger Tracking for Interaction”, http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.135.1334&rep=rep1&type=pdf`.Dec.2003
[6] Heung-Il Suk, Bong-Kee Sin, and Seong-Whan Lee, “Robust Modeling and Recognition of Hand Gestures with Dynamic Bayesian Network”,IEEE International Conference on Pattern Recognition, p.1-4,Dec.2008.
[7] Young-Joon Chai, Dong-Heon Jang, And Tae-Yong Kim,“Homogeneity Based Background Subtraction for Robust Hand Pose Recognition : Focusing On the Digital Game Interface, ” Computer Graphics, Imaging and Visualisation(CGIV), pp.292~297, 2007.
[8] Jae-Ho Shin , Se-Kee Kil, “Hand Region Extraction and Gesture Recognition using entropy analysis,” Computer Science and Network
Security, IJCSNS International Journal, Vol. 6, No. 2A, pp. 216-222, 2006.
[9] 曹文潔，猜拳機，國立中央大學電機工程研究所碩士論文，民國九十六年。
[10]陳佳源，結合平移距離與傅立葉特徵之手勢辨識系統，中原大學資訊工程學系碩士論文，民國九十六年。
[11]曾士宏，即時手勢辨識系統之應用，淡江大學電機工程學系碩士在職專班，民國九十七年。
[12]吳怡明，手勢辨識應用於遙控音樂播放系統，國立台灣科技大學電機工程系碩士學位論文，民國九十八年。
[13]K. Fukunaga, Introduction to Statistical Pattern Recognition, Boston:Academic Press, 1990.
[14]R. Y. Tsai, “A versatile camera calibration technique for high accuracy 3D machine vision metrology using off -the-shelf TV cameras and lenses,” IEEE Journal of Robotics and Automation 3 (1987): 323–344.
[15]D. Comaniciu, “Nonparametric information fusion for motion estimation,”IEEE Conference on Computer Vision and Pattern Recognition (vol. 1,pp. 59–66), 2003.
[16]G. R. Bradski,“ Computer vision face tracking for use in a perceptual user interface,” Intel Technology Journal, 2nd Quarter, 1998.
[17]Z. Zhang, “A fl exible new technique for camera calibration,” IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (2000):
1330–1334.
[18]R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge, UK: Cambridge University Press, 2006.
[19]http://www.wisdom.weizmann.ac.il/~vision/courses/2004_2/files/mean_shift/mean_shift.ppt
[20]黃信榮，基於立體視覺手勢辨識的人機互動系統，國立中央大學電機工程研究所碩士論文，民國九十八年。