本論文製作一個基於手勢辨識技術之穿戴式裝置，並將其應用於互動式的手語學習。在穿戴式裝置上，將彎曲感測器(flex sensor)與三軸加速度計裝設在手套上，用來量測手指彎曲角度與手部傾斜方向，然後運用微控制器DE0-Nano進行手勢偵測與辨識。在手勢處於動態變化或靜止的偵測上，本文提出一通道邊界機制，更新適應性卡爾曼濾波器參數，過程雜訊共變異矩陣與觀測雜訊共變異矩陣，的數值以達到穩定的狀態追蹤與濾波效果。然後利用所研擬轉折點與變化幅度的判斷方法，偵測出手勢由動態變化轉換為靜止的時機點，以此作為後續手勢辨識的起始點。在手勢辨識部分，首先利用決策樹將不同手勢進行初步分類，接著再由機率類神經網路運算得到辨識結果，此結果經由藍牙傳送至智慧型行動裝置以手語遊戲的方式呈現。實驗結果顯示所研擬演算法的可行性，並且降低了辨識時的計算負擔。

This thesis develops a wearable devices on the basis of a hand gesture recognition technology, and applies it to an interactive sign language learning. Both flex sensors and a three-axis accelerometer of which mounted on the glove as an important part of the wearable device are used to measure the angles of the fingers and the direction of hand. The microcontroller DE0-Nano operates with the measurements to perform state detection and gesture recognition of the hand. For accurate tracking and detection of the state of the hand, the adaptive Kalman filter plays an important role. We develop a new method to update the parameters process noise covariance and measurement noise covariance to ensure good tracking and noise filtering. A criterion operating on this data that utilizes the concepts of turning points and amplitude variation is developed to determine the time instant after which it is better suited for hand gesture recognition. As the first step the proposed hand gesture recognition method utilizes the concept of decision tree to give a preliminary classification of the filtered measurements of different gestures. Then a probability neural network processes the data to produce a final result. The result is transmitted via Bluetooth to a smart mobile device and displayed in a proposed sign language game. Experiment shows the validity of the proposed algorithm; furthermore, the computational burden during recognition is reduced.

致謝
中文摘要 I
英文摘要 II
目錄 III
圖目錄 VI
表目錄 IX
第1章 緒論 1
1.1 研究動機 1
1.2 研究背景 4
1.3 論文架構 7
第2章 背景知識 8
2.1 感測元件 8
2.1.1 彎曲感測器 8
2.1.2 三軸加速度計 11
2.2 DE0-Nano微控制器 13
2.3 藍牙傳輸 15
2.4 Android的開發環境介紹 17
2.4.1 Android的運作原理 18
2.4.2 Android的版面配置 20
第3章 動態手勢辨識演算法則 24
3.1 手勢變化偵測 25
3.1.1 適應性卡爾曼濾波器 26
3.1.2 手勢辨識時機點的判斷 30
3.2 手勢辨識 35
3.2.1 決策樹 36
3.2.2 機率類神經網路 38
第4章 穿戴式動態手勢辨識裝置之應用 43
4.1 系統設計方法 43
4.2 主控端-穿戴式手勢裝置之硬體設計 44
4.2.1 電路設計 44
4.2.2 韌體整合 47
4.3 被控端-手語互動遊戲之軟體設計 50
4.3.1 手語互動遊戲之初始設定 50
4.3.2 手語互動遊戲之運作 53
第5章 實驗結果 59
5.1 實驗環境介紹 59
5.2 實驗結果 61
5.2.1 動態手勢追蹤 61
5.2.2 動態手勢偵測 64
第6章 結論與未來研究方向 67
6.1 結論 67
6.2 未來研究方向 68
參考文獻 69

圖1.1 台灣身心障礙者人數統計資料(2015年第一季) 1
圖1.2 情境模擬的示意圖 2
圖1.3 人機輸入介面的研究(a)以視覺影像為基礎(b)以微機電感測器為基礎 4
圖2.1 電阻式的彎曲感測器 8
圖2.2 彎曲感測器電阻變化原理示意圖 9
圖2.3 新舊感測器電阻變化比較圖 10
圖2.4 彎曲次數與電阻值變化量的關係圖 10
圖2.5 電容式三軸加速度計 11
圖2.6 電容式加速度計運作時的結構示意圖 12
圖2.7 加速度計動作比較圖 13
圖2.8 DE0-Nano	13
圖2.9 韌體整合架構圖 15
圖2.10 微網示意圖 16
圖2.11 分散網示意圖 17
圖2.12 Android生命週期流程圖 19
圖2.13 Android介面製作示意圖 20
圖2.14 程式視覺與程式邏輯關係示意圖 21
圖2.15 父子關係示意圖 22
圖3.1 動態手勢辨識演算法則 24
圖3.2 A手勢至B手勢變換之狀態示意圖 25
圖3.3 卡爾曼濾波器演算法流程圖 27
圖3.4 未超出通道邊界之示意圖 28
圖3.5 超出通道邊界之示意圖 29
圖3.6 偵測曲線交叉點示意圖 30
圖3.7 交叉點型態(a)轉折向下(b)轉折向上 31
圖3.8 偽轉折點示意圖 32
圖3.9 判別通道寬度示意圖 33
圖3.10 狀態偵測示意圖(a)靜止狀態(b)非靜止狀態 34
圖3.11 手勢辨識時機點的判斷 35
圖3.12 決策樹判斷條件對照表 36
圖3.13 決策樹分類架構圖 37
圖3.14 傳統機率類神經網路的架構 38
圖3.15 改良式機率類神經網路的架構 41
圖4.1 系統架構圖 43
圖4.2 硬體電路圖 44
圖4.3 LM317電路示意圖 47
圖4.4 I2C傳輸格式 48
圖4.5 I2C溝通模塊 49
圖4.6 以FPGA實現I2C協定示意圖 49
圖4.7 初始設定介面 50
圖4.8 初始設定狀態機 51
圖4.9 藍牙傳輸格式 52
圖4.10 遊戲進行中的畫面 53
圖4.11 遊戲進行狀態機 54
圖4.12 排名畫面 57
圖4.13 積分排名流程圖 58
圖5.1 ASL美國手語示意圖 60
圖5.2 I與J、D與Z比較 65
圖5.3 手勢N靜止狀態發生變化 66

表 5.1 對照組結果統計 62
表 5.2 實驗組結果統計 63
表 5.3 偵測各手勢辨識時機方法的成功率 64

[1]
S. Mitra and T. Acharya, “Gesture Recognition: A Survey,” IEEE Systems, Man, and Cybernetics Society, Vol. 37, May 2007, PP. 311-324.
[2]
M.K. Bhuyan, D.R. Neog and M.K. Kar, “Fingertip Detection for Hand Pose Recognition,” International Journal on Computer Science and Engineering, Vol. 4, Mar. 2012, pp. 501-511.
[3]
C.W. Ng and S. Ranganath, “Gesture recognition via pose classification,” International Conference on 2000. Proceedings. Pattern Recognition, Barcelona, Spain, Sep. 3, 2000, pp. 669-704.
[4]
S. Wagner, B. Alefs and C. Picus, “Framework for a portable gesture interface,” International Conference on Automatic Face and Gesture Recognition, Southampton, United Kingdom, Apr. 2-6, 2006, pp. 275-280.
[5]
G.Tofighi, A.N. Venetsanopoulos, K. Raahemifar, S. Beheshti and H. Mohammadi, “Hand posture recognition using K-NN and Support Vector Machine classifiers evaluated on our proposed HandReader dataset,” International Conference on Digital Signal Processing, Fira, Santorini, July. 1-3, 2013, pp. 1-5.
[6]
L. Dong, H.P. Wang, Z.Y. Hao and J.T. Liu, “Robust hand posture recognition based on RGBD images,” The 26th Chinese Control and Decision Conference, Changsha, China, May 31 - June. 2, 2014, pp. 2735-2740.
[7]
X.M. Yin, D. Guo and M. Xie, “Hand image segmentation using color and RCE neural network,” Robotics and Autonomous Systems, Vol. 34, Mar 2001, pp. 235-250.
[8]
林本煌，改良型分層隱藏式馬可夫模型於手勢識別之研究，國立臺北科技大學電機工程系研究所碩士學位論文，中華民國100年7月。
[9]
A. Ramamoorthy, N. Vaswani, S. Chaudhury and S. Anerjee, “Recognition of dynamic hand gestures”, Pattern Recognition, Vol. 36, No. 9, Sep., 2003, pp. 2069-2081.
[10]
S. Ge , Y. Yang and T. Lee, “Hand gesture recognition and tracking based on distributed locally linear embedding,” Image and VisionComput, Vol. 26, No. 12, Dec. 2008, pp. 1607–1620.
[11]
T. Chouhan, A. Panse, A.K. Voona and S.M. Sameer, “Smart glove with gesture recognition ability for the hearing and speech impaired,” IEEE Global Humanitarian Technology Conference - South Asia Satellite, Trivandrum, Kerala, Sept. 26-27, 2014, pp. 105-110.
[12]
L.J. Li and S.L. Dai, “Bayesian neural network approach to hand gesture recognition system,” IEEE Chinese Guidance Navigation and Control Conference, Yantai, China, Aug. 8-10, 2014, pp. 2019-2023.
[13]
R-H. Liang and M. Ouhyoung, A Real-time Continuous Alphabetic Sign Langua ge to Speech Conversion VR System, Computer Graphics Forum, Vol. 14 No. 3, Blackwell Science Ltd, 1995.
[14]
C. Oz and M.C. Leu, “Recognition of finger spelling of American sign language with artificial neural network using position/orientation sensors and data glove”, Advances in Neural Networks–ISNN 2005, Heidelberg, Germany, May 30, 2005, pp, 157-164,
[15]
A. El-Sawah, N.D. Georganas, and E.M. Petriu, “A prototype for 3-D hand tracking and posture estimation,” IEEE Transactions on Instrumentation and Measurement, Vol. 57, Aug 2008, pp. 1627-1636.
[16]
S. Saengsri, V. Niennattrakul, and C.A. Ratanamahatana, “TFRS: Thai finger-spelling sign language recognition system,” International Conference on Digital Information and Communication Technology and it's Applications, Bangkok, Thailand, May 16-18, 2012, pp. 457-462.
[17]
P. Neto, D. Pereira and J. Norberto Pires, “Real-time and continuous hand gesture spotting: an approach based on artificial neural networks,” IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, May. 6-10, 2013, pp. 178-183.
[18]
彭耀德，可攜式姿勢監控系統之改良與評估，國立成功大學醫學工程研究所碩士學位論文，中華民國96年7月。
[19]
柯元旦，Android核心剖析，松崗科技股份有限公司，中華民國101年。
[20]
D.F. Specht, “Probabilistic Neural Networks”, Neural Networks, Vol. 3. pp. 109-118, 1990.
[21]
房祥騫，基於手式輸入之即時通系統，淡江大學電機工程學系碩士學位論文，中華民國102年6月。
[22]
趙令文，Android App開發者必修16堂課，PCUSER電腦人，中華民國102年。