本論文研製一個基於手勢輸入之即時通系統。此系統利用撓性感測器(flex sensor)與加速度計，搭配一組穿戴式機構，量測各手指的彎曲角度和手腕的傾斜方向，並將量測所得資訊以RS232傳輸協定傳至電腦端，經濾波後進行手勢辨識演算以產生相對應的文字。最後搭配TCP/IP協定，以自製之即時通介面與其他電腦進行互動。
在手勢辨識演算方面，本論文先採用一決策樹架構，將不同手勢進行初步分類，再由一改良式的機率類神經網路進行最後的手勢辨識。實驗顯示此方法降低了辨識時的計算負擔，提高了整體的辨識效率。

This thesis develops a messenger system based on hand gesture recognition technique. Flex sensors and an accelerometer combined with a wearable gesture sensing device are used to measure the bending angle of each finger and the wrist. The measurements are transmitted to PC by RS232 transmission protocol. After filtering to smooth the data, these data are processed through the gesture recognition algorithm to generate the corresponding text. Finally, the users can interact with the others by the homemade messenger system interface through TCP / IP protocol.
The proposed gesture recognition algorithm consists of two major steps, which first utilizes the concept of decision tree to classify the filtered measured data of different gestures, and then performs more delicate gesture recognition by a probability neural network to reach a verification result. Experiment shows that an improved verification rate with reduced computational burden of the problem is obtained by the proposed method.

目 錄

中文摘要.................................................................................................I
英文摘要................................................................................................II
目錄.......................................................................................................III
圖目錄...................................................................................................VI
表目錄....................................................................................................X
第1章	緒論....................................................................................................1
1.1	研究動機....................................................................................................1
1.2	研究背景....................................................................................................3
1.3	論文架構....................................................................................................6
第2章	背景知識....................................................................................................7
2.1	感測元件....................................................................................................7
2.1.1	撓性感測器....................................................................................................7
2.1.2	三軸加速度計....................................................................................................9
2.2	MSP430微控制器....................................................................................................14
2.3	通訊協定....................................................................................................16
2.3.1	I2C 同步串列通訊....................................................................................................16
2.3.2	RS232傳輸協定....................................................................................................19
2.3.3	TCP/IP傳輸協定....................................................................................................21
2.4	SQL資料庫....................................................................................................	22
第3章	具有辨識能力之穿戴式手部裝置....................................................................................................24
3.1	系統設計與架構....................................................................................................24
3.2	穿戴式手勢感測裝置之設計....................................................................................................28
3.2.1	flex sensor撓性感測器與放大電路....................................................................................................30
3.2.2	加速度計模組....................................................................................................33
3.2.3	以MSP430為核心之訊號轉換與傳輸處理模組....................................................................................................34
3.2.4	平均移動濾波器....................................................................................................37
3.3	手勢辨識演算法則....................................................................................................39
3.3.1	決策樹....................................................................................................41
3.3.2	機率類神經網路....................................................................................................43
第4章	即時通訊互動介面....................................................................................................49
4.1	通訊架構....................................................................................................49
4.2	即時通介面功能....................................................................................................57
4.2.1	個人化功能介紹....................................................................................................57
4.2.2	SQL資料庫設定....................................................................................................63
第5章	實驗結果....................................................................................................69
5.1	實驗模擬環境介紹....................................................................................................69
5.2	實驗結果....................................................................................................73
5.2.1	本文方法的辨識結果....................................................................................................73
5.2.2	三種方法的比較....................................................................................................76
第6章	結論與未來研究方向....................................................................................................80
6.1	結論....................................................................................................80
6.2	未來研究方向....................................................................................................81
參考文獻……………………………………………………..…...….82

圖目錄

圖 2.1   撓性感測器....................................................................................................7
圖 2.2   撓性感測器彎曲情況....................................................................................................8
圖 2.3   flex sensor電阻改變原理示意圖....................................................................................................8
圖 2.4   使用次數與總電阻變化量關係圖....................................................................................................9
圖 2.5   壓阻式加速度計結構運作圖....................................................................................................10
圖 2.6   電容式加速度計結構運作圖....................................................................................................11
圖 2.7   LSM303DLH電容式三軸加速度計架構圖....................................................................................................12
圖 2.8   加速度計動作前後圖....................................................................................................13
圖 2.9   MSP430之馮紐曼架構....................................................................................................15
圖 2.10  MSP430F1611內部架構圖....................................................................................................15
圖 2.11  I2C連線示意圖....................................................................................................17
圖 2.12  完整I2C資料轉換....................................................................................................18
圖 2.13  串列傳輸示意圖....................................................................................................19
圖 2.14  RS232傳輸格式....................................................................................................20
圖 2.15  TCP/IP網路模型....................................................................................................21
圖 2.16  TCP/IP封包格式....................................................................................................22
圖 3.1   使用情境示意圖....................................................................................................25
圖 3.2   本研究之系統架構圖....................................................................................................27
圖 3.3   穿戴式手勢感測裝置實體圖....................................................................................................28
圖 3.4   穿戴式手勢感測裝置的運作流程....................................................................................................29
圖 3.5   撓性感測器安裝情形....................................................................................................31
圖 3.6   撓性感測器放大電路....................................................................................................32
圖 3.7   感測器I2C接線方式....................................................................................................33
圖 3.8   RS232傳輸封包格式....................................................................................................35
圖 3.9   處理模組的運作狀態流程....................................................................................................36
圖 3.10  移動平均濾波器視窗移動....................................................................................................37
圖 3.11  濾波處理前後比較圖....................................................................................................38
圖 3.12  ASL美國手語示意圖....................................................................................................39
圖 3.13  決策樹判斷條件對照表....................................................................................................41
圖 3.14  決策樹架構圖....................................................................................................42
圖 3.15  傳統機率類神經網路的架構....................................................................................................43
圖 3.16  改良式機率類神經網路的架構....................................................................................................46
圖 4.1   本文即時通系統的網路架構....................................................................................................50
圖 4.2   連線狀態時客戶端的運作狀態流程圖....................................................................................................52
圖 4.3   連線狀態時伺服端的運作狀態流程圖....................................................................................................54
圖 4.4   伺服端在聊天時的運作狀態流程圖....................................................................................................56
圖 4.5   本文即時通介面....................................................................................................58
圖 4.6   通訊錄介面....................................................................................................59
圖 4.7   通訊錄聯絡人基本資料....................................................................................................59
圖 4.8   對話記錄記事本檔案....................................................................................................60
圖 4.9   對話記錄介面....................................................................................................61
圖 4.10  提示功能介面....................................................................................................62
圖 4.11  圖形化管理工具SSMSE....................................................................................................64
圖 4.12  SQL查詢語言的輸入....................................................................................................64
圖 4.13  建立資料庫....................................................................................................65
圖 4.14  建立資料表....................................................................................................67
圖 4.15  觀察資料表....................................................................................................67
圖 5.1   實驗環境....................................................................................................69
圖 5.2   實驗模擬介面....................................................................................................70
圖 5.3   SQL資料庫所儲存之訓練樣本....................................................................................................71
圖 5.4   儲存在Excel檔案之訓練樣本....................................................................................................72
圖 5.5   SQL Sever匯出匯入精靈....................................................................................................72
圖 5.6   實驗1之實驗模擬介面....................................................................................................73
圖 5.7   Y手勢於本文之類神經網路的機率密度函數....................................................................................................75
圖 5.8   實驗2之實驗模擬介面....................................................................................................76
圖 5.9   相似手勢之舉例....................................................................................................77
圖 5.10  本文之類神經網路的機率密度函數....................................................................................................78
圖 5.11  傳統類神經網路的機率密度函數....................................................................................................79

表目錄

表 3.1   不同方法之神經元總數與遞減率....................................................................................................48
表 5.1   本文方法之辨識結果....................................................................................................74
表 5.2   10次測試中不同方法的錯誤次數與辨識率....................................................................................................77

參考文獻

[1]A. El-Sawah, C. Joslin,  N.D. Georganas, and E.M. Petriu, “A Framework for 3D Hand Tracking and Gesture Recognition using Elements of Genetic Programming”, Fourth Canadian Conference on Computer and Robot Vision, CRV '07, pp. 495 – 502, May 28-30, 2007.
[2]P.K. Pisharady, P. Vadakkepat, and A.P. Loh, “Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds”, International Journal of Computer Vision, Vol. 101, Issue. 3, pp. 403-419, Feb 2013.
[3]M. Tang, “Recognizing Hand Gestures with Microsoft’s Kinect”, Tech-nical Report of Department of Electrical Engineering, Stanford University, Mar 2011.
[4]Z. Ren, J. Meng, J. Yuan, and Z. Zhang, “Robust hand gesture recognition with kinect sensor”, in Proc. MM '11 Proceedings of the 19th ACM international conference on Multimedia, ACM, New York, USA, pp. 759-760, Nov. 28 – Dec. 1, 2011.
[5]M. Nishiyama and K. Watanabe, “Wearable Sensing Glove With Embedded Hetero-Core Fiber-Optic Nerves for Unconstrained Hand Motion Capture”, IEEE Transactions on Instrumentation and Measurement , Vol. 58, No 12, pp. 3995-4000, Dec 2009.
[6]陳旻廷, 以資料手套輸入裝置之手勢操控虛擬人物系統之建構, 中原大學工業工程研究所碩士學位論文, 中華民國96年7月.
[7]J.L. Hernandez-Rebollar, R.W. Lindeman, and N. Kyriakopoulos, “A Multi-Class Pattern Recognition System for Practical Finger Spelling Translation”, Fourth IEEE Int'l Conference on Multimodal Interfaces (ICMI'02), Pittsburgh, USA, pp. 185-190, Oct 14-16, 2002.
[8]T.D. Bui and L.T. Nguyen, “Recognizing Postures in Vietnamese Sign Language With MEMS Accelerometers”, IEEE SENSORS JOURNAL, Vol. 7, No. 5, pp. 707-712, May 2007.
[9]D. Xu, “A Neural Network Approach for Hand Gesture Recognition in Virtual Reality Driving Training System of SPG”, The 18th International Conference on Pattern Recognition(ICPR'06), Vol.3, pp. 519 -522, 2006.
[10]楊世安, 應用於曲面設計之應變規彎度感測器之虛擬手套, 國立中興大學機械工程學系碩士學位論文, 中華民國94年.
[11]S.F. Ahmed, S.M.B. Ali, and S.S.M. Qureshi, “Electronic Speaking Glove for Speechless Patients A Tongue to a Dumb”, Sustainable Utilization and Development in Engineering and Technology (STUDENT) , pp. 56 -60, Nov 20-21, 2010.
[12]A.M.M. Ali, R. Ambar, M.M.A. Jamil, A.J.M. Wahi, S. Salim, “Artificial Hand Gripper Controller via Smart Glove for Rehabilitation Process”, 2012 International Conference on Biomedical Engineering (ICoBE), pp. 300 – 304, Feb 27-28, 2012.
[13]Spectrasymbol,  Flex Sensor Datasheet, 2006.
https://www.sparkfun.com/datasheets/Sensors/Flex/FLEXSENSOR
(REVA1).pdf
[14]彭耀德, 可攜式姿勢監控系統之改良與評估, 國立成功大學醫學工程研究所碩士學位論文, 中華民國96年7月.
[15]張家銘, 基於慣性感測的遠端控制虛擬釣魚系統, 淡江大學電機工程學系碩士學位論文, 中華民國101年6月.
[16]Texas Instruments, MSP430F161x MIXED SIGNAL MICROCON-TROLLER, 2011.
http://www.ti.com/lit/ds/symlink/msp430f1611.pdf
[17]陳慶逸, 林柏辰編著, VHDL數位電路實習與專題設計, 文魁資訊公司, 中華民國94年.
[18]王鴻儒作, SQL server 2005資料庫設計建置管理實務, 金禾資訊公司, 中華民國94年.
[19]李重寬，注音符號眼寫系統之可行性研究，中央大學電機工程研究所碩士論文，中華民國94年。
[20]美國楊百翰大學(Brigham Young University)網站
http://www.byui.edu/associations/asl/asl-useful-words
[21]D.F. Specht, “Probabilistic Neural Networks”, Neural Networks, Vol. 3. pp. 109-118, 1990.
[22]S.Vutinuntakasame, V.-R. Jaijongrak and S. Thiemjarus, “An Assistive Body Sensor Network Glove for Speech- and Hearing- Impaired Disabilities”, Body Sensor Networks, pp. 7 - 12, May 23-25, 2011.