現今日常生活中，人們經常習慣從網路取得日常生活中的資訊，例如：天氣、新聞、運勢等等資訊，而這些資訊卻要每天重複操作著一樣的滑鼠、鍵盤動作來取得，日月累積下來浪費了不少時間，本研究將這些資訊整合做成一個系統，透過手勢辨識的方式，讓使用者方便地取得資訊，提供給使用者另一種取得這些資訊的方式。
本論文實作一個提供日常生活所需的資訊查詢系統，使用者藉由手勢來操作。利用手勢查詢資訊功能後，系統會顯示資訊和利用語音回報給使用者。在一個家庭環境中，因為家庭每位成員所需的生活資訊不同，而本論文使用了人臉辨識來區別家庭中的每位成員，進而達到個人化的服務與設定。在本論文中，我們使用主成分分析法來做人臉辨識。我們也使用主成分分析法來做手勢辨識，之後我們再用這數種手勢來操作系統。在我們的實驗中，本系統建置在家庭環境下，在少量的人臉樣本有不錯的辨識率，在手勢辨識上辨識率為93.8%，每張frame的處理速度為30到50毫秒。

Nowadays people are used to get daily information from Internet such as weather condition, news and financial information, among others. Though, in order to receiving these daily information, users have to repeat same mouse and keyboard actions, inducing waste of time and inconvenience. In order to improve these situations, we propose in this paper a system design that can easily get daily information without mouse and keyboard actions and make people's life more convenient and easier. In this proposed system, we have implemented an approach that provides daily information retrieved from Internet, where users can operate this system with his hands’ movements. Once selected the function by hand gestures, the system will report action information to users by synthetized speech. In a typical family, since each member has different requirements and needs, the system utilizes face recognition to identify each user, bringing up personalized services to each user.
In this paper, we use principal component analysis method to recognize faces as also hand gestures, and then a number of hand gestures and system controls are acquired and stored into this system. Results from a set of experiments indicate that the proposed system in a family environment with small-scale of face recognition show good performance as also good result in hand gesture recognition.

第一章 緒論	1
1.1 研究動機與目的	1
1.2 論文組織介紹	4
第二章 相關研究	6
2.1 人臉偵測	6
2.2 人臉特徵擷取和辨識	8
2.3 手部追蹤	10
2.4 手勢辨識	11
第三章 系統架構	13
3.1 系統流程	14
3.2 人臉辨識	16
3.2.1 人臉偵測	18
3.2.2 人臉辨識	23
3.3 手勢辨識	30
3.3.1 手部偵測	31
3.3.2 手部追蹤	34
3.3.3 手勢辨識	40
3.4 日常資訊查詢	46
3.4.1 天氣資訊	46
3.4.2 新聞資訊	49
3.4.3 運勢資訊	53
3.4.4 行事曆資訊	54
3.5 語音輸出	57
第四章 系統實作	59
4.1 開發平台	59
4.2 系統介面	61
4.3 系統實測	68
4.3.1 手勢辨識Off-line數據	68
4.3.2 手勢辨識On-line數據	69
第五章 結論與未來展望	71
5.1結論	71
5.2 未來展望	72
參考文獻	73
附錄 英文論文	80

圖目錄
圖1：系統架構圖	13
圖2：系統示意圖	14
圖3：系統流程圖	16
圖4：臉部辨識流程圖	17
圖5：矩形特徵	19
圖6：積分影像	20
圖7：haar-like特徵	21
圖8：串聯式分類器示意圖	23
圖9：人臉訓練流程	24
圖10：影像投影到人臉空間示意圖	29
圖11：手勢辨識主要流程圖	31
圖12：MeanShift過程示意圖	35
圖13：MeanShift示意圖	37
圖14：追蹤框圖	39
圖15：本系統定義的十種手勢	41
圖16：手勢訓練流程	42
圖17：正規化步驟圖	43
圖18：One手勢的五種角度	43
圖19：十種手勢訓練樣本	44
圖20：手勢辨識流程	45
圖21：部分天氣資訊	47
圖22：處理後的天氣資訊	48
圖23：顯示中的天氣資訊	48
圖24：新聞標題資訊	49
圖25：處理後的新聞標題資訊	50
圖26：顯示中的新聞標題資訊	50
圖27：新聞內文資訊	51
圖28：處理後的新聞內文資訊	52
圖29：顯示中的新聞內文資訊	52
圖30：處女座今日運勢資訊	53
圖31：處理後的處女座今日運勢資訊	54
圖32：顯示中的處女座今日運勢資訊	54
圖33：使用者的行事曆資訊	55
圖34：處理後的行事曆資訊	56
圖35：顯示中的行事曆資訊	56
圖36：Microsoft Visual studio 2005新增專案圖	60
圖37：顯示介面	61
圖38：辨識使用者前之系統畫面	62
圖39：辨識使用者後之系統畫面	63
圖40：倒數追蹤手部之系統畫面	64
圖41：手勢對應功能之系統畫面	65
圖42：設定介面	66

表目錄
表1：兩種操作方式比較	12
表2：SAPI支援表	58
表3：手勢功能對應表	67
表4：測試樣本結果數據表	69
表5：即時辨識結果	70

[1]Ming-Hsuan Yang, David J. Kriegman and Narendra Ahuja, “Detecting Faces in Images: A Survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, pp. 34-58, 2002.
[2]A. Lanitis, C.J. Taylor, and T.F. Cootes, “An Automatic Face Identification System Using Flexible Appearance Models,” Image and Vision Computing, vol. 13, no. 5, pp. 393-401, 1995.
[3]J. Yang and A. Waibel, “A Real-Time Face Tracker,” Proceedings 3rd IEEE Workshop on Applications of Computer Vision, pp. 142-147, 1996.
[4]S. McKenna, S. Gong, and Y. Raja, “Modelling Facial Colour and Identity with Gaussian Mixtures,” Pattern Recognition, vol. 31, pp. 1883-1892, 1998.
[5]P. Viola and M. Jones, “Rapid Object Detection Using A Boosted Cascade of Simple Features,” Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 511-518, 2001.
[6]R. Lienhart and J. Maydt, “An Extended Set of Haar-like Features for Rapid Object Detection,” International Conference on Image Processing, vol. 1, pp. 900-903, 2002.
[7]Paul Viola, Michael J. Jones and Daniel Snow, “Detecting Pedestrians Using Patterns of Motion and Appearance,” Ninth IEEE International Conference on Computer Vision, vol. 2, pp. 734-741, 2003.
[8]A. Rajagopalan, K. Kumar, J. Karlekar, R. Manivasakan, M. Patil, U. Desai, P. Poonacha, and S. Chaudhuri, “Finding Faces in Photographs,” Sixth International Conference on Computer Vision, pp. 640-645, 1998.
[9]H. Rowley, S. Baluja, and T. Kanade, “Neural Network-based Face Detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, pp. 23-38, 1998.
[10]H. Schneiderman and T. Kanade, “Probabilistic Modeling of Local Appearance and Spatial Relationships for Object Recognition,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 45-51, 1998.
[11]E. Osuna, R. Freund, and F. Girosi, “Training Support Vector Machines: An Application to Face Detection,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 130-136, 1997.
[12]Elham Bagherian and Rahmita Wirza O.K. Rahmat, “Facial Feature Extraction for Face Recognition: A Review,” International Symposium on Information Technology, vol. 2, pp. 1-9, 2008.
[13]M. Turk and A. Pentland, “Eigenfaces for Recognition,” Journal of Cognitive Neuroscience, vol. 3, pp. 71-86, 1991.
[14]M.A. Turk and A.P. Pentland, “Face Recognition Using Eigenfaces,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 586-591, 1991.
[15]H. Moon and P.J. Phillips, “Computational and Performance Aspects of PCA-based Face Recognition Algorithms,” Perception, vol. 30, pp. 303-321, 2001.
[16]P. Belhumeur, J. Hespanha and D. Kriegman, “Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, pp. 711-720, 1997.
[17]W. Zhao, R. Chellappa and P.J. Phillips, “Subspace Linear Discriminant Analysis for Face Recognition,” IEEE Transaction on Image Processing, 1999.
[18]P. J. Phillips, “Support Vector Machines Applied to Face Recognition,” Conference on Advances in Neural Information Processing Systems II, pp. 803-809, 1999. 
[19]M. S. Bartlett, H. M. Lades and T. Sejnowski, “Independent Component Representation for Face Recognition,” In Proceedings, of SPIE Symposium on Electronic Imaging: Science and Technology, vol. 3299, pp. 528-539, 1998.
[20]J. Lu, K.N. Plataniotis and A.N. Venetsanopoulos, “Face Recognition Using LDA-Based Algorithms,” IEEE Transactions on Neural Networks, vol. 14, pp. 195-200, 2003
[21]A. V. Nefian and M. H. Hayes III, “Hidden Markov Models for Face Recognition,” International Conference on Acoustics, Speech and Signal Processing, pp. 2721–2724, 1998.
[22]Crowley, J, “A Local Feature Based Human Face Recognition System,” Conference on Intelligent Information Systems, pp. 32-36, 1994.
[23]J. Huang, B. Weyrauch, B. Heisele and V. Blanz, “Component-Based Face Recognition with 3D Morphable Models,” Conference on Computer Vision and Pattern Recognition Workshop, pp. 85-92, 2004.
[24]A. Pentland, B. Moghaddam, and T. Starner, “View-Based and Modular Eigenspaces for Face Recognition,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 84-90, 1994.
[25]B. D. Lucas and T. Kanade, “An Iterative Image Registration Technique with an Application to Stereo Vision,” Proceedings of the 1981 DARPA Imaging Understanding Workshop, pp. 121–130, 1981.
[26]G. R. Bradski, “Computer Vision Face Tracking for Use in A Perceptual User Interface,” Intel Technology Journal, vol. 2, pp. 1-15, 1998.
[27]G. R. Bradski, “Real Time Face and Object Tracking as A Component of A Perceptual User Interface,” Fourth IEEE Workshop on Applications of Computer Vision, pp. 214-219, 1998.
[28]D. Comaniciu, V. Ramesh and P. Meer, “Real-Time Tracking of Non-Rigid Objects Using Mean Shift,” IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 142-149, 2000.
[29]Eng-Jon Ong and Richard Bowden, “A Boosted Classifier Tree for Hand Shape Detection,” Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 889-894, 2004.
[30]Yikai Fang, Kongqiao Wang, Jian Cheng and Hanqing Lu, “A Real-Time Hand Gesture Recognition Method,” IEEE International Conference on Multimedia and Expo, pp. 995-998, 2007.
[31]Nobuharu Yasukochi, Aya Mitome and Rokuya Ishii, “A Recognition Method of Restricted Hand Shapes in Still Image and Moving Image as A Man-machine Interface,” Conference on Human System Interactions, pp. 306-310, 2008.
[32]Birk, H., T.B. Moeslund and C.B. Madsen, “Real-Time Recognition of Hand Alphabet Gestures Using Principal Component Analysis,” 10th Scandinavian Conference on Image Analysis, 1997.
[33]Hanning Zhou, Dennis J. Lin and Thomas S. Huang, “Static Hand Gesture Recognition Based on Local Orientation Histogram Feature Distribution Model,” Conference on Computer Vision and Pattern Recognition Workshop, pp. 161-169, 2004.
[34]Zhaojie Ju and Honghai Liu, “A Unified Fuzzy Framework for Human Hand Motion Recognition,” IEEE Transactions on Fuzzy Systems, pp. 1-13, 2011.