過去三維重建或辨識系統，大多數透過二維彩色影像和二維深度影像計算出三維空間座標，再進行三維影像，但這樣的運算量往往需耗費相當大的運算成本。本論文改採用可保留特徵向量方向性與彩色資訊的點雲系統，做為三維人臉重建與辨識的研究。點雲資料與二維影像不同的地方，傳統二維影像處理需不停計算二維影像每個像素點的資訊。點雲系統可以將二維彩色影像與二維深度影像合成為具三維空間座標的點雲模型做處理。可減少二維影像轉換三維模型運算複雜度，並建立三維空間座標KD-Tree查詢系統，加速三維模型查詢關鍵點的搜尋時間。

一般三維人臉重建取樣的樣本，往往需要使用設備昂貴的雷射掃描器，而本論文採用Microsoft KINCET感應器。它與昂貴的雷射掃描器相較下是屬於成本較為低廉的，KINECT具有深度資訊及彩色影像資訊，實驗環境以180°多視角掃描真實人臉三維表面影像，利用ICP演算法(Iteration Closest Point, ICP) 進行多視角人臉匹配，重建三維人臉模型。人臉辨識採用3D SIFT (3D Scale Invariant Feature Transform) 演算法提取人臉特徵關鍵點，並使用歐式距離計算三維空間座標特徵點與特徵點距離的權重關係。本論文提出的辨識方法在GavabDB公用資料庫辨識率可以達到83.6%。

In the past years, most of the three-dimensional reconstruction or recognition systems use two-dimensional image and its depth image to calculate the three-dimensional coordinates of the image to process the three-dimensional theme.  Such operations usually take a considerable amount of computing costs. This research proposes another approach, point cloud, which can preserve feature vectors and color information, for three-dimensional face reconstruction and recognition. In the conventional 2D approach, it keeps tracking the information of each pixel of the 2D image. On the other hand, the point cloud system directly synthesizes the 2D image and its depth image into a point cloud model with 3D coordinates. Therefore it can reduce the computation complexity significantly. It can further construct a 3D space coordinate KD-Tree query system to accelerate the query search speed for searching the key points of the 3D coordinate.

Generally, it uses some expensive equipments and laser scanners for three-dimensional facial reconstruction. In this research we try to use the Microsoft KINCET sensor to reconstruct the 3D human face. Compared with the expensive laser scanner, KINECT has the characteristics of cheap cost and can find the information of color image and depth image. In this research KINET is used to scan the human face in multi-view within 180. Then we use the iteration closest point (ICP) algorithm to match the multi-view human faces. By this approach the 3D data base group points of the human face can thus be established. The three-dimensional face model point cloud data via 3D SIFT (3D Scale Invariant Feature Transform) algorithm is applied to extract the feature key points. Then we use the three-dimensional coordinates of Euclidean distance to calculate the feature points and feature weights distance relationship to determine whether the face belongs to the same person. The experimental results show that under Gavab DB face database our approach has the recognition rate of 83.6%.

中文摘要	I 
英文摘要	II 
目錄	III
圖目錄	V
表目錄	VII
第一章 緒論	1
1.1研究動機	2
1.2研究主題與目標	3
1.3流程與系統架構	5
1.3.1三維人臉重建系統流程架構	5
1.3.2三維人臉辨識系統流程架構	7
1.4論文架構	8
第二章 國內外相關研究	9
2.1三維模型重建的相關研究	9
2.2 三維人臉辨識的相關研究	16
2.2.1局部特徵的方法	16
2.2.2全域特徵的方法	17
2.2.3樣板比對的方法	17
第三章 相關技術	19
3.1 點雲函式庫(Point Cloud Library)系統介紹	19
3.2  模型背景分割方法	21
3.3 點雲濾波器	23
3.4疊代最近點演算法 (Iteration Closest Point, ICP)	24
3.5 KD-Tree	27
3.6三維尺度不變特徵轉換 (3D Scale-Invariant Feature Transform)	31
3.6.1定位極值點	33
3.6.2特徵點描述	34
第四章 實驗結果	37
4.1研究設備與環境	37
4.2三維人臉重建實驗結果	38
4.3 三維人臉辨識實驗結果	41
4.3.1歐式距離	42
4.3.2公用資料庫實驗	44
4.3.3辨識效能比較	45
第五章 結論與未來展望	47

參考文獻	48

圖目錄
圖1.1微軟XBOX360 KINECT感測器[2]	4
圖1.2 三維人臉重建流程與架構	6
圖1.3三維人臉辨識流程與架構	8
圖2.1 史丹佛大學一百台陣列型攝影機建模系統[3]	11
圖2.2 Self-Recongigurable Camera Array系統[4]	12
圖2.3 使用多張影像上傳至雲端伺服器重建三維模型[5]	12
圖2.4 Autodesk 123D Catch應用程式重建模型介面[5]	13
圖2.5 立體模型量測基板[6]	13
圖2.6 使用立體模型量測基板重建臉部點雲模型[6]	14
圖2.7 環場攝影設備示意圖[7]	14
圖2.8 使用環場攝影設備重建之人臉模型[7]	15
圖2.9 微軟KINECT融合重建立體模型系統流程圖[8]	15
圖2.10 微軟KINECT融合重建出的立體模型[8]	15
圖3.1 加入或資助PCL的開發組織、研究所、公司[23]	20
圖3.2 KINEC近距離偵測模式與一般模式距離示意圖[25]	21
圖3.3 稀疏離群點分析和移除的效果對比圖[23]	23
圖3.4 三維KD-Tree結構圖[28]	28
圖3.5 KD-Tree 分割法	29
圖3.6 KD-Tree 對應的二元樹型態	30
圖3.7 3D-SIFT DoG高斯差分圖[32]	35
圖3.8 3D-SIFT 26相鄰取極大極小值示意圖[32]	35
圖3.9 3D-SIFT特徵點周圍梯度示意圖[33]	36
圖4.1 正面人臉	38
圖4.2 右側面臉	39
圖4.3 左側面臉	39
圖4.4 下巴面臉	40
圖4.5 利用3D-SIFT 擷取的全臉特徵關鍵點	42
圖4.6 Gavab DB 三維人臉模型資料	45


 
表目錄
表4.1 三維人臉模型重建比較表	40
表4.2本系統測試GavabDB效能分析	46
表4.3 本論文在GavabDB辨識率與其他論文比較表	46

[1]	http://www.face-rec.org/databases/
[2]	http://www.xbox.com/zh-TW/Kinect
[3]	B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams,M. Levoy, Mark H, “High Performance Imaging Using Large Camera Arrays,” In Proc. SIGGRAPH, 2005.
[4]	C. Zhang and T. Chen, “3A Self-Reconfigurable Camera Array,” in Eurographics Symposium on Rendering, 2004.
[5]	Wen-Kuo Hse and Pi-Ling Pai, “33D Modeling and Application from Panoramic Photography,” 2012台灣地理資訊學會年會暨學術研討會 , 2012.
[6]	陳煌杰：“獲取多方向影像於三維顏面模型重建 ”，國立台北科技大學土木防災研究所碩士論文，民國100年7月。
[7]	陳昭文,“多視角取像與三維重建技術,”影像與識別期刊, Vol.16 , No.2 , 2010.
[8]	S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. A. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. J. Davison, and A. Fitzgibbon, “KinectFusion: Real-time 3D reconstruction and interaction using a moving depth camera,”In Symposium on User Interface Software and Technology (UIST), 2011.
[9]	Ugur Halici and Tolga Inan, “3-D Face Recognition With Local Shape Descriptors,” IEEE Transactions on Information Forensics and Security, vol.7, no.2, pp.577,587, April 2012.
[10]	Y. Lee, H. Song, U. Yang, H. Shin, and K. Sohn, “Local feature based 3D face recognition,”In Proceedings of Audio- and Video-Based Biometric Person Authentication, AVBPA, pp. 909–918, 2005.
[11]	D. Huang, G. Zhang, M. Ardabilian, Wang, and L. Chen, “3D face recognition using distinctiveness enhanced facial repre- sentations and local feature hybrid matching,”In Fourth IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS), pp. 1–7, 2010.
[12]	T. C. Faltemier, K. W. Bowyer, and P. J. Flynn, “A region ensemble for 3D face recognition,”IEEE Transactions on Information Forensics and Security, 3(1):62–73, 2008.
[13]	C. McCool, V. Chandran, S. Sridharan, and C. Fookes, “3D face verification using a free-parts approach,”Pattern Recognition Letter, vol. 29, no. 9, pp. 1190–1196, Jul. 2008.
[14]	K. Sobottka and I. Pitas, “Face localization and facial feature extraction based on shape and color information,” 1996 Internet Conference on Image Processing, pp.483-486 vol.3, 16-19 Sep 1996.
[15]	M. Yachida, H. Wu, and Q. Chen, “Face Detection from Color Images Using a Fuzzy Pattern Matching Method,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no.6, pp. 557-563, June 1999.
[16]	H. Wu, Q. Chen, and M. Yachida, “Face Detection from Color Images Using a Fuzzy Pattern Matching Method,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 6, pp. 557-563, June 1999.
[17]	K. C. Yow and R. Cipolla, “A Probabilistic Framework for Perceptual Grouping of Features for Human Face Detection Method,”The Second International Conference on Automatic Face and Gesture Recognition, pp. 16-21, 1996.
[18]	C. Hesher, A. Srivastava, and G. Erlebacher, “A novel technique for face recognition using range imaging,” in Proc. 7th Int. Symp. Signal Processing and Its Applications, 2003, Jul. 2003, vol. 2, pp. 201–204.
[19]	H. Drira, B. Ben Amor, A. Srivastava, M. Daoudi and R. Slama, “3D Face Recognition under Expressions, Occlusions, and Pose Variations, ” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol.35, no.9, pp.2270-2283, Sept. 2013.
[20]	T. Sakai, M. Nagao, and S. Fujibayashi, “Line Extraction and Pattern Detection in a Photograph,” Pattern Recognition, vol.1, pp. 233-248, 1969.
[21]	I. Craw, H. Ellis, and J. Lishman, “Automatic Extraction of Face Features,” Pattern Recognition Letters, vol. 5, pp.183-187, 1987.
[22]	A. Yuille, P. Hallinan, and D. Cohen, “Features Extraction From Faces Using Deformable Templates,” 1989 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '89), pp.104-109, 1989.
[23]	http://pointclouds.org
[24]	朱德海，郭浩，蘇偉，點雲庫PCL學習教程，北京航空航天出版社，2012-10-1.
[25]	http://blogs.msdn.com/b/kinectforwindows/archive/2012/01/20/near-mode-what-it-is-and-isn-t.aspx
[26]	P. J. Besl and N. D. McKay, “A Method for Registration of 3D Shapes,” Trans. PAMI, vol.14, no. 2, pp. 239-254, 1992.
[27]	J. Friedman, J. Bentley, and R. Finkel,“An algorithm for finding best matches in logarithmic expected time,” ACM Trans. Math. Software, no. 3, pp. 209–226, Sep.1977.
[28]	http://en.wikipedia.org/wiki/K-d_tree
[29]	D. G. Lowe, “Object recognition from local scale-invariant features,” IEEE International Conference on The Proceedings of the Seventh, vol. 2, pp.1150-1157, Sep.1999.
[30]	C. Maes, T. Fabry, J. Keustermans, D. Smeets, P. Suetens, D. Vandermeulen, ”Feature Detection on 3D Face Surfaces for Pose Normalization and Recognition,” 2010 Fourth IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS), pp.1-6, Sept. 2010.
[31]	A.P. Witkin,“Scale-space filtering,” In International Joint Conference on Artificial Intelligence, Karlsruhe, Germany, pp. 1019-1022, 1983.
[32]	http://wenku.baidu.com/view/cf66d0b465ce050876321327.html
[33]	P. Scovanner, S. Ali, and M. Shah,“A 3-dimensional Sift Descriptor and its Application to Action Recognition,” In In Proc. of MULTIMEDIA '07, pages 357-360, 2007.
[34]	http://www.gavab.es/recursos_en.html
[35]	A.B. Moreno, A. Sanchez, E. Frias-Martinez and J.F. Velez,“Three-dimensional facial surface modeling applied to recognition,” Engineering Applications of Artificial Intelligence, vol.22, pp.1233-1244, 2010.
[36]	M. H. Mousavi, K. Faez, and A. Asghari,“Three dimensional face recognition using svm classifier,” In ICIS ’08: Proceedings of the Seventh IEEE/ACIS International Conference on Computer and Information Science, pp. 208–213, Washington, DC, USA,2008.