本論文提出基於FaceNet之偽裝人臉身份辨識系統，其目的為當辨識目標穿著偽裝物品時，系統可以正確識別身份。系統使用Single Shot Multibox Detector (SSD)檢測臉部區域，再使用FaceNet來提取臉部特徵。將乾淨臉的臉部特徵與身份類別輸入到支持向量機進行訓練，系統能得到辨識身份的分類器。以穿戴偽裝物品的偽裝臉測試系統，特徵萃取步驟與訓練階段相同，使用SSD檢測臉部區域，再利用FaceNet萃取臉部特徵，系統將測試資料的人臉特徵輸入到分類器進行辨識，能得到穿戴偽裝物的人員身份。實驗資料使用AR資料庫提供的人臉圖像。偽裝身份辨識系統使用100個身份訓練系統，使用相同身分的偽裝臉測試系統，系統辨識正確率為95.66%。提出方法與Abdenour Hadid等人提出辨識系統相比，所提出系統較能抵抗偽裝物件的影響。

This thesis proposes a disguised face recognition system based on FaceNet, which is used to recognize the identification of a person wearing disguised items. The proposed system uses the Single Shot Multibox Detector (SSD) to detect the face region of a person in an image. The facial features of a clean face are first extracted by the FaceNet and then used to train the support vector machine (SVM) classifier. For testing the accuracy of the proposed system, the inputs are the disguised faces of the identities in the training set. SDD is used to detect the facial region, FaceNet is used to extract facial features, and SVM classifier is used to find the identity of the person who wearing disguised items. To train and test the proposed disguised face recognition system, we use images provided in the AR dataset. We use the clean images of 100 identities to train our system and then test the trained systems with the same identities who wear disguised items. The system identification accuracy rate is 95.66%. Compared with the disguised recognition system proposed by Abdenour Hadid et al., our face recognition system is more resistant to the effects of disguised objects.

目錄
致謝	I
摘要	II
英文摘要	III
目錄	IV
圖目錄	VII
表目錄	XI
第一章	緒論	1
1.1	研究背景	1
1.2	研究動機與目的	6
第二章	相關研究	8
2.1	卷積式神經網路介紹	8
2.1.1	大規模視覺辨識比賽	9
2.1.2	卷積層（Convolutional Layer）	10
2.1.3	池化層（Pooling Layer）	11
2.1.4	線性整流層（Rectified Linear Units Layer, ReLU Layer）	12
2.1.5	過度擬合（Overfitting）	13
2.1.6	資料強化（Data Augmentation）	15
2.2	t-Distributed Stochastic Neighbor Embedding  （t-SNE）	16
2.3	Local Binary Patterns（LBP）	19
2.4	Improving the Recognition of Faces Occluded by Facial Accessories	22
第三章	系統架構	28
3.1	臉部偵測	29
3.2	FaceNet特徵萃取	33
3.3	特徵分類	41
第四章	實驗結果	44
4.1	使用資料庫	44
4.2	實驗環境	45
4.3	臉部偵測信心閥值	46
4.3.1	乾淨臉測試	46
4.3.2	偽裝臉測試	48
4.3.3	平均臉檢測	50
4.3.4	實驗結論	52
4.4	偽裝身份辨識系統	56
4.4.1	訓練資料	56
4.4.2	測試資料	57
4.4.3	辨識結果	58
4.5	系統與過去方法比較	60
第五章	結論	64
參考文獻	66

圖目錄
圖2- 1神經元的架構	8
圖2- 2池化層計算	12
圖2- 3 ReLU輸出	13
圖2- 4當發生過度擬合時資料分佈	14
圖2- 5 MNIST手寫數字測試資料[17]	17
圖2- 6 MNIST手寫數字經PCA處理後的結果	18
圖2- 7 MNIST手寫數字經t-SNE處理後的結果	18
圖2- 8灰階化的圖像	20
圖2- 9 LBP 計算說明-1	20
圖2- 10 LBP 計算說明-2	21
圖2- 11 LBP計算說明-3	21
圖2- 12左方為灰階圖像，右方為圖像的LBP特徵	21
圖2- 13測試圖像經過遮蔽物檢測器判斷遮蔽區域，保留非遮蔽區域[28]	22
圖2- 14 提出方法流程圖[28]	23
圖2- 15遮蔽物檢測器系統流程[28]	23
圖2- 16 AR Face Database部分資料[28]	25
圖2- 17 前處理後的圖像[28]	25
圖2- 18 實驗結果[28]	27
圖3- 1 訓練身份辨識系統流程圖	28
圖3- 2 測試身份辨識系統流程圖	29
圖3- 3 OpenCV卷積式神經網絡的臉部偵測器系統流程圖	30
圖3- 4 左方為原始圖像，右方為經過均值減法處理後的圖像	31
圖3- 5 訓練資料經過臉部偵測，偵測臉部位置	32
圖3- 6 擷取臉部圖像	33
圖3- 7 平均多張正臉製作成平均臉	33
圖3- 8 FaceNet的主要架構[40]	34
圖3- 9 GoogLeNet架構[41]	35
圖3- 10 Inception架構[41]	35
圖3- 11 FaceNet將資料映射到高維球體的一個點	36
圖3- 12 主要資料-錨點圖像（Anchor），不同類別資料-負向圖像（Negative），相同類別資料-正向圖像（positive）	37
圖3- 13 最大化錨點圖像與負向圖像的距離；最小化錨點圖像與正向圖像的距離	37
圖3- 14 經過FaceNet運算後會得到128維度的特徵	37
圖3- 15 MS-Celeb-1M datase的資料內容[45]	39
圖3- 16 原始資料使用t-SNE查看資料分佈	40
圖3- 17 FaceNet特徵使用t-SNE查看資料分佈	40
圖3- 18 LineB會比LineA擁有較高的穩健性	41
圖4- 1 表情變化	44
圖4- 2 乾淨臉部的光源變化	45
圖4- 3 穿戴太陽眼鏡與光源變化	45
圖4- 4 穿戴圍巾與光源變化	45
圖4- 5 部分測試資料，包含10個身份乾淨無偽裝臉部圖像	47
圖4- 6 乾淨臉測試結果	47
圖4- 7 使用臉部偵測時，偵測到多餘的區域	48
圖4- 8 部分測試資料，包含10個身份墨鏡偽裝臉部圖像	48
圖4- 9 部分測試資料，包含10個身份圍巾偽裝臉部圖像	49
圖4- 10 偽裝臉測試結果	49
圖4- 11 使用臉部偵測當閥值設為0.9時，會有3張偵測錯誤	50
圖4- 12 使用臉部偵測當閥值設為0.8時，能順利偵測臉部圖像實驗結論	50
圖4- 13 檢驗臉部偵測結果的平均臉	50
圖4- 14 偵測器偵測錯誤區域	51
圖4- 15 平均臉與測試臉的歐基里德距離正規化結果	52
圖4- 16 整合乾淨臉測試結果與偽裝臉測試結果	53
圖4- 17 臉部偵測錯誤圖像	54
圖4- 18 臉部偵測錯誤區域	54
圖4- 19 人眼剖面圖[53]	55
圖4- 20 訓練資料包含10個身份正視無偽裝臉部圖像	56
圖4- 21 第一身份的所有訓練圖片	56
圖4- 22 測試資料包含10個身份墨鏡偽裝臉部圖像	57
圖4- 23 測試資料包含10個身份圍巾偽裝臉部圖像	57
圖4- 24 第一身份的所有測試圖片	58
圖4- 25 辨識結果的錯誤圖像	58
圖4- 26 不同人數時，系統準確度的變化	60
圖4- 27 實驗所使用的資料數量	61
圖4- 28 訓練與測試的使用資料	61
圖4- 29 提出系統與[28]比較結果	62

表目錄
表2- 1 遮蔽檢測的結果[28]	26
表4- 1 臉部偵測結果的混淆矩陣	53
表4- 2 辨識結果的混淆矩陣	59

[1]	M. Arsenovic, S. Sladojevic, A. Anderla, and D. Stefanovic, "FaceTime — Deep learning based face recognition attendance system," in 2017 IEEE 15th International Symposium on Intelligent Systems and Informatics (SISY), 2017, pp. 53-58.
[2]	W. Ya, B. Tianlong, D. Chunhui, and Z. Ming, "Face recognition in real-world surveillance videos with deep learning method," in 2017 2nd International Conference on Image, Vision and Computing (ICIVC), 2017, pp. 239-243.
[3]	AustralianGovernment. smartgates  [Online]. Available: https://www.abf.gov.au/entering-and-leaving-australia/smartgates.
[4]	MarketsandMarkets. (2017). Facial Recognition Market  [Online]. Available: https://www.marketsandmarkets.com/Market-Reports/nanophotonics-advanced-technologies-and-global-market-125.html.
[5]	P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, pp. I-I.
[6]	R. Lienhart and J. Maydt, "An extended set of Haar-like features for rapid object detection," in Proceedings. International Conference on Image Processing, 2002, vol. 1, pp. I-I.
[7]	W. Liu et al., "Ssd: Single shot multibox detector," in European conference on computer vision, 2016: Springer, pp. 21-37.
[8]	H. Abdi and L. J. Williams, "Principal component analysis," Wiley Interdisciplinary Reviews: Computational Statistics, vol. 2, no. 4, pp. 433-459, 2010.
[9]	T. Ojala, M. Pietikainen, and D. Harwood, "Performance evaluation of texture measures with classification based on Kullback discrimination of distributions," in Proceedings of 12th International Conference on Pattern Recognition, 1994, vol. 1, pp. 582-585 vol.1.
[10]	N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 2005, vol. 1, pp. 886-893 vol. 1.
[11]	P. E. Greenwood and M. S. Nikulin, A guide to chi-squared testing. John Wiley & Sons, 1996.
[12]	C. Cortes and V. Vapnik, "Support-vector networks," Machine Learning, vol. 20, no. 3, pp. 273-297, 1995/09/01 1995.
[13]	H. Jia and A. M. Martinez, "Support Vector Machines in face recognition with occlusions," in 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 136-141.
[14]	J. M. Keller, M. R. Gray, and J. A. Givens, "A fuzzy K-nearest neighbor algorithm," IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-15, no. 4, pp. 580-585, 1985.
[15]	K. Q. Weinberger, J. Blitzer, and L. K. Saul, "Distance metric learning for large margin nearest neighbor classification," in Advances in neural information processing systems, 2006, pp. 1473-1480.
[16]	J. A. Hartigan and M. A. Wong, "Algorithm AS 136: A k-means clustering algorithm," Journal of the Royal Statistical Society. Series C (Applied Statistics), vol. 28, no. 1, pp. 100-108, 1979.
[17]	Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[18]	Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, p. 436, 2015.
[19]	R. J. Williams and D. Zipser, "A learning algorithm for continually running fully recurrent neural networks," Neural Computation, vol. 1, no. 2, pp. 270-280, 1989.
[20]	A. Hannun et al., "Deep speech: Scaling up end-to-end speech recognition," arXiv preprint arXiv:1412.5567, 2014.
[21]	C. Ding and D. Tao, "Trunk-branch ensemble convolutional neural networks for video-based face recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 4, pp. 1002-1014, 2017.
[22]	A. K. Jain, R. Bolle, and S. Pankanti, Biometrics: personal identification in networked society. Springer Science & Business Media, 2006.
[23]	A. K. Jain, A. Ross, and S. Prabhakar, "An introduction to biometric recognition," 2004.
[24]	AppleUAE. (2017). iPhone X — Introducing iPhone X — Apple  [Online]. Available: https://castnet.nctu.edu.tw/castnet/article/11140?issueID=665.
[25]	G. Righi, J. J. Peissig, and M. J. Tarr, "Recognizing disguised faces," Visual Cognition, vol. 20, no. 2, pp. 143-169, 2012.
[26]	I. Pavlidis and P. Symosek, "The imaging issue in an automatic face/disguise detection system," in Proceedings IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications (Cat. No.PR00640), 2000, pp. 15-24.
[27]	T. I. Dhamecha, R. Singh, M. Vatsa, and A. Kumar, "Recognizing disguised faces: Human and machine evaluation," PloS one, vol. 9, no. 7, p. e99212, 2014.
[28]	R. Min, A. Hadid, and J. Dugelay, "Improving the recognition of faces occluded by facial accessories," in Face and Gesture 2011, 2011, pp. 442-447.
[29]	黃彪鈺, "偽裝人臉辨識之預處理系統," 淡江大學電機工程學系碩士班碩士論文, 2016.
[30]	Z. Jiang, Z. Lin, and L. S. Davis, "Label consistent K-SVD: Learning a discriminative dictionary for recognition," IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 11, pp. 2651-2664, 2013.
[31]	傅建鈞, "以卷積式神經網路為基礎之偽裝人臉辨識系統," 淡江大學電機工程學系碩士在職專班碩士論文, 2018.
[32]	Z. Yin, B. Wan, F. Yuan, X. Xia, and J. Shi, "A deep normalization and convolutional neural network for image smoke detection," IEEE Access, vol. 5, pp. 18429-18438, 2017.
[33]	Y. Tajima, K. Ito, T. Aoki, T. Hosoi, S. Nagashima, and K. Kobayashi, "Performance improvement of face recognition algorithms using occluded-region detection," in 2013 International Conference on Biometrics (ICB), 2013, pp. 1-8.
[34]	A. Singh, D. Patil, G. M. Reddy, and S. Omkar, "Disguised Face Identification (DFI) with Facial KeyPoints Using Spatial Fusion Convolutional Network," in 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), 2017, pp. 1648-1655.
[35]	T. Pfister, J. Charles, and A. Zisserman, "Flowing convnets for human pose estimation in videos," in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1913-1921.
[36]	A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in Neural Information Processing Systems, 2012, pp. 1097-1105.
[37]	T. Ahonen, A. Hadid, and M. Pietikäinen, "Face recognition with local binary patterns," in European Conference on Computer Vision, 2004: Springer, pp. 469-481.
[38]	C. Zhu, C. Bichot, and L. Chen, "Multi-scale Color Local Binary Patterns for Visual Object Classes Recognition," in 2010 20th International Conference on Pattern Recognition, 2010, pp. 3065-3068.
[39]	A. Sepas-Moghaddam, P. L. Correia, and F. Pereira, "Light field local binary patterns description for face recognition," in 2017 IEEE International Conference on Image Processing (ICIP), 2017, pp. 3815-3819.
[40]	F. Schroff, D. Kalenichenko, and J. Philbin, "FaceNet: A unified embedding for face recognition and clustering," in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 815-823.
[41]	C. Szegedy et al., "Going deeper with convolutions," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1-9.
[42]	E. P. Xing, M. I. Jordan, S. J. Russell, and A. Y. Ng, "Distance metric learning with application to clustering with side-information," in Advances in Neural Information Processing Systems, 2003, pp. 521-528.
[43]	L. v. d. Maaten and G. Hinton, "Visualizing data using t-SNE," Journal of Machine Learning Research, vol. 9, no. Nov, pp. 2579-2605, 2008.
[44]	Y. Guo, L. Zhang, Y. Hu, X. He, and J. Gao, "Ms-celeb-1m: A dataset and benchmark for large-scale face recognition," in European Conference on Computer Vision, 2016: Springer, pp. 87-102.
[45]	Microsoft. (2016). MSR Image Recognition Challenge (IRC)  [Online]. Available: https://www.microsoft.com/en-us/research/project/ms-celeb-1m-challenge-recognizing-one-million-celebrities-real-world/.
[46]	C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," ACM Trans. Intell. Syst. Technol., vol. 2, no. 3, pp. 1-27, 2011.
[47]	B. Gu and V. S. Sheng, "A Robust Regularization Path Algorithm for  -Support Vector Classification," IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 5, pp. 1241-1248, 2017.
[48]	Y. Tang, "Deep learning using linear support vector machines," arXiv preprint arXiv:1306.0239, 2013.
[49]	Y. Koshiba and S. Abe, "Comparison of L1 and L2 support vector machines," in Proceedings of the International Joint Conference on Neural Networks, 2003., 2003, vol. 3, pp. 2054-2059 vol.3.
[50]	A. M. Martinez and R. Benavente, "The AR Face Database," CVC Technical Report #24, June 1998.
[51]	G. Elsayed et al., "Adversarial examples that fool both computer vision and time-limited humans," in Advances in Neural Information Processing Systems, 2018, pp. 3910-3920.
[52]	J. Su, D. V. Vargas, and K. Sakurai, "One pixel attack for fooling deep neural networks," IEEE Transactions on Evolutionary Computation, 2019.
[53]	R. C. Gonzalez and R. E. Woods, Digital Image Processing. 2009, p. 34.