這項工作的主要目的是開發一種面部識別系統，該系統對於姿勢，光線遮擋和種族差異很大的數據庫非常有效。面部識別系統的問題解決了這項工作。為了開發可以在有限深度的大型數據庫上成功進行處理和訓練的捲積神經網絡，可以提取特徵並了解將數據饋送到網絡的複雜細節。為了檢測和提取面部的複雜特徵，我們選擇了新設計的多任務卷積神經網絡（MTCNN）算法，該算法成功地提高了速度，並證明與以前已知的傳統方法執行的任務相同。作為Haar Cascade和Voila jones算法。面部的總體任務由從頭開始發展的捲積神經網絡執行。在進行了實驗之後，為模型獲得的最佳參數在調整後顯示出相當大的改進，並且在將學習能力模型部署到測試集中的不可見圖像之後進行了測試。使用現有方法進行索引，該模型適用於面部識別，預識別和分類任務。

The main intention of this work is to develop a Face recognition system that is effective for the very large scale of the database with variation in pose, illumination occlusion, and ethnicity. The problem of the facial recognition system address in this work. To develop a convolution neural network that can successfully process and train on a large database with limited depth, can extract the features and learn the complex details of the data feed to the network has been performed. For the detection and extraction of the complex features from the face we opt for newly designed Multi-Task Convolution Neural Network (MTCNN) Algorithm which successfully increases the speed and proved to be an effective method for the same task that was executed by the previously known traditional methods such as Haar Cascade and Voila jones algorithms. The overall task of face recognition was performed over the convolution neural network developed from scratch. The optimal parameters obtained for the model after performing several experiments showed considerable improvement after tuning and the learning ability model was tested after deploying it for unseen images from the test set. Compared to the pre-existing methods, the proposed model is applicable for face recognition, precognition, and classification tasks.

Table of Contents

Chinese abstract	i
English abstract	ii
Acknowledgement	iv
Table of Contents	v
List of Figure	vii
List of Table	ix
CHAPTER 1: INTRODUCTION	1
1.1 Motivation	1
1.2 Problem statement	2
1.3 Related Work	3
1.3.1	Transfer Learning	4
1.4 Approach	4
1.5 Application	5
CHAPTER 2: LITERATURE REVIEW	7
2.1 Face recognition method	7
2.1.1	Geometry based method	8
2.1.2	Feature Based Method	8
2.1.3	Holistic based method	10
2.1.4	Hybrid methods	10
2.1.5	Deep learning method	11
2.2	Proposed Method	12
CHAPTER 3: THEORETICAL OVERVIW OF CONVOLUTIONAL NEURAL NETWORK	14
3.1	Neural Networks	14
3.1.1	Forward Propagation	16
3.1.2	Activation Function	17
3.1.3	Backpropagation	20
3.1.4	Loss Computation	21
3.1.5	Optimization	22
3.2	Convolution Neural Network	25
3.2.1	Layers	26
3.2.2	Issues and solutions	32
CHAPTER 4: FACE RECOGNITION METHODOLOY	35
4.1	Face Dataset	35
4.1.1	Oxford VGG Face Dataset	36
4.1.2	Data Analysis	37
4.2 Pre-processing	40
4.2.1	Face detection and extraction	41
4.2.2	Transformation of Image	45
4.2.3	Colour	45
4.3	Hardware	45
4.3.1	CUDA	46
4.4	Dataset split	47
4.5	Architecture of Neural Network	47
4.6	Visualisation	51
4.6.1	Input layer	51
4.6.2	Convolution layer	52
4.6.3	Activation layer (ReLU)	56
4.6.4	Pooling layer	60
4.6.5	Weights and parameter sharing	64
CHAPTER 5: EXPERIMENT AND RESULT	66
5.1	Hyper-Parameter Tuning	66
5.1.1	Effect of Numbers of layers	66
5.1.2	Effect of Learning rate	68
5.1.3	Effect of Batch Normalization	69
5.2	Test and recognition results	71
5.3	Comparison	74
CHAPTER 6: CONCLUSION AND FUTURE WORK	75
6.1	Conclusion	75
6.2	Future work	76
REFERENCES	77

 
List of Figure



Figure 2.1 Geometry Based face Recognition	8
Figure 2.2  Face Graph (Feature based approach)	9
Figure 2.3 Example of Deep Neural Network	12
Figure 3.1 An artificial neural network comprise of weight and bias is one of the basic network where neurons are the basic building block.	15
Figure 3.2 Forward propagation	16
Figure 3.3 Graphical representation Sigmoid Activation Function	18
Figure 3.4 Graphical representation of Tanh Activation functions	19
Figure 3.5 Graphical representations of ReLU	20
Figure 3.6 Back Propagation process in Neural Network	21
Figure 3.7 First and second order of momentum	24
Figure 3.8 CNN Architecture	26
Figure 3.9 working of Convolution layer when pass through the filter	28
Figure 3.10 Down sampling of input which is of (224x224x64) by the pooling layer and single slicing of depth with stride of 2	30
Figure 3.11 An illustrative diagram of fully connected layer depicts the prediction task	31
Figure 3.12 Graphical representation of an over-fitted graph	33
Figure 4.1 Few faces VGG face dataset	37
Figure 4.2 Non Uniformity of images	38
Figure 4.3 Image in different illumination condition	39
Figure 4.4 Illustration of Occlusion images present in dataset	40
Figure 4.5 Workflow for face recognition pre-process system	41
Figure 4.6 MTCNN P-Net network structure	42
Figure 4.7 R-Net Structure	43
Figure 4.8 O-Net structure	43
Figure 4.9 Illustration of Face detection (a) for image consists of two face single image, (b) Image of object over face	44
Figure 4.10 Architecture flow for the CNN model	48
Figure 4.11 CNN input layer visualization	52
Figure 4.12 Convolution matrix operation demonstration	53
Figure 4.13 Illustration of feature map by convolution layer shown through (a), (b) and (c)	56
Figure 4.14 Feature map for the activation layers illustrated in (a), (b) and (c) for overall CNN	59
Figure 4.15 Max pool operation in CNN for down sampling process	60
Figure 4.16 Feature map for the max pooling can be seen further in (a), (b) and (c)	63
Figure 4.17 Weight visualization of CNN, first layer	65
Figure 5.1 Accuracy graph after change in the number of layers in the CNN model.	67
Figure 5.2 Effect on accuracy by changing the learning rates: (a) learning rate 0.01,     (b) learning rate = 0.001 and (c) learning rate = 1e-5	69
Figure 5.3 Effect of Batch normalization on CNN results that average epoch for training set is 130 since the model achieved the threshold	71
Figure 5.4 Testing of data in (a) and (b) after completion of training phase	73


List of Table

Table 4.1 Face Dataset used during training	36
Table 4.2 System specification (a) For GPU, (b) For CPU	46
Table 4.3 Description of layer’s arrangement of model	49
Table 5.1  Effect of increasing in number of CNN layers	67
Table 5.2 Effect of changing in learning rate	68
Table 5.3 Effect of batch normalization	70
Table 5.4 Comparison of proposed and existing model	74

[1]	Shang-Hung Lin. “An introduction to face recognition technology”. In: Informing  Sci. Int. J. an Emerg. Transdiscipl. 3 (2000), pp. 1–7.
[2]	Wenyi Zhao et al. “Face recognition: A literature survey”. In: ACM computing surveys (CSUR) 35.4 (2003), pp. 399–458.
[3]	Charles A Nelson. “The development and neural bases of  face  recognition”.  In:  Infant and Child Development: An International Journal of Research and Practice 10.1-2 (2001), pp. 3–18.
[4]	Musab  Co¸skun  et  al.  “An  overview  of  popular  deep  learning  methods”.  In:  Eur J Tech 7.2 (2017), pp. 165–176.
[5]
Nojun Kwak. “Feature extraction for classification problems and its application to face recognition”. In: Pattern Recognition 41.5 (2008), pp. 1701–1717.

[6]	Shiguang Shan et al. “Illumination normalization for robust face recognition against varying lighting conditions”. In: 2003 IEEE International SOI Conference. Proceed- ings (Cat. No. 03CH37443). IEEE. 2003, pp. 157–164.
[7]	Alharbi Hazza et al. “An overview of feature-based methods for digital modulation classification”. In: 2013 1st International Conference on Communications, Signal Processing, and their Applications (ICCSPA). IEEE. 2013, pp. 1–6.
[8]	Sandeep Kumar, Sukhwinder Singh, and Jagdish Kumar. “A study on face recogni-  tion techniques with age and gender classification”. In: 2017 International Con- ference on Computing, Communication and Automation (ICCCA). IEEE. 2017, pp. 1001–1006.
[9]	Y Vijaya Lata et al. “Facial recognition using eigenfaces by PCA”. In: International Journal of Recent Trends in Engineering 1.1 (2009), p. 587.
[10]	Divyakant  T Meva  and CK Kumbharana. “Study of different trends and techniques     in face recognition”. In: International Journal of Computer Applications 96.8 (2014).
[11]	Trupti M Kodinariya. “Hybrid Approach to Face Recognition System using Prin- ciple component and Independent component with score based fusion process”. In: arXiv preprint arXiv:1401.0395 (2014).
[11]	Trupti M Kodinariya. “Hybrid Approach to Face Recognition System using Prin- ciple component and Independent component with score based fusion process”. In: arXiv preprint arXiv:1401.0395 (2014).
[12]	Eugene Charniak. Introduction to deep learning. The MIT Press, 2019.
[13]	Jiuxiang Gu et al. “Recent advances in convolutional neural networks”. In: Pattern Recognition 77 (2018), pp. 354–377.
[14]	Krishna Dharavath, Fazal Ahmed Talukdar, and Rabul Hussain Laskar. “Improving  face recognition rate with image preprocessing”. In: Indian Journal of Science and Technology 7.8 (2014), pp. 1170–1175.
[15]	Soufiane Hayou, Arnaud Doucet, and Judith Rousseau. “On the impact of the acti- vation function on deep neural networks training”. In: arXiv preprint arXiv:1902.06853 (2019).
[16]	Sang-Hoon Oh. “Improving the error backpropagation algorithm with a  modified  error function”. In: IEEE Transactions on Neural Networks 8.3 (1997), pp. 799–803.
[17]	Varun Ranganathan and S Natarajan. “A new backpropagation algorithm without gradient descent”. In: arXiv preprint arXiv:1802.00027 (2018).
[18]	Jingyi Xu et al. “A semantic loss function for deep learning with symbolic knowl- edge”. In: International Conference on Machine Learning. 2018, pp. 5502–5511.
[19]	Sandy Putra Siregar and Anjar Wanto. “Analysis of Artificial Neural Network Ac- curacy Using Backpropagation Algorithm In Predicting Process (Forecasting)”. In: IJISTECH (International Journal of Information System & Technology) 1.1 (2017), pp. 34–42.
[20]	Kenji Kawaguchi and Jiaoyang Huang. “Gradient descent finds global minima for generalizable deep neural networks of practical sizes”. In: 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE. 2019, pp. 92–99.
[21]	Saleh Albelwi and Ausif Mahmood. “A framework for designing the architectures of deep convolutional neural networks”. In: Entropy 19.6 (2017), p. 242.
[22]	Jie Wang and Zihao Li. “Research on face recognition based on CNN”. In: Proceed- ings of the IOP Conference. 2018, pp. 170–177.
[23]	Xiaoguang Chen et al. “Convolution neural  network  for  automatic  facial  expres-  sion recognition”. In: 2017 International conference on applied system innovation (ICASI). IEEE. 2017, pp. 814–817.
[24]	SH Shabbeer Basha et al. “Impact of Fully Connected Layers on Performance of Convolutional Neural Networks for Image Classification”. In: arXiv (2019), arXiv– 1902.
[25]	Relly Petrescu. “Face Recognition as a Biometric Application”. In: Journal of Mechatronics and Robotics 3 (Jan. 2019), pp. 237–257. DOI: 10.3844/jmrsp.2019. 237.257.
[26]	Omkar M Parkhi, Andrea Vedaldi, and Andrew Zisserman. “Deep face recognition”. In: (2015).
[27]	Dong Chen et al. “Joint cascade face detection and alignment”. In: European con- ference on computer vision. Springer. 2014, pp. 109–122.
[28]	Hongchang Ku and Wei Dong. “Face Recognition Based on MTCNN and Convolu- tional Neural Network”. Frontiers in Signal Processing, vol. 4, no. 1, 2020.
[29]	Keiron O’Shea and Ryan Nash. “An introduction to convolutional neural networks”.  In: arXiv preprint arXiv:1511.08458 (2015).
[30]	CUDA Nvidia. “Nvidia cuda c programming guide”. In: Nvidia Corporation 120.18 (2011), p. 8.
[31]	Matthew D Zeiler and Rob Fergus. “Visualizing and understanding convolutional networks”. In: European conference on computer vision. Springer. 2014, pp. 818– 833.
[32]	Tahmina Zebin et al. “Design and implementation of a convolutional neural network on an edge computing smartphone for human activity recognition”. In: IEEE Access 7 (2019), pp. 133509–133520.
[33]	Steve Lawrence et al. “Face recognition: A convolutional neural-network approach”.  In: IEEE transactions on neural networks 8.1 (1997), pp. 98–113.
[34]	Hidenori Ide and Takio Kurita. “Improvement of learning for CNN with ReLU activation by sparse regularization”. In: 2017 International Joint Conference on Neural Networks (IJCNN). IEEE. 2017, pp. 2684–2691.
[35]	Alessandro Giusti et al. “Fast image scanning with deep max-pooling convolutional neural networks”. In: 2013 IEEE International Conference on Image Processing. IEEE. 2013, pp. 4034–4038.
[36]	Jonathan Lorraine and David Duvenaud. “Stochastic hyperparameter optimization through hypernetworks”. In: arXiv preprint arXiv:1802.09419 (2018).
[37]	Leslie N Smith. “A disciplined approach to neural  network  hyper-parameters:  Part 1–learning rate, batch size, momentum, and weight decay”. In: arXiv preprint arXiv:1803.09820 (2018).
[38]	Keunwoo Choi, George Fazekas, and Mark Sandler. “Automatic tagging using deep convolutional neural networks”. In: arXiv preprint arXiv:1606.00298 (2016).
[39]	D Randall Wilson and Tony R Martinez. “The need for small learning rates on large problems”. In: IJCNN’01. International Joint Conference on Neural Networks. Proceedings (Cat. No. 01CH37222). Vol. 1. IEEE. 2001, pp. 115–119.
[40]	Yanghao Li et al. “Revisiting batch normalization for practical domain adaptation”. In: arXiv preprint arXiv:1603.04779 (2016).
[41]	Sergey Ioffe and Christian Szegedy. “Batch normalization: Accelerating deep net- work training by reducing internal covariate shift”. In: arXiv preprint arXiv:1502.03167 (2015).
[42]	Nils Bjorck et al. “Understanding batch normalization”. In: Advances in Neural Information Processing Systems. 2018, pp. 7694–7705.
[43]	Hayit Greenspan, Bram Van Ginneken, and Ronald M Summers. “Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique”. In: IEEE Transactions on Medical Imaging 35.5 (2016), pp. 1153–1159.