缺陷識別在現代工業中已是不可或缺的一部分，如何以機器取代人力是此領域的重要課題。本文提出一種基於深度學習方法的表面缺陷識別系統，該系統透過設計的高速圖像擷取平台捕捉RGB圖像信息，並透過改進的卷積神經網絡(CNN)架構來識別表面缺陷。基於SurfNet架構得到的靈感，系統中提出的CNN模型命名為SurfNetv2，它包括一個特徵提取模塊和一個缺陷識別模塊。我們將提出的網路模型應用於矽酸鈣板(CSB)表面缺陷識別任務上，並通過端到端有監督式學習方法來訓練模型。我們定義矽酸鈣板表面上的缺陷類別分為四類，即崩潰、髒污、不均勻和正常。當模型處理解析度為128×128的圖像時，實現了約199.38 fps的實時計算速度，並分別在私有矽酸鈣板表面缺陷數據集、公共NEU數據集和公共DeepPCB實驗中，實現了約99.90％、99.75％和91.15%的準確率。實驗結果證明提出的系統可以做到矽酸鈣板表面缺陷實時識別。

Defect detection has become an indispensable part of modern industry. How to replace manpower with machines is an important issue in this field. This thesis proposes a surface defect recognition system based on deep learning method. The system captures RGB image information through a designed high-speed image acquisition platform, and recognizes surface defects through an improved convolutional neural network (CNN) architecture. Inspired by the SurfNet architecture, the CNN model proposed in the thesis is named SurfNetv2, which includes a feature extraction module and a defect recognition module. We applied the proposed network model to the surface defect recognition task of calcium silicate board (CSB), and trained the model through an end-to-end supervised learning method. The recognized surface defect types of CSB are divided into four categories, namely crash, dirty, uneven and normal. The model achieves a real-time calculation speed of about 199.38 fps when processing images with a resolution of 128×128. It also achieves the accuracy rates about 99.90%, 99.75% and 91.15% in the experiments of the private CSB surface defect dataset, the public NEU dataset and the public DeepPCB dataset, respectively. Experimental results show that the system can realize real-time identification of CSB surface defects.

目錄
中文摘要........................I
英文摘要........................II
目錄............................III
圖目錄..........................VI
表目錄..........................VIII
第一章 序論.....................1
1.1  研究背景...................1
1.2  研究動機與目的.............3
1.3  系統架構...................4
1.3.1	硬體....................5
1.3.2	軟體流程................6
1.4  論文架構...................7
1.5  論文貢獻...................8
第二章 相關研究.................10
2.1  缺陷識別...................10
2.1.1	機器視覺方法............10
2.1.2	頻譜分析................12
2.1.3	色彩空間轉換............13
2.2  深度學習方法...............14
2.2.1	語意分割................14
2.2.2	偵測....................15
2.2.3	分類....................16
2.2.4	現有方法改進............16
2.3  卷積神經網路...............17
2.4  文獻總結...................18
第三章 提出的表面缺陷識別模型...20
3.1  特徵擷取層.................20
3.1.1	卷積模塊................20
3.1.2	殘差模塊................22
3.1.3	特徵擷取模塊............23
3.2  缺陷識別層.................24
3.2.1	全局平均池化層..........25
3.2.2	輸出層..................25
3.3  表面缺陷識別網路模型.......26
3.3.1 	SurfNetv2...............27
3.3.2 	額外的架構設計..........28
3.4  訓練.......................30
第四章 實驗結果與分析...........32
4.1  軟硬體介紹.................32
4.2  數據集.....................33
4.2.1	數據收集................33
4.2.2	數據集創建..............34
4.2.3	訓練數據集..............35
4.3  實驗模型與性能驗證.........38
4.4  實驗分析...................41
4.4.1	私有矽酸鈣板缺陷數據集..41
4.4.2	公共NEU缺陷數據.........44
4.4.3	DeepPCB.................47
4.4.4	小結....................48
4.5  SurfNetv2模塊數量驗證......49
第五章 結論與未來展望...........50
參考文獻........................52

圖目錄
圖1.1 機器視覺的構成要素................................2
圖1.2 矽酸鈣板表面缺陷識別透過人工檢查方式..............3
圖1.3 高速圖像擷取平台..................................5
圖1.4 提出的表面缺陷識別系統架構........................6
圖3.1 SurfNet基礎卷積模塊...............................21
圖3.2 SurfNetv2基礎卷積模塊.............................21
圖3.3 SurfNet殘差模塊...................................22
圖3.4 SurfNetv2殘差模塊.................................22
圖3.5 SurfNet特徵擷取模塊...............................23
圖3.6 SurfNetv2特徵擷取模塊.............................23
圖3.7 多類別表面缺陷識別模型............................24
圖3.8 SurfNet網路模型架構...............................27
圖3.9 SurfNetv2網路模型架構.............................28
圖4.1 矽酸鈣板表面缺陷樣本收集..........................33
圖4.2 私有矽酸鈣板表面缺陷數據集的四個類別；
      從左上到右下分別為: 正常、不均勻、髒污和碰撞......34
圖4.3 應用多個仿射變換來增強數據........................35
圖4.4 公共NEU數據集類別.................................37
圖4.5 公共DeepPCB數據集類別.............................38

表目錄
表 3.1 SurfNetv2與額外的模型架構........29
表 4.1 軟硬體規格.......................32
表 4.2 私有矽酸鈣板表面缺陷數據集.......36
表 4.3 公共NEU數據集....................36
表 4.4 公共DeepPCB數據集................37
表 4.5 實驗模型架構.....................39
表 4.6 實驗模型訓練epoch設定............40
表 4.7 私有矽酸鈣板缺陷數據集的實驗結果.43
表 4.8 公共NEU數據集的實驗結果..........46
表 4.9 公共DeepPCB數據集的實驗結果......48
表 4.10 不同SurfNetv2模塊下的性能驗證...49

[1]	S. Arikan, K. Varanasi and D. Stricker, “Surface defect classification in real-time using convolutional neural networks,” April 2019, arXiv:1904.04671.
[2]	K. Song and Y. Yan, “A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects,” Appl. Surf. Sci., vol. 285, pp. 858-864, Nov. 2013.
[3]	H.-M. Ma, G.-D. Su, J.-Y. Wang and Z. Ni, “A glass bottle defect detection system without touching,” in IEEE Int. Conf. on Mach. Learn. and Cybern. (ICMLC), Beijing, China, 2002, doi: 10.1109/ICMLC.2002.1174411.
[4]	L. Chen, Y. Liang and K. Wang, “Inspection of rail surface defect based on machine vision system,” in IEEE The 2nd Int. Asia Conf. on Inform. in Control, Automat. and Robot., Wuhan, China, April 2010, doi:  10.1109/CAR.2010.5456793.
[5]	S. Fu and Z. Jiang, “Research on image-based detection and recognition technologies for cracks on rail surface,” in IEEE Int. Conf. on Robots and Intell. System, Haikou, China, June 2019, doi: 10.1109/ICRIS.2019.00033.
[6]	F. Gao, Z. Li, G. Xiao, X. Yuan and Z. Han, “An online inspection system of surface defects for copper strip based on computer vision,” in IEEE 5th Int. Congr. on Image and Signal Process., Chongqing, China, Oct. 2012, doi: 10.1109/CISP.2012.6469775.
[7]	M. Chu, A. Wang, R. Gong and M. Sha, “Strip steel surface defect recognition based on novel feature extraction and enhanced least squares twin support vector machine,” J-STAGE ISIJ Int., vol. 54, pp. 1638-1645, 2014.
[8]	X. Liu, F. Xue and L. Teng, “Surface defect detection based on gradient LBP,” in IEEE 3rd Int. Conf. on Image, Vision and Comput. (ICIVC), Chongqing, China, June 2018, doi: 10.1109/ICIVC.2018.8492798.
[9]	J. Duan, L. Luo, J. Li, X. Gao and W. Zhang, “Measuring of train wheel surface defect based on linear CCD imaging,” in 18th Int. Wheelset Congr. (IWC), Chengdu, China, Nov. 2016, doi: 10.1109/IWC.2016.8068369.
[10]	A. Zhou, H. Zheng, M. Li and W. Shao, “Defect inspection algorithm of metal surface based on machine vision,” in IEEE 12th Int. Conf. on             Technol. and Mechatronics Automat. (ICMTMA), Phuket, Thailand, Feb. 2020, doi: 10.1109/ICMTMA50254.2020.00017.
[11]	G. Wu, H. Zhang, X. Sun, J. Xu and K. Xu, “A bran-new feature extraction method and its application to surface defect recognition of hot rolled strips,” in IEEE Int. Conf. on Automat. and Logistics, Jinan, China, Aug. 2007.
[12]	Z. Yu, X. Li, H. Yu, D. Xie, A. Liu and H. Lv, “Research on surface defect inspection for small magnetic rings,” in IEEE Int. Conf. on Mechatronics and Automat., Changchun, China, Aug. 2009, doi: 10.1109/ICMA.2009.5246333.
[13]	J. Choi and C. Kim, “Unsupervised detection of surface defects: A two-step approach,” in 19th IEEE Int. Conf. on Image Process., Orlando, FL, USA, Sept. 2012, doi: 10.1109/ICIP.2012.6467040.
[14]	J. Ma, Y. Wang, C. Shi and C. Lu, “Fast surface defect detection using improved gabor filters,” in 25th IEEE Int. Conf. on Image Process. (ICIP), Athens, Greece, Oct. 2018, doi: 10.1109/ICIP.2018.8451351.
[15]	D.-M. Tsai and C.-K. Huang, “Defect detection in electronic surfaces using template-based Fourier image reconstruction,” IEEE Trans. on Compon., Packag. and Manuf. Technol., vol. 9, pp. 163-172, Jan. 2019, doi: 10.1109/TCPMT.2018.2873744.
[16]	D. Tsai, C. Lin, and K. Huang, “Defect detection in colored texture surfaces using Gabor filters,” The Imag. Sci. J., vol. 53, pp. 27-37, 2005.
[17]	H.-Q. Bong, Q.-B. Truong, H.-C. Nguyen and M.-T. Nguyen, “Vision-based inspection system for leather surface defect detection and classification,” in 5th NAFOSTED Conf. on Inf. and Comput. Sci. (NICS), Ho Chi Minh City, Vietnam, Nov. 2018, doi: 10.1109/NICS.2018.8606836.
[18]	Q. Chang, Y. Zhang and Z. Sun, “Research on surface defect detection algorithm of ice-cream bars based on clustering,” in IEEE 3rd Inf. Technol., Netw., Electron. and Automat. Control Conf. (ITNEC), Chengdu, China, March 2019, doi: 10.1109/ITNEC.2019.8729001.
[19]	I. Ivasenko and V. Chervatyuk, “Detection of rust defects of protective coatings based on HSV color model,” in IEEE 2nd Ukraine Conf. on Elect. and Comput. Eng. (UKRCON), Lviv, Ukraine, Ukraine, July 2019, doi: 10.1109/UKRCON.2019.8879804.
[20]	X. Tao, D. Zhang, W. Ma, X. Liu and D. Xu, “Automatic metallic surface defect detection and recognition with convolutional neural networks,” MDPI Appl. Sci., vol. 8, Sep. 2018, doi: 10.3390/app8091575.
[21]	H. Sison, P. Konghuayrob and S. Kaitwanidvilai, “A convolutional neural network for segmentation of background texture and defect on copper clad lamination surface,” in Int. Conf. on Eng., Appl. Sci., and Technol. (ICEAST), Phuket, Thailand, July, 2018, doi: 10.1109/ICEAST.2018.8434483.
[22]	L. Qiu, X. Wu and Z. Yu, “A high-efficiency fully convolutional networks for pixel-wise surface defect detection,” in IEEE Access, vol. 7, pp. 15884 – 15893, January 2019, doi: 10.1109/ACCESS.2019.2894420.
[23]	Y.-F. Chen, F.-S. Yang, E. Su, and C.-C. Ho, “Automatic defect detection system based on deep convolutional neural networks,” in Int. Conf. on Eng., Sci., and Ind. Appl. (ICESI), Tokyo, Japan, Aug. 2019, doi: 10.1109/ICESI.2019.8863029.
[24]	K. Han, M. Sun, X. Zhou, G. Zhang, H. Dang and Z. Liu, “A new method in wheel hub surface defect detection: Object detection algorithm based on deep learning,” in Int. Conf. on Adv. Mechatronic Syst. (ICAMechS), Xiamen, China, Dec. 2017, doi: 10.1109/ICAMechS.2017.8316494.
[25]	S. Ren, K. He, R. Girshick and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” in Neural Inf. Process. Syst., Montreal, Canada, 2015.
[26]	S. Yanan, Z. Hui, L. Li and Z. Hang, “Rail surface defect detection method based on YOLOv3 deep learning networks,” in Chin. Automat. Congr. (CAC), Xi'an, China, Dec. 2018, doi: 10.1109/CAC.2018.8623082.
[27]	R. Wei and Y. Bi, “Research on recognition technology of aluminum profile surface defects based on deep learning,” MDPI Mater., vol. 12, May 2019, doi: 10.3390/ma12101681.
[28]	Y. He, K. Song, Q. Meng and Y. Yan, “An end-to-end steel surface defect detection approach via fusing multiple hierarchical features,” in IEEE Trans. on Instrum. and Meas., vol. 69, pp. 1493 – 1504, May 2019, doi: 10.1109/TIM.2019.2915404.
[29]	L. M. Azizah, S. F. Umayah, S. Riyadi, C. Damarjati and N. A. Utama, “Deep learning implementation using convolutional neural network in mangosteen surface defect detection,” in 7th IEEE Int. Conf. on Control Syst., Comput. and Eng. (ICCSCE), Penang, Malaysia, Nov. 2017, doi: 10.1109/ICCSCE.2017.8284412.
[30]	S. Cheon, H. Lee, C. O. Kim, and S. H. Lee, “Convolutional neural network for wafer surface defect classification and the detection of unknown defect class,” in IEEE Trans. on Semicond. Manuf., vol. 32, pp. 163 – 170, March 2019, doi: 10.1109/TSM.2019.2902657.
[31]	M. S. Kim, T. Park and P. Park, “Classification of steel surface defect using convolutional neural network with few images,” in 12th Asian Control Conf. (ASCC), Kitakyushu-shi, Japan, June 2019.
[32]	J. Bromley, I. Guyon and Y. LeCun, “Signature verification using a” siamese” time delay neural network,” in Advances in neural inf. Process. Syst, 1994, pp. 737-744.
[33]	J. Ren, R. Ren, M. Green and X. Huang, “Defect detection from X-Ray images using a three-stage deep learning algorithm,” in IEEE Can. Conf. of Elect. and Comput. Eng. (CCECE), Edmonton, AB, Canada, May 2019, doi: 10.1109/CCECE.2019.8861944.
[34]	Y.-G. Kim, D.-U. Lim, J.-H. Ryu and T.-H. Park, “SMD defect classification by convolution neural network and PCB image transform,” in IEEE 3rd Int. Conf. on Comput., Commun. and Secur. (ICCCS), Kathmandu, Nepal, Oct. 2018, doi: 10.1109/CCCS.2018.8586818.
[35]	Y. Gao, L. Gao, X. Li and X. V. Wang, “A multi-level information fusion-based deep leaning method for vision-based defect recognition,” in IEEE Trans. on Instrum. and Meas., Oct. 2019, pp. 1-1, doi: 10.1109/TIM.2019.2947800.
[36]	K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Int. Conf. on Learn. Representations, San Diego, USA, Sep. 2015.
[37]	P. Isola, J.-Y. Zhu, T. Zhou and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in IEEE Conf. on Comput. Vision and Pattern Recognit., Honolulu, Hawaii, USA, July 2017.
[38]	Y.-W. Lu, K.-L. Liu and C.-Y. Hsu, “Conditional generative adversarial network for defect classification with class imbalance,” in IEEE Int. Conf. on Smart Manuf., Ind. & Logistics Eng. (SMILE), Hangzhou, China, April 2019, doi:  10.1109/SMILE45626.2019.8965320.
[39]	G. Huang, Z. Liu and L. van der Maaten, “Densely connected convolutional networks,” in IEEE Conf. on Comput. Vision and Pattern Recognit. (CVPR), Honolulu, HI, USA, July 2017, doi: 10.1109/CVPR.2017.243.
[40]	S. Guan, M. Lei and H. Lu, “A Steel Surface Defect Recognition Algorithm Based on Improved Deep Learning Network Model Using Feature Visualization and Quality Evaluation,” in IEEE Access, vol. 8, pp. 49885 – 49895, March. 2020, doi: 10.1109/ACCESS.2020.2979755.
[41]	S. Ioffe and C. Szegedy, “Batch normalization: accelerating deep network training by reducing internal covariate shift,” in Proc. of the 32nd Int. Conf. on Int. Conf. on Mach. Learn., vol. 37, pp. 448-456, July 2015.
[42]	G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever and R. R. Salakhutdinov, “Improving neural networks by preventing co-adaptation of feature detectors,” June 2012, arXiv:1207.0580.
[43]	K. He, X. Zhang, S. Ren and J. Sun, “Deep residual learning for image recognition,” in IEEE Conf. on Comput. Vision and Pattern Recognit., Las Vegas, NV, USA, 2016, doi: 10.1109/CVPR.2016.90.
[44]	K. He, X. Zhang, S. Ren and J. Sun, “Identity mappings in deep residual networks,” July 2016, arXiv:1603.05027.
[45]	K. He, X. Zhang, S. Ren and J. Sun, “Delving deep into rectifiers: surpassing human-level performance on ImageNet classification,” in IEEE Int. Conf. on Comput. Vision, Santiago, Chile, Dec. 2015, pp. 1026-1034, doi: 10.1109/ICCV.2015.123.
[46]	V. Nair, G. E. Hinton, “Rectified linear units improve restricted Boltzmann machines,” in Proc. of the 27th Int. Conf. on Mach. Learn., Haifa, Israel, 2010.
[47]	M. Lin, Q. Chen and S. Yan, “Network in network,” in Int. Conf. on Learn. Representations, Banff, Canada, Dec. 2014.
[48]	S. Tang, F. He, X. Huang and J. Yang, “Online PCB defect detector on a new PCB defect dataset,” Feb. 2019, arXiv:1902.06197.
[49]	M. Sandler, A. Howard, M. Zhu, A. Zhmoginov and L.-C. Chen, “MobileNetV2: Inverted residuals and linear bottlenecks,” in IEEE/CVF Conf. on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, June 2018, doi: 10.1109/CVPR.2018.00474.
[50]	B. Zoph, V. Vasudevan, J. Shlens and Q. V. Le, “Learning transferable architectures for scalable image recognition,” in IEEE/CVF Conf. on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, June 2018, doi: 10.1109/CVPR.2018.00907