超解析是對一張給定的低解析影像生成一張相對解析度較高的影像。一張影像相對於多張高解析影像，因此超解析問題是一個欠定逆向問題（underdetermined inverse problem）。近年來，已經有許多的文獻針對超解析的欠定逆向問題提出各種方法，其中卷積類神經網路（Convolutional Neural Network, CNN）已被證實有不錯的結果[1-5]。C. Dong等人[1]提出用卷積神經網路有效解決這個問題。J. Kim等人[2]提出的更多層的卷積神經網路之超解析法（Very Deep Super Resolution, VDSR）可以改進C. Dong等人的方法。不過J. Kim等人用訓練殘差影像（residue image）取代C. Dong等人的直接訓練高解析影像。因吾人考慮到J. Kim等人所提的VDSR的改進原因也許還包含以訓練殘差影像（residue image）取代訓練高解析影像。本論文將研究以訓練殘差影像（residue image）取代訓練高解析影像來改善C. Dong等人的方法，並比較VDSR以訓練殘差影像和訓練高解析影像的結果，與分析其執行時間。C. Dong等人所提的卷積神經網路沒有做邊框填充（padding），因此最後得的高解析影像的尺寸會減小，本論文也會提出一些方法來解決這個問題，並且對各種相關問題做詳細分析且提出一個更好的超解析方法。

Super resolution is a technique to enhance the resolution of video or images. Super resolution is an underdetermined inverse problem, since an image corresponds to many possible images with higher resolution. Recently, there have been many methods of super resolution proposed in literatures, in which convolutional neural networks have been confirmed to have good results [1-5]. C. Dong et al. [1] proposed a convolutional neural network structure to effectively solve the super resolution problem. J. Kim et al. [2] proposed a much deeper convolutional neural network (Very Deep Super Resolution, VDSR) to improve C. Dong et al.’s method. However, unlike VDSR proposed by J. Kim et al. which trained residue images, SRCNN proposed by C. Dong et al. directly trained high-resolution images. Consequently, we surmise that the improvement of VDSR is due to not only the depth of the neural network structure but also the training of residue images. This paper studies and compares the performance of training high-resolution images and training residue images associated with the two neural network structures, SRCNN and VDSR. Some deep CNNs proceed zero padding which pads the input to each convolutional layer with zeros around the border so that the feature maps remain the same size. SRCNN proposed by C. Dong et al. does not carry out padding, so the size of the resulting high-resolution images is smaller than expected. The study also proposes two revised versions of SRCNN that remain the size the same as the input image.

目錄
第一章	緒論	1
第二章	相關文獻回顧	3
第三章	相關理論基礎	5
3.1	類神經網路	5
3.2	卷積神經網路	6
3.2.1	卷積層	7
3.3	激活函數（activation function）	10
3.4	池化層（pooling layers）	13
第四章	研究方法	14
4.1	超解析卷積神經網路	14
4.1.1	區塊擷取與表示（patch extraction and representation）	16
4.1.2	非線性對映（non-linear mapping）	17
4.1.3	殘差影像重建（residue reconstruction）	17
4.2	卷積神經網路與稀疏編碼的對應關係	18
4.3	卷積神經網路的訓練（training）	20
4.4	卷積神經網路的測試	20
4.5	邊框填充（padding）	21
4.6	無邊框填充結果結合有邊框填充結果的邊框（RLSRCNN2）	24
第五章	實驗結果	25
5.1	訓練樣本的建立	25
5.2	實驗結果比較（comparisons）	26
5.3	彩色影像的超解析重建	43
第六章	結論與未來研究	45
參考文獻	47
附錄：英文論文	51


 
圖目錄
圖 1單一神經元的計算	5
圖2 倒傳遞網路（BPN）架構	6
圖3 LeNet-5架構圖	7
圖4 濾波器移動步距為1的卷積特徵圖運算範例	8
圖5 濾波器移動步距為2的卷積特徵圖運算範例	9
圖6 多張特徵圖的卷積運算示意圖	10
圖1 雙彎曲函數圖形	11
圖1  雙曲正切函數圖形	11
圖1 線性整流函數圖形	12
圖10 池化層示意圖	13
圖11 超解析之卷積神經網路架構圖	15
圖12 區塊擷取示意圖	16
圖13 影像重建示意圖	18
圖14 卷積神經網路與稀疏編碼的對應關係	20
圖15 卷積神經網路的測試架構圖	21
圖16 無邊框填充的卷積運算示意圖（f = 3）	22
圖17 邊框填充的卷積運算示意圖（f = 3）	23
圖18 無邊框填充結果的邊框結合有邊框填充結果的示意圖	24
圖19 訓練資料集建立的示意圖	26
圖20 Set14影像（Lena）的影像之無邊框填充超解析重建結果與局部放大圖	29
圖21 Set14影像（monarch）的影像之無邊框填充超解析重建結果與局部放大圖	30
圖22 Set14影像（ppt3）的影像之無邊框填充超解析重建結果與局部放大圖	30
圖23 FERET影像集的影像之無邊框填充超解析重建結果與局部放大圖	31
圖24 FERET影像集的影像之無邊框填充超解析重建結果與局部放大圖	32
圖25 Barbara測試結果與局部放大圖	33
圖26 Barbara測試結果與局部放大圖	34
圖27 Set14影像（Lena）有邊框填充的超解析重建之結果與局部放大圖	39
圖28 Set14影像（monarch）有邊框填充的影像超解析重建之結果與局部放大圖	40
圖29 Set14影像（ppt3）有邊框填充的影像超解析重建之結果與局部放大圖	41
圖30 FERET影像集有邊框填充的影像超解析重建之結果與局部放大圖	42
圖31 FERET影像集有邊框填充的影像超解析重建之結果與局部放大圖	43
圖32 彩色影像（Lena）的影像超解析重建之結果：	44

 
表目錄
表 1 SRCNN 和RLSRCNN無邊框填充架構之重建影像的PSNR比較	27
表2 SRCNN 和RLSRCNN無邊框填充架構之重建影像的PSNR比較	28
表3 SRCNNP、VDSR、RLSRCNNP和RLSRCNN2之重建影像的PSNR比較	35
表4 SRCNN 和RLSRCNN之重建影像的PSNR比較	37
表5 影像超解析PSNR與訓練資料集之短訓練時間比較	37
表6 影像超解析PSNR與訓練資料集之長訓練時間比較	38

[1]	C. Dong, C. C. Loy, K. He, and X. Tang, “Image super resolution using deep convolutional networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 295-307, June 2015.
[2]	J. Kim, J. K. Lee, and K. M. Lee, “Accurate image super resolution using very deep convolutional networks,” Computer Vision and Pattern Recognition, arXiv: 1511.04587v2 (cs.CV), Nov. 2015.
[3]	Y. Tai, J. Yang, and X. Liu, “Image super resolution via deep recursive residual network,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 2790-2798, July 2017.
[4]	Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, “Residual dense network for image super resolution,” Computer Vision and Pattern Recognition, arXiv:1802.08797v2 (cs.CV), Feb. 2018.
[5]	B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, “Enhanced deep residual networks for single image super resolution,” Computer Vision and Pattern Recognition, arXiv:1707.02921v1 (cs.CV), Jul. 2017.
[6]	K. Fukushima, “Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position,” Biological Cybernetics, vol. 36, pp. 193-202, 1980.
[7]	Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition.” in Intelligent signal processing, pp. 306-351, IEEE Press, 2001.
[8]	W. Freeman, T. Jones, and E. Pasztor, “Example-based super resolution,” IEEE Computer graphics and Applications, 2002, 22(2): 55-65.
[9]	K. S. Ni and T. Q. Nguyen, “Image super resolution using support vector regression,” IEEE Transactions on Image Processing, vol. 16, pp. 1596-1610, June 2007.
[10]	J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super resolution via sparse representation,” IEEE Transactions on Image Processing, vol. 19, pp. 2861-2873, 2010.
[11]	H. Chang, D. Y. Yeung, and Y. Xiong, “Super resolution through neighbor embedding,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. I-275-I-282, Washington, DC, USA, June-July 2004.
[12]	R. Zeyde, M. Elad, and M. Protter, “On single image scale-up using sparse-representations,” International Conference on Curves and Surfaces, pp. 711-730, 2012.
[13]	P. J. Phillips, H. Moon, S. A. Rizvi, and P. J. Rauss, “The FERET evaluation methodology for face-recognition algorithms,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, pp. 1090 - 1104, Oct. 2000.
[14]	R. Timofte, V. De, and L. V. Gool, “Anchored neighborhood regression for fast example-based super resolution,” IEEE International Conference on Computer Vision, pp. 1920 - 1927, Dec. 2013.
[15]	K. Zeng, J. Yu, R. Wang, C. Li, and D. Tao, “Coupled deep autoencoder for single image super resolution,” IEEE Transactions on Cybernetics, vol. 47, pp. 27-37, Nov. 2015.
[16]	X. Gao, K. Zhang, D. Tao, and X. Li, “Image super resolution with sparse neighbor embedding,” IEEE Transactions on Image Processing, vol. 21, pp. 3194-3205, March 2012.
[17]	T. C. Ho and B. Zeng, “Super resolution images by support vector regression on edge pixels,” IEEE International Symposium on Intelligent Signal Processing and Communication Systems, pp. 674-677, Nov-Dec. 2007.
[18]	A. B. Rao and J. V. Rao, “Super resolution of quality images through sparse representation,” ICT and Critical Infrastructure: Proceedings of the 48th Annual Convention of Computer Society of India, vol. II, pp. 49-56, 2014.
[19]	K. I. Kim and Y. Kwon, “Example-based learning for single-image super resolution,” in: Rigoll G. (eds) Pattern Recognition. DAGM 2008. Lecture Notes in Computer Science, vol. 5096, pp. 456-465, 2008, Springer, Berlin, Heidelberg.
[20]	M. Sharma, S. Chaudhury, and B. Lall, “Deep learning based frameworks for image super resolution and noise-resilient,” IEEE International Joint Conference on Neural Networks, pp. 744-751, May 2017.
[21]	J. Yang, J. Wright, T. Huang, Y. Ma, “Image super resolution as sparse representation of raw image patches,” IEEE Conference on Computer Vision and Pattern Recognition, vol. 19, pp. 1-8, 2008.
[22]	J. Yang, J. Wright, T.S. Huang, Y. Ma, “Image super resolution via sparse representation,” IEEE Transactions on Image Processing, vol. 19, pp. 2861-2873, 2010.
[23]	W. T. Freeman, E. C. Pasztor, and O. T. Carmichael, “Learning low-level vision,” IEEE International Conference on Computer Vision, vol. 2, pp. 25-47, 1999.
[24]	J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super resolution via sparse representation,” IEEE Transactions on Image Processing, vol. 19, pp. 2861-2873, 2010.