一般超解析方法多以固定的退化方法對高解析樣本先產生低解析樣本，再以此成對的高、低解析影像訓練模型，但是如此訓練的模型對於真實影像的超解析結果卻往往成效不彰。這是因為模型沒有考慮低解析影像可能有的雜訊干擾，然而真實的低解析影像卻又沒有相對應的高解析影像可供訓練。基於這樣的困境，本篇論文提出一個以非監督式學習為基礎的雙向循環模型，能夠對真實影像達到超解析的目的。和過去大量使用成對資料集進行訓練的監督式學習方法不同，我們使用了非成對樣本進行訓練。整體架構由兩個子網路所組成，低至高Low-to-High網路和高至低High-to-Low網路。訓練High-to-Low網路時我們以高解析影像加入隨機雜訊當成輸入，讓生成的低解析影像具有多樣性以模擬真實影像可能遇到的各式雜訊。這些生成的低解析樣本則是用以訓練Low-to-High網路以達到超解析的目的。在子網路中我們主要採用生成對抗網路 (GAN)，GAN中的判別器 (discriminator)可以使影像在高解析與低解析互相轉換時，讓生成的影像分佈更接近真實的目標資料，而在兩個子網路結合時搭配循環一致性(cycle consistency)的機制，讓影像在高解析度空間與低解析度空間中進行轉換時能保持原本的內容。我們的方法雖然對於固定退化的測試集並沒有大幅優於監督式學習的方法，但對於非固定退化的測試樣本和真實不具有Ground Truth的低解析樣本展現了良好的適應能力。對於實際應用言，我們所提出的系統對於影像拍攝時可能受到的雜訊具有強健性仍然可以達到超解析目的。

In previous existing super resolution (SR) methods, a designated degradation operation is used to generate low-resolution (LR) images from high-resolution (HR) ones. Then, the paired images, LR images and their HR counterparts, are used for training SR models. However, these trained models often lead to unsatisfactory results when applying on real-world images with unknown degradation. To solve this problem, based on unsupervised learning, we propose a bi-directed SR network for real-world images corrupted by unknown degradation. The network comprises bi subnetworks: High-to-Low module and Low-to-High module. The High-to-Low module takes HR images and generates LR images. To avoid having LR images generated through a designated degradation, a random noise vector is appended as a part of input to simulate real-world images. The Low-to-High module takes these generated LR images as training samples for the SR task. We use the Generative Adversarial Network (GAN) in both subnetworks to generate  visually realistic HR and LR images. And we use the idea of cycle consistency to preserve the content of the images when the images are converted between HR images and LR images in the bi-directed model. Our model shows good adaptability for unknown degraded public test samples as well as those real-world low-resolution samples which do not have Ground Truth.

目錄
第一章 緒論 1
1.1 超解析(Super-Resolution)簡介 1
1.2 研究動機與目的  2
第二章 文獻回顧  4
2.1 監督式學習方法(Supervised learning) 4
2.2 非監督式學習方法(Unsupervised learning) 5
2.3 間接式監督(Indirect Supervised) 6
第三章 研究方法 7
3.1 問題定義 7
3.2 網路架構  7
3.3 High-to-Low Generator 9
3.4 Low-to-High generator 11
3.5 Discriminator 12
3.6 Cycle consistency 13
第四章 實驗 14
4.1 實驗資料建立 14
4.2 實驗細節 14
4.3 衡量方法  15
4.4 實驗結果  16
4.5 實驗結果視覺化比較  17
4.6 與其他方法比較  23
4.7 影像後處理  24
第五章 結論與未來展望 27
5.1 結論 27
5.2 未來展望  27
參考文獻 28 
附錄:英文論文 31

圖目錄
圖 1. 相同影像在不同解析度下的視覺變化 1
圖 2. 放大影像時使用超解析技術的範例，由左至右分別為原圖、直接放大的大圖及運用超解析放大的大圖 2
圖 3. SRCNN 的架構，參考自[1] 5
圖 4. Cycle-in-cycle gan 架構圖，來自[9] 5
圖 5. 超解析任務中間接式監督學習與其他傳統方法的比較，(a)監督式學習方法，(b)非監督式學習方法，(c)間接是學習方法，來源自[17] 6
圖 6. 雙向超解析網路訓練時整體架構 8
圖 7. Cycle consistency 示意圖，圖片來自[10] 9
圖 8. High-to-Low generator 架構圖 9
圖 9. Low-to-High generator 結構 11
圖 10. Discriminator架構	 13
圖 11. 由低解析為輸入的cycle consistency訓練 13
圖 12. 測試流程，測試時只使用Low-to-High generator模組 15
圖 13. 影像monarch的視覺比較，(a)低解析的輸入圖，其4X 的超解析結果分別使用 (b)使用bicubic插值法，(c)本文方法。 18
圖 14. 超解析Monarch後結果的局部放大圖，左圖為bicubic插值法，右圖為我們提出的方法 19
圖 15. NTIRE 18 Track4的測試結果比較，(a)低解析的輸入圖，其4X 的超解析結果分別使用 (b)bicubic插值法，(c)本文方法。 20
圖 16. 在圖15中結果圖之局部放大比較，左圖為bicubic插值法，右圖為本文方法 20
圖 17. NTIRE 2020 Track2 資料集中部分影像進行超解析後結果，左方為原圖，右方為超解析後結果。 22
圖 18. 模型改變原圖明亮度的範例，左圖為原圖，右圖為超解析成果 25
圖 19. Histogram matching範例	25

表目錄
表 1. NTIRE競賽資料集規格比較 17
表 2. 實驗數據對照表，數字為PSNR↑ / SSIM↑ 17
表 3. 我們的結果與其他非監督式學習方法之比較，表中數字為PSNR/SSIM 24
表 4. 我們的結果與其他非監督式學習方法之比較，表中數字為PSNR/SSIM以bicubic方法為基準之增長比例 24
表 5.. 模型加入Histogram matching (HM)後處理與原始結果比較 26
表 6. 模型加入HM後與其他非監督式學習方法在NTIRE 2018資料集上比較，數值為PSNR/SSIM以bicubic方法為基準之增長比例 26

[1] Dong, C., Loy, C., He, K., & Tang, X. (2015). Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2), 295–307.
[2] Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A., Bishop, R., Rueckert, D., & Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1874–1883).
[3] Kim, J., Lee, J. K., & Lee, K. M. (2016). Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1646-1654).
[4] Kim, J., Lee, J. K., & Lee, K. M. (2016). Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1637-1645).
[5] Lai, W. S., Huang, J. B., Ahuja, N., & Yang, M. H. (2018). Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE transactions on pattern analysis and machine intelligence, 41(11), 2599-2613.
[6] Tong, T., Li, G., Liu, X., & Gao, Q. (2017). Image super-resolution using dense skip connections. In Proceedings of the IEEE international conference on computer vision (pp. 4799-4807).
[7] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 27.
[8] Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., & others (2017). Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4681–4690).
[9] Yuan, Y., Liu, S., Zhang, J., Zhang, Y., Dong, C., & Lin, L. (2018). Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 701-710). 
[10] Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision (pp. 2223-2232).
[11] Bulat, A., Yang, J., & Tzimiropoulos, G. (2018). To learn image super-resolution, use a gan to learn how to do image degradation first. In Proceedings of the European conference on computer vision (ECCV) (pp. 185-200).
[12] Mirza, M., & Osindero, S. (2014). Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784.
[13] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
[14] Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
[15] Miyato, T., Kataoka, T., Koyama, M., & Yoshida, Y. (2018). Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957. [16] Kingma, D., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
[17] Chen, S., Han, Z., Dai, E., Jia, X., Liu, Z., Xing, L., Zou, X., Xu, C., Liu, J., & Tian, Q. (2020). Unsupervised image super-resolution with an indirect supervised path. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 468–469).
[18] Wang, Z., Chen, J., & Hoi, S. (2020). Deep Learning for Image Super-resolution: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence