近年來，有許多機器學習圖片偵測的研究取得了顯著進展，其中包括R-CNN、YOLO、ResNet、SimCLR和MoCo等方法。例如，SimCLR利用對比學習的方式達成自監督式學習，通過對圖像進行隨機裁剪和變色，既保留了重要特徵，又減少了對圖片標籤的需求。然而，隨著圖片輸入種類的增加，可能會導致災難性遺忘，使機器忘記之前學習過的圖片特徵。為了解決災難性遺忘，已有許多研究提出持續學習的方法。CaSSLe架構就是其中一種融合自監督式學習與持續學習的方法，旨在避免機器災難性遺忘的同時，在圖片偵測上取得優異的效果。這種方法不僅提高了模型對新資料的適應能力，還維持了其對舊資料的準確度，有效解決了機器學習中常見的遺忘問題。本研究的主要目的是探索如何通過融合自監督式學習與持續學習的模型，來達到在避免機器災難性遺忘的同時，提升圖片偵測的準確性。我們提出了一種新的方法，結合了CaSSLe的持續學習架構，與基於質心計算的方式，來進一步強化現有持續學習的效率，提高模型預測的命中率

In recent years, there has been significant progress in research on machine learning for image detection, with notable examples including R-CNN, YOLO, ResNet, SimCLR, and MoCo. SimCLR, for instance, uses contrastive learning to achieve self-supervised learning with a small amount of labeled data by randomly cropping and changing the color of images, which preserves features while reducing the need for image labels. However, as the variety of image inputs increases, there is risk of catastrophic forgetting, where the machine forgets previously learned image features. To achieve the goal of preventing catastrophic forgetting, many recent works has been studying the possibility of Continual Learning. For example, the CaSSLe architecture is a framework that combines self-supervised learning with continual learning, aiming to prevent catastrophic forgetting while achieving excellent results in image detection. This approach not only enhances the model's adaptability to new dataset but also maintains accuracy of predicting previous dataset, effectively addressing the common forgetting problem in machine learning. The primary objective of this research is to explore how to achieve high accuracy in image detection while avoiding catastrophic forgetting by integrating models of self-supervised learning and continual learning. We propose a novel method that combines CaSSLe's continual learning framework with Centroid Calculations of image features to improve the ability of machine predictions on image objects.

Table of contents
Chinese Abstract	I
Abstract	III
Table of Contents 	VI
List of Figures 	V
List of Tables 	VI
Chapter 1 Introduction	1
Chapter 2 Related Work 	4
2.1 Image Classification	4
2.2 Contrastive Learning 	5
2.3 Continual Learning	6
Chapter 3 Method 	9
3.1 CaSSLe Framework	10
3.2 Centroid Calculation	11
Chapter 4 Experiment 	13
4.1 Evaluation	13
Chapter 5 Conclusion 	19
Reference 	20

List of Figures
Figure 1. C2SSL Flowchart	9
Figure 2. CaSSLe Framework 	10
Figure 3. Top-1 and Top-5 Accuracy for SimCLR + CaSSLe on Cifar-100 dataset 	15
Figure 4. Top-1 and Top-5 Accuracy for SimCLR + CaSSLe on ImageNet-100 dataset.	15
Figure 5. Top-1 Accuracy & Top-5 Accuracy for Centroid on Cifar-100	16
Figure 6. Top-1 Accuracy and Top-5 Accuracy for Centroid on ImageNet-100.	16
Figure 7. Top-1 Accuracy for SimCLR & Centroid on CaSSLe Framework in Cifar-100 and ImageNet-100.	16
Figure 8. Top-5 Accuracy for SimCLR & Centroid on CaSSLe Framework in Cifar-100 and ImageNet-100.	17

List of Tables
Table 1. Accuracy of task 1 to task 5 on Cifar-100.	17
Table 2. Accuracy of task 1 to task 5 on ImageNet-100.	18
Table 3. Top-5 Accuracy on Cifar-100 and ImageNet-100.	18
Table 4. Top-1 Accuracy on Cifar-100 and ImageNet-100.	18

[1]	Robins, Anthony. "Catastrophic forgetting, rehearsal and pseudorehearsal." Connection Science 7.2 (1995): 123-146.
[2]	De Lange, Matthias, et al. "Continual learning: A comparative study on how to defy forgetting in classification tasks." arXiv preprint arXiv:1909.08383 2.6 (2019): 2.
[3]	Chen, Ting, et al. "A simple framework for contrastive learning of visual representations." International conference on machine learning. PMLR, 2020.
[4]	Van de Ven, Gido M., and Andreas S. Tolias. "Three scenarios for continual learning." arXiv preprint arXiv:1904.07734 (2019).
[5]	Fini, Enrico, et al. "Self-supervised models are continual learners." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
[6]	LeCun, Yann, et al. "Gradient-based learning applied to document recognition." Proceedings of the IEEE 86.11 (1998): 2278-2324.
[7]	He, Kaiming, et al. "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[8]	Chen, Ting, et al. "Big self-supervised models are strong semi-supervised learners." Advances in neural information processing systems 33 (2020): 22243-22255.
[9]	He, Kaiming, et al. "Momentum contrast for unsupervised visual representation learning." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.
[10]	Chen, Xinlei, et al. "Improved baselines with momentum contrastive learning." arXiv preprint arXiv:2003.04297 (2020).
[11]	Wang, Yabin, Zhiwu Huang, and Xiaopeng Hong. "S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning." Advances in Neural Information Processing Systems 35 (2022): 5682-5695.
[12]	Lin, Zhiwei, Yongtao Wang, and Hongxiang Lin. "Continual contrastive learning for image classification." 2022 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2022.
[13]	Zhang, Shu, et al. "Use all the labels: A hierarchical multi-label contrastive learning framework." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
[14]	Xiao, Tete, et al. "What should not be contrastive in contrastive learning." arXiv preprint arXiv:2008.05659 (2020).
[15]	Khosla, Prannay, et al. "Supervised contrastive learning." Advances in neural information processing systems 33 (2020): 18661-18673.
[16]	Alex Krizhevsky, "Learning multiple layers of features from tiny images." (2009): 7.
[17]	Russakovsky, Olga, et al. "Imagenet large scale visual recognition challenge." International journal of computer vision 115 (2015): 211-252.
[18]	Peng, Xingchao, et al. "Moment matching for multi-source domain adaptation." Proceedings of the IEEE/CVF international conference on computer vision. 2019.
[19]	Douillard, Arthur, et al. "Podnet: Pooled outputs distillation for small-tasks incremental learning." Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, proceedings, part XX 16. Springer International Publishing, 2020.
[20]	Kirkpatrick, James, et al. "Overcoming catastrophic forgetting in neural networks." Proceedings of the national academy of sciences 114.13 (2017): 3521-3526.
[21]	Cha, Hyuntak, Jaeho Lee, and Jinwoo Shin. "Co2l: Contrastive continual learning." Proceedings of the IEEE/CVF International conference on computer vision. 2021.
[22]	Zenke, Friedemann, Ben Poole, and Surya Ganguli. "Continual learning through synaptic intelligence." International conference on machine learning. PMLR, 2017.
[23]	Prabhu, Ameya, Philip HS Torr, and Puneet K. Dokania. "Gdumb: A simple approach that questions our progress in continual learning." Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. Springer International Publishing, 2020.
[24]	Lopez-Paz, David, and Marc'Aurelio Ranzato. "Gradient episodic memory for continual learning." Advances in neural information processing systems 30 (2017).
[25]	Dwibedi, Debidatta, et al. "With a little help from my friends: Nearest-neighbor contrastive learning of visual representations." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.
[26]	Rao, Dushyant, et al. "Continual unsupervised representation learning." Advances in neural information processing systems 32 (2019).
[27]	Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network." arXiv preprint arXiv:1503.02531 (2015).
[28]	Wu, Yue, et al. "Large scale incremental learning." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.
[29]	Jung, Heechul, et al. "Less-forgetting learning in deep neural networks." arXiv preprint arXiv:1607.00122 (2016).
[30]	Li, Zhizhong, and Derek Hoiem. "Learning without forgetting." IEEE transactions on pattern analysis and machine intelligence 40.12 (2017): 2935-2947.
[31]	Chen, Xinlei, and Kaiming He. "Exploring simple siamese representation learning." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.
[32]	Chen, Xinlei, Saining Xie, and Kaiming He. "An empirical study of training self-supervised vision transformers." Proceedings of the IEEE/CVF international conference on computer vision. 2021.
[33]	Dosovitskiy, Alexey, et al. "An image is worth 16x16 words: Transformers for image recognition at scale." arXiv preprint arXiv:2010.11929 (2020).
[34]	Wen, Yandong, et al. "A discriminative feature learning approach for deep face recognition." Computer vision–ECCV 2016: 14th European conference, amsterdam, the netherlands, October 11–14, 2016, proceedings, part VII 14. Springer International Publishing, 2016.
[35]	Fang, Zhiyuan, et al. "Seed: Self-supervised distillation for visual representation." arXiv preprint arXiv:2101.04731 (2021).
[36]	Oord, Aaron van den, Yazhe Li, and Oriol Vinyals. "Representation learning with contrastive predictive coding." arXiv preprint arXiv:1807.03748 (2018).
[37]	Schwarz, Jonathan, et al. "Progress & compress: A scalable framework for continual learning." International conference on machine learning. PMLR, 2018.