隨著人工智慧和自然語言處理技術的迅速發展，基於深度學習的問答系統在各個領域得到廣泛應用。然而，針對專業領域如天文學的問答系統，仍面臨資料豐富度和回答準確性的挑戰。傳統問答系統依賴靜態知識庫，難以滿足動態且複雜的用戶需求。為解決此問題，本研究提出了一個基於檢索增強與ChatGPT生成的天文問答系統。
該系統結合高效檢索技術與生成模型，旨在提供更準確和豐富的答案，滿足用戶查詢需求。系統首先檢索最相關的問題並返回答案，如果找不到滿意的答案，則調用ChatGPT生成答案。為增強答案的全面性，系統還會從維基百科中補充相關內容。這種多層次、多技術融合的方法，有效提升了天文問答的準確性和用戶滿意度。
本研究的目標是提升天文問答系統的智能化水平，使其在專業性和即時性方面達到新高度，從而推動相關領域的知識普及和學術交流。實驗顯示，基於檢索增強與ChatGPT生成的天文問答系統在多項評估指標上均優於傳統問答系統。在精確率、召回率和F1-score方面，該系統的表現顯著提升，尤其在處理大量複雜查詢時，精確率達到95%以上。此外，系統的回答質量和相關性也顯著提高，用戶滿意度明顯增強。這表明該系統能有效應對天文學領域的專業問題，提供高質量、即時的回答，達到提升天文知識普及和學術交流的目標。

With the rapid development of artificial intelligence and natural language processing technology, question answering systems based on deep learning have been widely used in various fields. However, question and answer systems for professional fields such as astronomy still face challenges in data richness and answer accuracy. Traditional question answering systems rely on static knowledge bases and are difficult to meet dynamic and complex user needs. To solve this problem, this study proposes an astronomical question and answer system based on retrieval enhancement and ChatGPT generation.
The system combines efficient retrieval technology with generative models to provide more accurate and rich answers to meet user query needs. The system first retrieves the most relevant questions and returns answers. If no satisfactory answer is found, ChatGPT is called to generate the answer. To enhance the comprehensiveness of the answers, the system will also supplement relevant content from Wikipedia. This multi-level, multi-technology integration method effectively improves the accuracy and user satisfaction of astronomical Q&A.
The goal of this research is to improve the intelligence level of the astronomical question and answer system, bringing it to a new level in terms of professionalism and immediacy, thereby promoting the popularization of knowledge and academic exchanges in related fields. Experiments show that the astronomical question and answer system based on retrieval enhancement and ChatGPT generation is better than the traditional question and answer system in multiple evaluation indicators. In terms of precision, recall and F1-score, the system's performance has been significantly improved. Especially when processing a large number of complex queries, the precision rate has reached more than 95%. In addition, the system's answer quality and relevance have also been significantly improved, and user satisfaction has been significantly enhanced. This shows that the system can effectively respond to professional questions in the field of astronomy, provide high-quality, immediate answers, and achieve the goal of improving the popularization of astronomical knowledge and academic exchanges.

Contents
CONTENTS............................................................................................VII
FIGURE CATALOG..............................................................................VIII
TABLE OF CONTENTS...........................................................................X
CHAPTER 1、INTRODUCTION.............................................................1
CHAPTER 2: RELATED RESEARCH .....................................................6
2-1 SBERT（SENTENCE-BERT） .......................................................6
2-2 CHATGPT .......................................................................................9
2-3 RAG(RETRIEVAL-AUGMENTED GENERATION) .............................12
CHAPTER 3、BACKGROUND KNOWLEDGE ..................................17
3-1 SENTENCE BERT INTRODUCTION .................................................17
3-2 CHATGPT INTRODUCTION ............................................................19
3-3 RETRIEVAL-AUGMENTED GENERATION TECHNOLOGY..................20
CHAPTER 4、SYSTEM ARCHITECTURE..........................................22
4-1 ENVIRONMENT AND PROBLEM DESCRIPTION ................................22
4-1-1 Background and Motivation..............................................22
4-1-2 Objectives..........................................................................22
4-2 OVERALL SYSTEM ARCHITECTURE ...............................................23
4-3 DATA COLLECTION AND PREPROCESSING......................................25
4-4 DATA STORAGE AND QUERYING ....................................................26
4-5 VECTOR RETRIEVAL AND ANSWER GENERATION ..........................28
4-6 USER INTERFACE...........................................................................30
CHAPTER 5、EXPERIMENTAL ANALYSIS.......................................35
5-1 ENVIRONMENTAL SETUP...............................................................35
5-2 EXPERIMENTAL DATA...................................................................36
5-3 EXPERIMENTAL RESULTS..............................................................36
5-4 DISCUSSION ..................................................................................44
CHAPTER 6、CONCLUSION ...............................................................46
REFERENCES .........................................................................................48

Figure Catalog
FIGURE 1、DATA PREPROCESSING..................................................23
FIGURE 2、DATA STORAGE AND QUERY .......................................24
FIGURE 3、VECTOR SEARCH AND ANSWER GENERATION ......24
FIGURE 4、USER INTERFACE............................................................24
FIGURE 5、DATASETS: ASTRONOMICAL OBSERVATORY FAQS 
(LEFT), WIKIPEDIA (RIGHT) ...............................................................25
FIGURE 6、DATA PREPROCESSING AND DATA STORAGE & 
QUERYING..............................................................................................28
FIGURE 7、SENTENCE BERT ARCHITECTURE ..............................29
FIGURE 8、USER QUERY CONVERSION TO VECTOR AND 
COSINE SIMILARITY COMPARISON.................................................30
FIGURE 9、USER QUERY VECTOR MATCHING AND ANSWER 
GENERATION .........................................................................................30
FIGURE 10、QUESTION-ANSWERING FLOWCHART....................33
FIGURE 11、LINE CHATBOT INTERFACE........................................34
FIGURE 12、ACCURACY VS. AVERAGE RESPONSE TIME FOR 
DIFFERENT QA DATASET SIZES ........................................................41
FIGURE 13、PRECISION VS. AVERAGE RESPONSE TIME FOR 
DIFFERENT QA DATASET SIZES ........................................................41
FIGURE 14、RECALL VS. AVERAGE RESPONSE TIME FOR 
DIFFERENT QA DATASET SIZES ........................................................42
IX
FIGURE 15、F1-SCORE VS. AVERAGE RESPONSE TIME FOR 
DIFFERENT QA DATASET SIZES ........................................................42
FIGURE 16、COMPARISON OF EVALUATION RESULTS 
BETWEEN THIS THESIS MODELS AND RESEARCH[12] ...............44

Table of Contents 
TABLE 1、COMPARATIVE ANALYSIS OF RELATED RESEARCH15
TABLE 2、MODEL TRAINING ENVIRONMENT..............................35
TABLE 3、EXAMPLE OF A CONFUSION MATRIX..........................37
TABLE 4、EVALUATION RESULTS OF DIFFERENT MODELS .....39
TABLE 5、COMPARISON OF EVALUATION RESULTS 
BETWEEN RESEARCH [12] AND THIS THESIS MODELS ..............43

References
[1]	N. Reimers and I. Gurevych, "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks," Ubiquitous Knowledge Processing Lab (UKP-TUDA), Department of Computer Science, Technische Universität Darmstadt, arXiv preprint arXiv:1908.10084, Aug. 27, 2019.
[2]	T.-T. Ha, V.-N. Nguyen, K.-H. Nguyen, K.-A. Nguyen, and Q.-K. Than, "Utilizing SBERT For Finding Similar Questions in Community Question Answering," in *Proc. 13th Int. Conf. on Knowledge and Systems Engineering (KSE)*, Hanoi, Vietnam, 2021, pp. 1-6. doi: 10.1109/KSE53942.2021.9648830.
[3]	K. Tsumuraya, M. Uehara, M. Amano, and Y. Adachi, "Topic-Based Clustering of Japanese Sentences Using Sentence-BERT," in *Proc. 2022 10th Int. Symp. on Computing and Networking Workshops (CANDARW)*, 2022, pp. 255-260, doi: 10.1109/CANDARW57323.2022.00044.
[4]	S. M. M. R. Naqvi, S. Batool, M. Ahmed, H. U. Khan, and M. A. Shahid, "A Novel Approach for Building Domain-Specific Chatbots by Exploring Sentence Transformers-based Encoding," in *Proc. 2023 Int. Conf. on IT and Industrial Technologies (ICIT)*, 2023, pp. 1-8. doi: 10.1109/ICIT59216.2023.10335884.
[5]	B. Wang and C.-C. J. Kuo, "SBERT-WK: A Sentence Embedding Method by Dissecting BERT-Based Word Models," in *IEEE/ACM Transactions on Audio, Speech, and Language Processing*, vol. 28, pp. 2146-2157, July 2020, doi: 10.1109/TASLP.2020.3008390.
[6]	Y. Zhuang, X. Lin, B. Liu, and C. Xu, "V-SBERT: A Mixture Model for Closed-Domain Question-Answering Systems Based on Natural Language Processing and Deep Learning," in *Proc. 2023 6th Int. Conf. on Data Science and Information Technology (DSIT)*, 2023, pp. 328-333. doi: 10.1109/DSIT60026.2023.00058.
[7]	Y. Tan, D. Min, Y. Li, W. Li, N. Hu, Y. Chen, and G. Qi, "Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Family," in *Proc. 2023 6th Int. Conf. on Data Science and Information Technology (DSIT)*, 2023, pp. 1-20. doi: 10.48550/arXiv.2303.07992.
[8]	Q. Zhong, L. Ding, J. Liu, B. Du, and D. Tao, "Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT," arXiv preprint arXiv:2302.10198, 2023.
[9]	A. Bahrini, M. Khamoshifar, H. Abbasimehr, R. J. Riggs, M. Esmaeili, R. M. Majdabadkohne, and M. Pasehvar, "ChatGPT: Applications, Opportunities, and Threats," presented at the IEEE Systems and Information Engineering Design Symposium (SIEDS), 2023, pp. 1-15. doi: 10.48550/arXiv.2304.09103.
[10]	Y. Chen, H. Deng, C.-H. Chen, and C.-L. Chung, "Efficient Artificial Intelligence-Teaching Assistant Based on ChatGPT," 2023 International Conference on Smart Systems for Applications in Electrical Sciences (ICSSES), 2023, pp. 1-15. doi: 10.1109/ICSSES58299.2023.10200077.
[11]	C. Richardson and L. Heck, "Commonsense Reasoning for Conversational AI: A Survey of the State of the Art," presented at the Workshop on Knowledge Augmented Methods for Natural Language Processing, in conjunction with AAAI 2023, 2023.
[12]	H. Bahak, F. Taheri, Z. Zojaji, and A. Kazemi, "Evaluating ChatGPT as a Question Answering System: A Comprehensive Analysis and Comparison with Existing Models," arXiv preprint arXiv:2312.07592, 2023.
[13]	S. Barnett, S. Kurniawan, S. Thudumu, Z. Brannelly, and M. Abdelrazek, "Seven Failure Points When Engineering a Retrieval Augmented Generation System," 2024 IEEE/ACM 3rd International Conference on AI Engineering – Software Engineering for AI (CAIN), Lisbon, Portugal, 2024, pp. 1-6. doi: 10.1145/3644815.3644945.
[14]	S. Siriwardhana, R. Weerasekera, E. Wen, T. Kaluarachchi, R. Rana, and S. Nanayakkara, "Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering," arXiv preprint arXiv:2210.02627, 2022.
[15]	P. Omrani, A. Hosseini, K. Hooshanfar, Z. Ebrahimian, R. Toosi, and M. A. Akhaee, "Hybrid Retrieval-Augmented Generation Approach for LLMs Query Response Enhancement," 2024 10th International Conference on Web Research (ICWR), 2024. doi: 10.1109/ICWR61162.2024.10533345.
[16]	S. Vakayil, A. J., D. S. Juliet, and S. Vakayil, "RAG-Based LLM Chatbot Using Llama-2," 2024 7th International Conference on Devices, Circuits and Systems (ICDCS), Coimbatore, India, 2024. doi: 10.1109/ICDCS59278.2024.10561020.
[17]	M. Maryamah, M. M. Irfani, E. B. T. Raharjo, N. A. Rahmi, M. Ghani, and I. K. Raharjana, "Chatbots in Academia: A Retrieval-Augmented Generation Approach for Improved Efficient Information Access," 2024 16th International Conference on Knowledge and Smart Technology (KST), Surabaya, Indonesia, 2024. doi: 10.1109/KST61284.2024.10499652.
[18]	D. Hopp, "Free to Play: UN Trade and Development’s Experience with Developing its Own Open-Source RAG LLM Application," presented at the seventy-second plenary session of the Conference of European Statisticians, Geneva, Switzerland, 2024. doi: 10.48550/arXiv.2407.16896.
[19]	J. Chen, H. Lin, X. Han, and L. Sun, "Benchmarking Large Language Models in Retrieval-Augmented Generation," arXiv preprint arXiv:2310.12345, 2023.
[20]	H. Yang, M. Zhang, D. Wei, and J. Guo, "SRAG: Speech Retrieval-Augmented Generation for Spoken Language Understanding," 2024 IEEE 2nd International Conference on Control, Electronics and Computer Technology (ICCECT), Jilin, China, 2024. doi: 10.1109/ICCECT60629.2024.10546001.
[21]	T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, "Language Models are Few-Shot Learners," arXiv preprint arXiv:2005.14165, 2020.
[22]	P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks," arXiv preprint arXiv:2005.11401, 2020.