客服機器人，通常需要瞭解某一個領域的知識，並具有理解客戶的問題，並能將知識生成為答案，以便回應給客戶。由於大語言模型的發展快速，近年來也有眾多的研究以RAG的技術來發展客服機器人，但其中的困難在於如何有效整合靜態知識庫與動態用戶查詢。
本研究以台灣大學網頁為例，利用階層式知識圖譜和大語言模型實現一個高效的問答機器人系統，主要服務對象為高中生及其家長。首先，我們透過爬取並整合各大學的網站內容，建立了一個結構化且階層分明的知識圖譜，用以支撐高效的問答機器人系統。這個知識圖譜按照訊息的層次結構進行組織，從最上層的廣泛分類開始逐步細化到具體內容。具體而言，第一層主要包括大類別，如「行政管理」、「教學課程」和「學生活動」等，這些類別如同網站的大標題，為用戶查詢提供初步導向。隨後，每個大類別進一步細分為更具體的次級類別或節點，例如在「教學課程」下可以分為「本科課程」、「研究所課程」等。最後，在這些次級節點下，還有更細致的訊息層次，如具體課程詳情、教師資訊等，形成第三層。這種從上至下的訊息層次結構使得機器人在處理查詢時能夠根據關鍵字逐層縮小搜索範圍，從而快速精確地定位到用戶需要的訊息，解決了過去單一層次知識圖譜難以滿足複雜查詢的問題。透過訓練圖神經網路，系統能夠理解並有效利用這種層次結構，優化訊息檢索的過程，解決了現有系統對於隱含或模糊問題解析不足的問題。當使用者提出問題時，系統會利用大語言模型進行語義解析，將問題轉化為向量，在知識圖譜中進行匹配。對於不能立即回答的問題，系統會搜索Wiki知識圖譜，補充現有知識圖譜的不足，提供即時且準確的回答。
相較於現有以RAG為基礎的QA客服機器人，本研究的主要貢獻包括：1) 提出一個結合階層式知識圖譜的問答系統架構，有效提升了問答的廣度和深度；2) 首次將圖神經網路應用於大語言模型，以提高問題理解的精準度；3) 增強了機器人對於複雜和多層次查詢的回應能力。實驗顯示，本系統在處理多層次學術查詢方面，相較於傳統RAG模型，回答準確率提高了2.3%，顯著改善了用戶的查詢體驗。

Customer service robots typically need to understand knowledge in a specific domain and are capable of comprehending customer inquiries and generating knowledge-based answers to respond to customers. Due to the rapid development of large language models, many studies in recent years have employed Retrieval-Augmented Generation (RAG) technology to develop customer service robots, but a challenge lies in effectively integrating static knowledge bases with dynamic user queries. 
This study takes the example of the National Taiwan University website to implement an efficient question-answering robot system using a hierarchical knowledge graph and a large language model, primarily serving high school students and their parents. Initially, by crawling and integrating the content of various university websites, we established a structured and clearly hierarchical knowledge graph to support an efficient question-answering robot system. This knowledge graph is organized according to the hierarchy of information, starting from the top-level broad categories and gradually refining to specific content. Specifically, the first layer primarily includes major categories such as "Administrative Management," "Educational Courses," and "Student Activities," which serve as the website's main headings, providing initial guidance for user queries. Subsequently, each major category is further divided into more specific subcategories or nodes, such as "Undergraduate Courses" and "Graduate Courses" under "Educational Courses." Lastly, under these subcategories, there are even more detailed levels of information, such as specific course details and faculty information, forming the third layer. This top-down hierarchical structure allows the robot to narrow down the search scope layer by layer based on keywords, thereby quickly and precisely locating the information needed by the user, solving the problem that past single-layer knowledge graphs could not meet complex queries. Through training a graph neural network, the system can understand and effectively utilize this hierarchical structure, optimizing the information retrieval process and solving the issue of inadequate resolution of implicit or vague queries by existing systems. When users pose questions, the system uses the large language model for semantic analysis, transforming the questions into vectors and matching them in the knowledge graph. For questions that cannot be answered immediately, the system searches the Wiki knowledge graph to supplement the existing knowledge graph's deficiencies, providing timely and accurate answers.
Compared to existing RAG-based QA customer service robots, the main contributions of this study include: 1) proposing a question-answering system architecture that integrates a hierarchical knowledge graph, significantly enhancing the breadth and depth of question-answering; 2) applying a graph neural network to a large language model for the first time, improving the precision of problem understanding; 3) enhancing the robot's response capabilities for complex and multi-layered queries. Experiments show that in handling multi-layered academic queries, this system improves the accuracy of responses by 2.3% compared to traditional RAG models, significantly enhancing the user's query experience.

LIST OF CONTENT

LIST OF CONTENT	VII
LIST OF FIGURE	IX
LIST OF TABLE	XII
CHAPTER 1 INTRODUCTION	1
1-1Research Background	1
1-2 Research Motivation	3
1-3 Research Objectives	4
1-4 Research Contribution	6
CHAPTER 2 RELATED WORKS	10
2-1 Keyword Matching – Based Approaches	10
2-2 Vector Matching – Based Approaches	12
2-3 Overview	15
CHAPTER 3 BACKGROUND KNOWLEDGE	17
3-1 KeyBERT	17
3-2 CKIP	18
3-3 GNN	20
3-3-1 GCN	22
3-4 LLama	25
3-5 Cosine Similarity	38
3-6 Fuzzy Wuzzy	39
CHAPTER 4 SYSTEM DESIGN	40
4-1 Overall Architecture	40
4-2 Data Preprocessing	42
4-3 Building the Hierarchical Knowledge Graph	47
4-4 Training the Graph Neural Network	51
4-5 Entity Alignment	53
4-6 User Query Processing Workflow	55
CHAPTER 5 EXPERIMENTAL ANALYSIS	59
5-1 Environment and System Parameter Setup	59
5-2 Dataset	60
5-3 Experimental Data and Results	62
5-4 Future Work	70
CHAPTER 6 CONCLUSION	72
REFERENCES	73

LIST OF FIGURE
FIGURE 1、Research Motivation	4
FIGURE 2、Research Objectuves	5
FIGURE 3、Overall Objectives	6
FIGURE 4、Research Contribution	9
FIGURE 5、KeyBERT structure	18
FIGURE 6、CKIP sturcture	20
FIGURE 7、GNN structure	21
FIGURE 8、GNN Common Tasks	22
FIGURE 9、GCN structure	22
FIGURE 10、Transfer of the Convolution Concept	24
FIGURE 11、Llama structure	25
FIGURE 12、Llama training phase	26
FIGURE 13、LLama Parameter Counts (as referenced in [23])	27
FIGURE 14、DataSets	28
FIGURE 15、LLama Tokenizer	29
FIGURE 16、Llama Embeddings	29
FIGURE 17、Llama Parametric chart	30
FIGURE 18、Comparison of Attention Mechanisms	31
FIGURE 19、LLama Downstream Task: Text Classification	33
FIGURE 20、LLama Downstream Task: Textual Entailment (Supportive)	34
FIGURE 21、LLama Downstream Task: Textual Entailment (Contradictory)	35
FIGURE 22、LLama Downstream Task: Similarity Calculation	36
FIGURE 23、LLama Downstream Task: Multiple-Choice Reading Comprehension	37
FIGURE 24、Overall System Architecture and Workflow	42
FIGURE 25、Data Acquisition	43
FIGURE 26、Enhancing Semantic Understanding During Crawling	44
FIGURE 27、Total Data	44
FIGURE 28、Text Transformation into Triple Relationships	45
FIGURE 29、text-embedding-ada-002 model	48
FIGURE 30、Neo4j Database	49
FIGURE 31、Hierarchical Knowledge Graph	51
FIGURE 32、Horizontal Feature Aggregation	52
FIGURE 33、Vertical Feature Aggregation	52
FIGURE 34、WikiKG Content	54
FIGURE 35、Keyword Alignment with WikiKG	54
FIGURE 36、User Query Matching with Hierarchical Knowledge Graph	56
FIGURE 37、User Query Matching Path	56
FIGURE 38、Determining if the User's Question Can Be Answered	57
FIGURE 39、Resolving Incorrect Path Matching	58
FIGURE 40、Loss Values During Training Process	64
FIGURE 41、Precision Comparison Based on Training Data Volume	65
FIGURE 42、Recall Comparison Based on Training Data Volume	66
FIGURE 43、F1-Score Comparison Based on Training Data Volume	66
FIGURE 44、Impact of Embedding Length on Model Performance	67
FIGURE 45、BLEU-Score Similarity Comparison	69

LIST OF TABLE
TABLE 1、Comparison of Related Studies	15
TABLE 2、Hardware Configuration	60
TABLE 3、Software Configuration	60
TABLE 4、Model Ablation Study	63
TABLE 5、Comparative Analysis of Question to Knowledge Graph Entity Matching	69

REFERENCES
[1]	X. Huang, J. Zhang, D. Li, and P. Li, "Knowledge graph embedding based question answering," in Proceedings of the twelfth ACM international conference on web search and data mining, 2019, pp. 105-113. 
[2]	J. Jiang, K. Zhou, W. X. Zhao, and J.-R. Wen, "Unikgqa: Unified retrieval and reasoning for solving multi-hop question answering over knowledge graph," arXiv preprint arXiv:2212.00959, 2022.
[3]	P. Lewis et al., "PAQ: 65 million probably-asked questions and what you can do with them," Transactions of the Association for Computational Linguistics, vol. 9, pp. 1098-1115, 2021.
[4]	S. Pan, L. Luo, Y. Wang, C. Chen, J. Wang, and X. Wu, "Unifying large language models and knowledge graphs: A roadmap," IEEE Transactions on Knowledge and Data Engineering, 2024.
[5]	S. Sobolevsky, "Hierarchical graph neural networks," arXiv preprint arXiv:2105.03388, 2021.
[6]	W. Ding, J. Li, L. Luo, and Y. Qu, "Enhancing complex question answering over knowledge graphs through evidence pattern retrieval," in Proceedings of the ACM on Web Conference 2024, 2024, pp. 2106-2115. 
[7]	L. Yang, H. Guo, Y. Dai, and W. Chen, "A Method for Complex Question-Answering over Knowledge Graph," Applied Sciences, vol. 13, no. 8, p. 5055, 2023.
[8]	Z. Zuo, Z. Zhu, W. Wu, W. Wang, J. Qi, and L. Zhong, "Improving question answering over knowledge graphs with a chunked learning network," Electronics, vol. 12, no. 15, p. 3363, 2023.
[9]	C. Mavromatis et al., "Tempoqr: temporal question reasoning over knowledge graphs," in Proceedings of the AAAI conference on artificial intelligence, 2022, vol. 36, no. 5, pp. 5825-5833. 
[10]	D. Vollmers, R. Jalota, D. Moussallem, H. Topiwala, A.-C. Ngonga Ngomo, and R. Usbeck, "Knowledge graph question answering using graph-pattern isomorphism," in Further with Knowledge Graphs: IOS Press, 2021, pp. 103-117.
[11]	S. Cai, Q. Ma, Y. Hou, and G. Zeng, "Knowledge Graph Multi-Hop Question Answering Based on Dependent Syntactic Semantic Augmented Graph Networks," Electronics, vol. 13, no. 8, p. 1436, 2024.
[12]	Q. Zhang et al., "NOAHQA: Numerical reasoning with interpretable graph question answering dataset," arXiv preprint arXiv:2109.10604, 2021.
[13]	S. Skandan, S. Kanungo, S. Devaraj, S. Gupta, and S. Narayan, "Question Answering System using Knowledge Graphs," in 2023 International Conference on Inventive Computation Technologies (ICICT), 2023: IEEE, pp. 656-661. 
[14]	A. Saxena, A. Kochsiek, and R. Gemulla, "Sequence-to-sequence knowledge graph completion and question answering," arXiv preprint arXiv:2203.10321, 2022.
[15]	H. Xiong, S. Wang, M. Tang, L. Wang, and X. Lin, "Knowledge graph question answering with semantic oriented fusion model," Knowledge-Based Systems, vol. 221, p. 106954, 2021.
[16]	P. Sen, S. Mavadia, and A. Saffari, "Knowledge graph-augmented language models for complex question answering," in Proceedings of the 1st Workshop on Natural Language Reasoning and Structured Explanations (NLRSE), 2023, pp. 1-8. 
[17]	S. Aghaei, S. Masoudi, T. R. Chhetri, and A. Fensel, "Question answering over knowledge graphs: a graph-driven approach," in 2022 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), 2022: IEEE, pp. 296-302. 
[18]	L. Jiang and R. Usbeck, "Knowledge Graph Question Answering Datasets and Their Generalizability: Are They Enough for Future Research?," in Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022, pp. 3209-3218. 
[19]	M. Salnikov et al., "Large Language Models Meet Knowledge Graphs to Answer Factoid Questions," arXiv preprint arXiv:2310.02166, 2023.
[20]	S.-L. Huang, K.-J. Chen, W.-Y. Ma, S.-C. Lin, and Y.-M. Hsieh, "Semantic relation identification for consecutive predicative constituents in Chinese," Lingua Sinica, vol. 3, pp. 1-31, 2017.
[21]	F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, "The graph neural network model," IEEE transactions on neural networks, vol. 20, no. 1, pp. 61-80, 2008.
[22]	T. N. Kipf and M. Welling, "Semi-supervised classification with graph convolutional networks," arXiv preprint arXiv:1609.02907, 2016.
[23]	H. Touvron et al., "Llama 2: Open foundation and fine-tuned chat models," arXiv preprint arXiv:2307.09288, 2023.