本研究結合閒聊系統與問答系統的優點嘗試實現一個NBA主題的多輪對話系統。答案回應方面使用Bert的問答模型，該模型是由台達電研究所的公開數據集訓練而成。聊天機器人背後的問答語料則使用網路爬蟲，爬取PTT NBA板上的文章。當用戶輸入一個問題，系統將依據BM25演算法從問答語料庫中檢索一篇與問句最相關的文章，之後將問句與該篇文章一起送進問答模型，從文章中擷取與問題最適合的答案回覆用戶。此外在多輪對話方面，本研究設計了一套判斷規則來解決對話中發生的指代與信息遺漏問題。

In this research, we combine the advantages of chit-chat dialogue system and question answering system to build a thematic multi-turn dialogue system for NBA. To answer the question, we use the Bert QA model which was trained by the delta reading comprehension dataset. The corpus of system is obtained by crawling articles from the PTT NBA discussion board. When the user enters a question, the system will use the bm25 algorithm to retrieve an article most relevant to the question, and then send the question along with the article into the Bert model and extract the most suitable answer to the question from the article. In addition, we design a set of rules to solve the problems of reference and information omissions of the multi-turn dialogue system.

目錄:
第一章 緒論	1
1.1研究背景	1
1.2研究動機	1
1.3研究目的	2
1.4論文架構	2
第二章 文獻探討	3
2.1自然語言處理(Natural Language Processing)	3
2.2聊天機器人( Chat bot )	4
2.3聊天機器人模型	4
2.3.1樣板式聊天機器人(Rule Based Model)	4
2.3.2檢索式聊天機器人(Retrieval Based Model)	5
2.3.3生成式聊天機器人(Generative Based Model)	5
2.4問答系統	5
2.5 Encoder-Decoder	6
2.6 Transformer	7
2.6.1 Attention	8
2.6.2 Self-Attention	9
2.7 BERT (Bidirectional Encoder Representations from Transformers)	10
2.7.1預訓練	10
2.7.2問答模型	11
2.8多輪對話	13
2.9 Okapi BM25	15
2.10 Word2Vec (Word to Vector)	16
2.10.1 CBOW(Continuous Bag-of-Word)	16
2.10.2 Skip-gram	17
第三章 研究方法與系統架構	18
3.1問題定義	18
3.2系統架構	19
3.2.1系統開發工具與實作環境	20
3.3資料蒐集與前處理	20
3.3.1斷詞	22
3.4資料儲存	23
3.5檢索式模型架構	24
3.5.1相似詞模型(Word2Vec Model)	25
3.6 BERT問答模型	26
3.6.1答案回覆	26
3.7多輪對話模型	27
3.7.1信息遺漏	27
3.7.2指代	28
3.7.3判斷規則	28
第四章 系統結果呈現與評估	34
4.1多輪對話結果	34
4.2使用者問卷調查	36
4.2.1問卷調查結果	36
五、結論、研究貢獻與未來展望	41
5.1結論	        41
5.2研究貢獻	41
5.3未來展望	41
參考文獻	        43
圖目錄:
圖 1 ENCODER-DECODER架構圖	7
圖 2注意力機制運作情境	        9
圖 3 將BERT應於於問答任務	10
圖 4  BERT模型輸入表示	        13
圖 5 BERT問答訓練示意圖	        13
圖 6 CBOW模型結構圖	        16
圖 7 SKIP-GRAM模型的結構圖	17
圖 8主題式多輪對話系統架構圖	19
圖 9資料蒐集流程	        21
圖 10爬蟲程式片段1	21
圖 11爬蟲程式片段2	22
圖 12 抓取文章的JSON格式	22
圖 13 MONGODB儲存的文章	24
圖 14文章檢索流程示意圖	25
圖 15問答模型輸出的候選答案	27
圖 16情況五處理範例	31
圖 17情況六處理範例	32
圖 18情況七處理範例	32
圖 19問題一結果	37
圖 20問題二結果	37
圖 21問題三結果	38
圖 22問題四結果	38
圖 23問題五結果	39
圖 24問題六結果	39
圖 25問題七結果	40
表目錄:
表 1聊天機器人種類	1
表 2 BERT問答模型訓練語料範例	12
表 3重寫當前對話範例	14
表 4 國內外閱讀理解資料集的簡介	18
表 5 開發工具與環境	20
表 6加入自定義字典的差異	23
表 7 MONGO DB中資料儲存的格式	24
表 8 CBOW與SKIP-GRAM結果比較	26
表 9 PTT文章標題	28
表 10 NBA常用詞彙範例	29
表 11情況一處理範例	29
表 12情況二處理範例	30
表 13情況三範例	30
表 14情況四範例	30
表 15情況八處理範例	33
表 16多輪對話範例一	34
表 17多輪對話範例二	35
表 18多輪對話範例三	35
表 19多輪對話範例四	35
表 20多輪對話範例四	36
表 21 系統回覆擦邊球答案的範例	37
表 22系統意見歸納結果	40

[1] Barwise, J. 1977. "An introduction to first-order logic," In Studies in Logic and the Foundations of Mathematics, pp. 5-46.
[2] LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. 2015. "Deep learning," nature 521.7553 , pp. 436-444.
[3] Weizenbaum, Joseph. 1996. "ELIZA—a computer program for the study of natural language communication between man and machine," Communications of the ACM 9.1, pp. 36-45.
[4] Alison, Darcy., et al. 2017. "Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial," JMIR mental health.
[5] Zhang, Hainan, et al. 2018. "Reinforcing Coherence for Sequence to Sequence Model in Dialogue Generation," In IJCAI, pp. 4567-4573.
[6] Sutskever, Vinyals., et al. 2014. "Recurrent neural network regularization," arXiv preprint arXiv:1409.2329.
[7] Soares, M. A. C., et al. 2020. "A literature review on question answering techniques, paradigms and systems," Journal of King Saud University –Computer and Information Sciences, pp. 635-346.
[8] Malik, N., et al. 2013. "Domain knowledge enriched framework for restricted domain question answering system," 2013 IEEE International Conference on Computational Intelligence and Computing Research.
[9] Moldovan, D., et al. 2003. "Performance issues and error analysis in an open-domain question answering system," ACM Transactions on Information Systems.
[10] Bhoir, V., Potey, M. A. 2014. "Question answering system: A heuristic approach," The Fifth International Conference on the Applications of Digital Information and Web Technologies.
[11] Vaswani, Ashish., et al. 2017. "Attention is all you need," Advances in neural information processing systems 30.
[12] Devlin, J., et al. 2019. "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," arXiv preprint arXiv.
[13] Henderson, M., et al. 2013. " Deep neural network approach for the dialog state tracking challenge, " Proceedings of the SIGDIAL 2013 Conference, pp. 467-471.
[14] Gao, S., et al. 2019. "Dialog state tracking: A neural reading comprehension approach," arXiv preprint arXiv.
[15] Su, H., et al. 2019. "Improving multi-turn dialogue modelling with utterance ReWriter," arXiv preprint arXiv.
[16] Robertson, Stephen E., and K. Sparck Jones. 1976. "Relevance weighting of search terms," Journal of the American Society for Information science 27.3, pp. 129-146.
[17] Mikolov, Tomas., et al. 2013. "Distributed representations of words and phrases and their compositionality," Advances in neural information processing systems 26.