本論文嘗試利用人工智慧技術來建立一個心理分析的聊天機器人。我們使用心理諮詢問答語料庫和社區問答JSON版之資料庫為訓練資料集，利用自然語言處理(Natural Language Processing)技術進行語言模型的預訓練，以建立心理分析聊天機器人。其中使用了無監督式的GPT-2模型做為學習模型，學習字與字之間的關係與上下文句的文意理解，並使用情感字典搭配情感分析(sentiment analysis)技術來增加訊息特徵，加強GPT-2模型對情感上的學習，最終生成具有心理分析的回覆文本。本實驗使用BLEU自然語言評測指標來檢測其生成模型的成果，其表現明顯優於其他的生成式模型，且與優化後的檢索式模型相差無幾，證明了生成式模型也能做到與檢索式模型的高品質回覆。

This research attempts to use Artificial Intelligence technology to build a chatbot for psychoanalysis. We use the psychological consultation question and answer corpus and the JSON version of the community question and answer database as the training dataset, and use Natural Language Processing technology to pre-train the language model to build a chatbot for psychoanalysis. Among them, the unsupervised GPT-2 model is used as the learning model to learn the relationship between words and the textual understanding of contextual sentences. In addition, we use sentiment dictionary with sentiment analysis technology to increase information features so as to strengthen the emotional learning of the GPT-2 model.  Finally, a response text with psychological analysis is generated. This experiment uses the BLEU natural language evaluation index to measure the results of its generative model. The experimental performance is significantly better than other generative models, and it is almost the same as the optimized retrieval model, which proves that the generative model can also do the same as the retrieval model for high-quality responses from our proposed chatbot.

目錄
中文摘要	i
英文摘要	ii
目錄	iii
圖目錄	vi
表目錄	viii
第一章 緒論	1
1.1研究背景與動機	1
1.2研究動機與目的	2
1.3論文架構	3
第二章 文獻探討	4
2.1心理分析聊天機器人	4
2.2自然語言處理	4
2.3 Transformer	8
2.4聊天機器人	13
2.5 情感分析	15
2.6 GPT-2	17
第三章 心理分析聊天機器人系統	21
3.1 系統流程架構	21
3.2.1 心理諮詢問答語料庫	24
3.2.2 社區問答json版之資料庫	26
3.2.3 字彙字典	28
3.2.4 資料前處理	33
3.3 GPT-2前文摘要	35
3.3.1 GPT-2摘要模型-前處理	37
3.4情感標註	38
3.4.1 情感字典	38
3.4.2 結巴分詞	41
3.4.3 情感標註流程	42
第四章 實驗結果	43
4.1 實驗設備與樣本環境	43
4.2 實驗評比指標	43
4.2.1 BLEU評分	44
4.3 實驗的對照組	45
4.4 實驗結果呈現	47
第五章 結論與未來展望	50
參考文獻	51
附錄一 英文論文	54

 
圖目錄
圖 1 馬可夫模型之狀態變遷	5
圖 2 循環神經網路架構	6
圖 3 長短期記憶模型架構	7
圖 4  GATED RECURRENT NETWORKS	8
圖 5  TRANSFORMER架構圖	10
圖 6  SCALED DOT-PRODUCT ATTENTION	11
圖 7  MUTI-HEAD ATTENTION	12
圖 8 聊天機器人類型	14
圖 9情感字典情感分析架構	16
圖 10  OVERVIEW OF ECM	17
圖 11  GPT-2架構圖	18
圖 12  SELF- ATTENTION與MASKED SELF- ATTENTION	19
圖 13 心理分析聊天機器人架構流程圖	23
圖 14  GPT-2訓練架構	24
圖 15  心理諮詢問答語料庫資料結構	25
圖 16  心理諮詢問答語料庫	26
圖 17  社區問答JSON版之資料庫	27
圖 18  GPT-2字彙字典	28
圖 19  GPT-2字彙字典參數定義	29
圖 20  字彙向量化後之相對性	31
圖 21 字與字之相關性	32
圖 22  自注意力機制範例	33
圖 23  訓練用JSON格式	35
圖 24  GPT-2文本應用	36
圖 25  多輪對話範例	37
圖 26  情感字典	38
圖 27  情感字典分類	39
圖 28  有向無環圖	41
圖 29  情感標註流程範例	42


 
表目錄
表 1  社區問答JSON版之資料庫-標籤定義	27
表 2  前處理步驟	34
表 3  原21大分類與代表字詞	39
表 4  實驗設備與設置的軟體環境	43
表 5  實驗的總訓練時間	43
表 6  實驗對照組A成果	46
表 7  實驗對照組B成果	46
表 8  實驗對照	47
表 9  實驗結果(一)	47
表 10  實驗結果(二)	48
表 11  實驗結果(三)	48
表 12 實驗結果(四)	49

[1]	Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” 33rd Conference on Neural Information Processing Systems, arXiv:1810.04805, 2019.

[2]	Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever, “Language models are unsupervised multitask learners,” Technical report, OpenAI, 2019

[3]	Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser and Illia Polosukhin, “Attention Is All You Need,” 31st Conference on Neural Information Processing Systems, arXiv:1706.03762, 2017

[4]	https://www.twreporter.org/a/suicide-prevention-in-taiwan

[5]	Alan M. Turing, “Computing Machinery and Intelligence,” Parsing the Turing Test, pp.23-65, 2009

[6]	Jeffrey L. Elman, “Finding Structure in Time,” Cognitive Science, 1990.

[7]	Ilya Sutskever, Oriol Vinyals and Quoc V.Le, “Sequence to Sequence Learning with Neural Networks,” arXiv:1409.3215, 2014

[8]	Alex Graves, Abdel-rahman, Mohamed and Geoffrey Hinton, “Speech Recognition With Deep Recurrent Neural Networks,”2013 IEEE international conference on acoustics, speech and signal processing, arXiv:1303.5778, 2013.

[9]	Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio, “Neural Machine Translation by Jointly Learning to Align and Translate,” arXiv:1409.0473, 2014

[10]	Felix A. Gers, Jurgen Schmidhuber and Fred Cummins, “Learning to Forget: Continual Prediction with LSTM,” 1999 Ninth International Conference on Artificial Neural Networks ICANN 99, vol. 2, pp.850-855, 1999.


[11]	Junyoung Chung, Caglar Gulcehre, KyungHyun Cho and Yoshua Bengio “Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling,” arXiv:1412.3555, 2014.

[12]	Joseph Weizenbaum, “ELIZA — A Computer Program for the Study of Natural Language Communication between Man and Machine," Communications of the Association for Computing Machinery, vol. 9, pp.36-45, 1966

[13]	Rozann Hansford, Leo Joskowicz and Alan David Kalvin “An overview of computer-integrated surgery at the IBM Thomas J. Watson Research Center”, IBM Journal of Research and Development, vol. 40, pp.163-183, 1996

[14]	Chih-kuo Chen, “Effects of Chatbots on the English Learning of Senior High School Students in Taiwan,” Department of English, National Taiwan Normal University, 2013

[15]	Hao-Min Yan, “Applying Web Text Mining Approaches to Build Domain Knowledge Base: Case Studies on Customer Services and Chatbots,” Department of Information Engineering, National Jinan International University, 2017

[16]	Liang Zhang, Yan Yang, Jie Zhou, Chengcai Chen and Liang He, “Retrieval-Polished Response Generation for Chatbot,” IEEE Access, vol. 8, pp.123882 – 123890, 2020

[17]	Jiun-Hao Jhan, “Empathetic and Retrieval-based Chatbot using Deep Reinforcement Learning,” Institute of Telecommunications Engineering, National Taiwan University, 2020

[18]	Chao-Peng Liu, “Empathetic Generative-based Chatbot with Emotion Understanding via Reinforcement Learning,” Institute of Electrical Engineering, National Taiwan University, 2020

[19]	Yi-Jhen Lin, “Implementation of Chatbots on the Functional and Emotional Aspects of Major Epidemics : Taking COVID-19 for Example,” Department of Information and Financial Management, National Taipei University of Technology, 2021

[20]	Ye, Qiang, Ziqiong Zhang, and Rob Law. “Sentiment classification of online reviews to travel destinations by supervised machine learning approaches,” Expert systems with applications, vol. 36 pp. 6527-6535, 2009

[21]	Yiou Lin, Hang Lei, Jia Wu and Xiaoyu Li, “An Empirical Study on Sentiment Classification of Chinese Review using Word Embedding,” Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation: Posters, pp.258-266, 2015

[22]	V. Sahayak, V. Shete and A. Pathan, “Sentiment analysis on twitter data,” International Journal of Innovative Research in Advanced Engineering (IJIRAE), vol. 2, no. 1, pp. 178-183, 2015.

[23]	Kuang-Chih Li, “Predicting Online News Popularity Using Web Content,” National Taiwan University Institute of Electrical Engineering, 2017

[24]	Md. Rakibul Hasan, Maisha Maliha; M. Arifuzzaman, “Sentiment Analysis with NLP on Twitter Data,” 2019 International Conference on Computer, Communication, Chemical, Materials and Electronic Engineering, pp.98-101, 2019.

[25]	Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu and Bing Liu, “Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory,” arXiv:1704.01074, 2018

[26]	Yu-Sen Shih, “Commodity Sales Forecasting by combining Deep Learning Long Short-Term Memory Network (LSTM) with sentiment analysis” Department of Information and Financial Management, National Taipei University of Technology, 2019

[27]	Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu, “Bleu: A Method for Automatic Evaluation of Machine Translation,” Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311-318, 2002

[28]	https://github.com/chatopera/efaqa-corpus-zh

[29]	https://github.com/brightmart/nlp_chinese_corpus