利用人工智慧技術來創作詩文與對聯是一個非常有趣的事。本研究使用全唐詩及全宋詩當成訓練資料庫，並利用Generative Pre-trained Transformer (GPT-2) 語言模型來建立古詩與對聯生成系統。使用檢測器來檢視篩選符合古詩及對聯規則所生成句子，再結合Jieba及BERT詞向量，讓生成的詩句能夠詞性兩兩相對，使詩的美感能夠更上一層樓。本研究使用了人工評比以及古詩音節評分兩項指標，人工評比又分為流暢度、主題性、意境三項，而實現對照組選用近兩年的三篇古詩生成論文所生成的詩句。研究結果顯示，本研究提出的方法，在流暢度、意境及音節評分都位居第一，而主題性則排名第二，證明了此方法的有效性。

It is a very interesting to use Artificial Intelligence technology to generate poetry and couplets, In this study, we use the entire Tang poetry and the Song poetry as the training dataset, and use the Generative Pre-trained Transformer (GPT-2) language model to build a Chinese classical poetry and couplet generation system. Use the inspector to select and generated verses that meet the rules of classical poetry and couplet. Combined with Jieba and BERT word vectors, the generated verses can be opposite each other in terms of parts of speech, so that the beauty of the poem can be improved. This study used two metrics: human evaluation and classical poetry syllable score. In addition, the human evaluation was divided into three parts: fluency, theme, and artistic conception. The experimental control group selected the verses generated by the latest three classical poetry generation papers in the past two years. The results show that the method proposed in this study rank first in fluency, artistic conception and syllable score, while thematic is ranked second, which proves the effectiveness of this method.

目錄
中文摘要	ii
英文摘要	iii
目錄	iv
圖目錄	vi
表目錄	viii
第一章 緒論	1
1.1研究背景	1
1.2研究動機與目的	2
第二章 文獻探討	3
2.1古詩文學創作	3
2.2自然語言處理	3
2.3 Transformer	6
2.4古詩生成	9
2.5對聯生成	16
第三章 古詩生成系統	18
 3.1系統架構	18

3.1.1 古詩生成系統架構	18
3.1.2 對聯生成系統架構	20
3.2 資料庫	20
  3.2.1 全唐詩資料庫	20
  3.2.2 全宋詩資料庫	21
  3.2.3 對聯資料庫	21
3.3前處理	22
3.3.1資料前處理	22
3.3.2詞彙字典	24
3.4 GPT-2參數設定	27
3.5 GPT-2模型	29
3.6 GPT-2生成候選詞	32
3.7古詩規則	33
3.7.1平仄規則	33
3.7.2 韻腳規則	35
3.7.3平水韻	36
3.8 平仄檢測	37
3.9 韻腳檢測	38
3.10 Jieba分詞	39
3.11 詞性標註	42
3.12 BERT詞向量	44
3.13 餘弦距離	45
第四章 實驗結果	48
4.1 實驗設備與樣本環境	48
4.2 實驗評比指標	49
  4.2.1 人工評比	49
  4.2.2 古詩音節評分	49
4.3 實驗的對照組	49
4.4 實驗結果呈現	52
第五章 結論與未來展望	55
參考文獻	56
附錄一 英文論文	61

 
圖目錄
圖 1  循環神經網路架構	4
圖 2  長短期記憶模型架構	6
圖 3  自注意力機制和多頭注意力機制	8
圖 4  Transformer架構	9
圖 5  循環神經網路古詩生成之架構圖	11
圖 6  基於主題模型和統計機器翻譯方法之架構圖	12
圖 7  主題性的關鍵字詞	12
圖 8  兩階段模型架構圖	13
圖 9  以圖像為輸入之古詩生成架構圖	14
圖 10  AnchiBERT 模型之架構圖	15
圖 11  古詩生成架構流程圖	19
圖 12  對聯生成架構流程圖	20
圖 13  對聯範例	22
圖 14  全唐詩及全宋詩資料庫	24
圖 15  字彙字典	25
圖 16  字彙向量化後之相對性	28
圖 17  字與字之相關性	29
圖 18  18 Transformer之解碼器	30
圖 19  GPT-2之四種模型	31
圖 20  自注意力(左) 與 遮罩自注意力(右)	32
圖 21  平仄字典	38
圖 22  韻腳字典	39
圖 23  trie樹示意圖	40
圖 24  有向無環圖之示意圖	41
圖 25  詞向量圖像化	44



 
表目錄

表 1  字與位置對應	26
表 2  五言絕句平仄格式	34
表 3  七言絕句平仄格式	34
表 4  王維所作之相思	35
表 5  平水韻	36
表 6  jieba詞性	42
表 7  詞性合併	43
表 8  餘弦距離比較	47
表 9  實驗設備與設置的軟體環境	48
表 10  實驗的總訓練時間與參數	48
表 11  實驗對照組之各個實驗成果	50
表 12  各實驗成果評分	51

參考文獻
[1]	Guo Zhipeng, Xiaoyuan Yi, Maosong Sun, Wenhao Li, Cheng Yang, Jiannan Liang, Huimin Chen, Yuhui Zhang and Ruoyu Li, “Jiuge--A Human-Machine Collaborative Chinese Classical Poetry Generation System” Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 25-30, 2019
[2]	Brent Smith and Greg Linden, “Two Decades of Recommender Systems at Amazon.com,” IEEE Internet Computing, vol. 21, pp. 12-18, 2017
[3]	Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser and Illia Polosukhin, “Attention Is All You Need,” arXiv:1706.03762, 2017.
[4]	Ilya Sutskever, Oriol Vinyals and Quoc V.Le, “Sequence to Sequence Learning with Neural Networks,” arXiv:1409.3215, 2014
[5]	Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” arXiv:1810.04805, 2019.
[6]	Alec Radford, Karthik Narasimhan, Tim Salimans and Ilya Sutskever, “Improving Language Understanding by Generative Pre-Training,” Technical report, OpenAI, 2018
[7]	Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei and Ilya Sutskever, “Language Models are Unsupervised Multitask Learners,” Technical report, OpenAI, 2019
[8]	Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis and Luke Zettlemoyer, “Multilingual Denoising Pre-training for Neural Machine Translation,” arXiv:2001.08210, 2020
[9]	Julia El Zini, Yara Rizk, Mariette Awad and Jumana Antoun, “Towards A Deep Learning Question-Answering Specialized Chatbot for Objective Structured Clinical Examinations,” 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1-9, 2019
[10]	Oriol Vinyals and Quoc Le, “A Neural Conversational Model,” arXiv:1506.05869, 2015
[11]	Leila Arras, Grégoire Montavon, Klaus-Robert Müller and Wojciech Samek, “Explaining Recurrent Neural Network Predictions in Sentiment Analysis,” arXiv:1706.07206, 2017
[12]	Joon Son Chung, Arsha Nagrani and Andrew Zisserman, “VoxCeleb2: Deep Speaker Recognition,” arXiv:1806.05622, 2018
[13]	Leonard E. Baum and Ted Petrie, “Statistical Inference for Probabilistic Functions of Finite State Markov Chains,” The Annals of Mathematical Statistics, vol. 37, no. 6, pp. 1554-1563, 1966.
[14]	Mu Li, Jianfeng Gao, Changning Huang and Jianfeng Li, “Unsupervised Training for Overlapping Ambiguity Resolution in Chinese Word Segmentation,” Proceedings of the second SIGHAN workshop on Chinese language processing, vol. 17, pp. 1-7, 2003. 
[15]	Jeffrey L. Elman, “Finding Structure in Time,” Cognitive Science, vol. 14, pp. 179-211, 1990.
[16]	Ilya Sutskever, James Martens and Geoffrey Hinton, “Generating text with recurrent neural networks,” Proceedings of the 28th international conference on machine learning, pp. 1017–1024, 2011.
[17]	Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio, “ Neural Machine Translation by Jointly Learning to Align and Translate,” arXiv:1409.0473, 2014
[18]	Felix A. Gers, Jurgen Schmidhuber and Fred Cummins, “Learning to Forget: Continual Prediction with LSTM,” 1999 Ninth International Conference on Artificial Neural Networks ICANN 99, vol. 12, pp. 2451-2471, 1999.
[19]	Simon Colton, Jacob Goodwin and Tony Veale, “Full-FACE Poetry Generation,” Proceedings of ICCC-2012, the 3rd International Conference on Computational Creativity, pp. 95-102, 2012
[20]	原博和劉穎, “基於語意相似度的集句對詩系统,” 清華大學中國語言文學系, 2013
[21]	Xingxing Zhang and Mirella Lapata, “Chinese Poetry Generation with Recurrent Neural Networks,” Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 670-680, 2014
[22]	蔣銳瀅, 崔磊, 何晶, 周明, 潘志庚, “基於主題模型和統計機器翻譯方法的中文格律詩自動生成,” 計算機學報, 2015
[23]	Zhe Wang, Wei He, Hua Wu, Haiyang Wu, Wei Li, Haifeng Wang and Enhong Chen, “Chinese Poetry Generation with Planning based Neural Network,” arXiv:1610.09889, 2016
[24]	Xiaoyu Wang, Xian Zhong and Lin Li, “Generating Chinese Classical Poems Based on Images,” Proceedings of the International MultiConference of Engineers and Computer Scientists, pp. 40-44, 2018
[25]	Yujian Zhang, Yanjie Ren and Xiaoxiao Shang, “Chinese Poetry Generator with Extended Input Format,” University of Science and Technology of China, 2018
[26]	Xiaopeng Yang, Xiaowen Lin, Shunda Suo, and Ming Li, “Generating Thematic Chinese Poetry using Conditional Variational Autoencoders with Hybrid Decoders,” Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18), pp. 4539-4545, 2018
[27]	Yi Liao, Yasheng Wang, Qun Liu and Xin Jiang, “GPT-based Generation for Classical Chinese Poetry,” arXiv:1907.00151, 2019
[28]	梁健楠, 孫茂松, 楊成, 陳慧敏, 劉正皓, “基於神經網路的集句詩自動生成,” 計算機學報, 2019
[29]	Huimin Chen, Xiaoyuan Yi, Maosong Sun, Wenhao Li, Cheng Yang and Zhipeng Guo, “Sentiment-Controllable Chinese Poetry Generation,” Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, pp. 4925-4931, 2019 
[30]	Dayiheng Liu, Jiancheng Lv and Yunxia Li, “Generating Style-Specific Chinese Tang Poetry With a Simple Actor-Critic Model,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 3, pp. 313-321, 2019
[31]	Xiaoyuan Yi, Ruoyu Li, Cheng Yang, Wenhao Li and Maosong Sun, “MixPoet: Diverse Poetry Generation via Learning Controllable Mixed Latent Space,” arXiv:2003.06094, 2020
[32]	Huishuang Tian, Kexin Yang, Dayiheng Liu and Jiancheng Lv, “AnchiBERT: A Pre-Trained Model for Ancient Chinese Language Understanding and Generation,” arXiv:2009.11473, 2020
[33]	Long Jiang and Ming Zhou, “Generating Chinese Couplets using a Statistical MT Approach, ” Proceedings of the 22nd International Conference on Computational Linguistics, vol. 1, pp. 377–384, 2008
[34]	Rui Yan, Cheng-Te Li, Xiaohua Hu and Ming Zhang, “Chinese Couplet Generation with Neural Network Structures,” Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 2347-2357, 2016
[35]	Jiyuan Zhang, Zheling Zhang, Shiyue Zhang and Dong Wang ”VV-Couplet: An open source Chinese couplet generation system,” Proceedings, APSIPA Annual Summit and Conference, pp. 1756-1760, 2018
[36]	Yusen Liu, Dayiheng Liu and Jiancheng Lv, “Deep Poetry: A Chinese Classical Poetry Generation System,” The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20), pp. 13626-13627, 2020
[37]	廖榮凡,  沈希忠,  劉爽, “更具有感情色彩的詩歌生成模型,” 計算機系統應用, pp. 46-51, 2020
[38]	https://github.com/chinese-poetry/chinese-poetry
[39]https://github.com/wb14123/couplet-dataset