由於對用戶興趣以及嗜好的精確預測，矩陣分解（matrix factorization，MF）技術已被廣泛應用於推薦系統中。先前基於矩陣分解的方法通過從使用者（user）和項目（item）中提取潛在因子（latent factor）來調整總體評級以進行推薦。然而，在實際應用中，人們的偏好通常會隨著時間的推進而發生改變，傳統基於矩陣分解的方法已經無法正確地捕捉用戶和興趣之間的變化。在這篇論文當中，通過將遞歸神經網絡（recurrent neural network，RNN）結合到矩陣分解中，我們開發了一種新穎的推薦系統M-RNN-F，以有效地描述用戶隨時間的偏好演變，提出了兩種學習模型來捕捉演化模式並預測未來的用戶偏好。實驗結果顯示，M-RNN-F的性能優於其他最先進的推薦演算法。此外，我們在現實世界數據集上進行實驗，以證明其實用性。

Matrix factorization (MF) technique has been widely utilized in recommendation systems due to the precise prediction of users’ interests. Prior MF-based methods adapt the overall rating to make the recommendation by extracting latent factors from users and items. However, in real applications, people’s preferences usually vary with time; the traditional MF-based methods could not properly capture the change of users’ interests. In this thesis, by incorporating the recurrent neural network (RNN) into MF, we developed a novel recommendation system, M-RNN-F, to effectively describe the preference evolution of users over time. Two learning models are proposed to capture the evolution pattern and predict the user preference in the future. The experimental results show that M-RNN-F performs better than other state-of-the-art recommendation algorithms. In addition, we conduct the experiments on real world dataset to demonstrate the practicability.

Abstract (Chinese)  I
Abstract  II
Table of Contents  IV
List of Figures  VI
List of Tables  VII
Chapter 1 Introduction	1

Chapter 2 Related Works	6
2.1	 Matrix Factorization	6
2.2	 Recommendation on MF	9

Chapter 3 Preliminary	12

Chapter 4 Proposed Recommendation System: M-RNN-F   13
4.1	 Feedback Matrix Transformation & Factorization 14
	Algorithm 1: Feedback Sequence Transformation   14
	Definition 1 (Feedback Matrix and Sequence)   15
	Definition 2 (Preference and Characteristic Matrices)   15
4.2	 Evolution Learning	17
	4.2.1  Dependent Learning	17
	4.2.2  Independent Learning	20
4.3	 Prediction and Recommendation	24

Chapter 5 Experiments	25
5.1	 Experiment Setup	26
5.2	 Analysis on Overall Performance	29
5.3	Comparing Model Performance on Precision and Recall	34
5.4	Discussion of LSTM Architecture and Activation Functions on Training Efficiency	42

Chapter 6 Conclusion	47

Reference	48


List of Figures

Fig. 1: The example rating matrix with preference evolution	3
Fig. 2: The architecture of M-RNN-F system	13
Fig. 3: The concept of dependent learning model	18
Fig. 4: The concept of independent learning model	21
Fig. 5: Four distinct results of a confusion matrix	34
Fig. 6: Precision on three models of MovieLens 1M Dataset	36
Fig. 7: Precision on three models of MovieLens 100k Dataset	37
Fig. 8: Recall on three models of MovieLens 1M Dataset	38
Fig. 9: Recall on three models of MovieLens 100k Dataset	39
Fig. 10: F1-score on three models of MovieLens 1M Dataset	40
Fig. 11: F1-score on three models of MovieLens 100k Dataset	41


List of Tables

Table 1: The MovieLens Datasets	26
Table 2: MAE@ d on the MovieLens 1M Dataset	29
Table 3: MAE@ d on the MovieLens 100k Dataset	30
Table 4: RMSE@ d on the MovieLens 1M Dataset	31
Table 5: RMSE@ d on the MovieLens 100k Dataset	31
Table 6: ACC@ d on the MovieLens 1M Dataset	32
Table 7: ACC@ d on the MovieLens 100k Dataset	33
Table 8: Comparison of MAE@ d on the MovieLens 1M Dataset	42
Table 9: Comparison of MAE@ d on the MovieLens 100k Dataset	43
Table 10: Comparison of RMSE@ d on the MovieLens 1M Dataset	44
Table 11: Comparison of RMSE@ d on the MovieLens 100k Dataset	44
Table 12: Comparison of ACC@ d on the MovieLens 1M Dataset	45
Table 13: Comparison of ACC@ d on the MovieLens 100k Dataset	46

[1]	M. Abdi, G. Okeyo and R. Mwangi, “Matrix Factorization Techniques for Context-Aware Collaborative Filtering Recommender Systems: A Survey,” Computer and Information Science, Vol. 11, No. 2, 2018.

[2]	F. CHUA, R. Oentaryo and E. LIM, “Modeling Temporal Adoptions Using Dynamic Matrix Factorization,” IEEE 13th International Conference on Data Mining (ICDM), pp. 91-100, 2013.

[3]	Y. Du, C. Xu and D. Tao, “Privileged Matrix Factorization for Collaborative Filtering,” The 26th International Joint Conference on Artificial Intelligence (IJCAI), pp. 1610-1616, 2017.

[4]	R. Gemulla, E. Nijkamp, P. Haas and Y. Sismanis, “Large-scale matrix factorization with distributed stochastic gradient descent,” The 17th ACM SIGKDD international conference on Knowledge discovery and data mining (SIGKDD), pp. 69-77, 2011.

[5]	J. He, X. Li, L. Liao, D. Song, and K. Cheung, “Inferring a personalized next point-of-interest recommendation model with latent behavior patterns,” The 13th AAAI Conference on Artificial Intelligence (AAAI), pp. 137-143, 2016.

[6]	X. He, H. Zhang, M. Kan and T. Chua, “Fast matrix factorization for online recommendation with implicit feedback,” The 39th International ACM Conference on Research and Development in Information Retrieval (SIGIR), pp. 549-558, 2016.

[7]	S. Huang, M. Xu, M. Xie, M. Sugiyama, G. Niu and S. Chen, “Active Feature Acquisition with Supervised Matrix Completion,” The 24th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (SIGKDD), pp. 1571-1579, 2018.

[8]	M. Jamali and M. Ester, “A Matrix Factorization Technique with Trust Propagation for Recommendation in Social Networks,” The 4th ACM Conference on Recommender Systems (RecSys), pp. 135-142, 2010.

[9]	J. Kawale, H. Bui, B. Kveton, L. Thanh and S. Chawla, “Efficient Thompson Sampling for Online Matrix-Factorization Recommendation,” The 28th Conference on Neural Information Processing Systems (NIPS), pp. 1297-1305, 2015.

[10]	Y. Koren, R. Bell, and C. Volinsky, “Matrix Factorization Techniques for Recommender Systems,” IEEE Computer, Vol. 42, pp. 30-37, 2009.

[11]	D. Liang, J. Altosaar, L. Charlin and D. Blei, “Factorization Meets the Item Embedding: Regularizing Matrix Factorization with Item Co-occurrence,” The 10th ACM Conference on Recommender Systems (RecSys), pp. 59-66, 2016.

[12]	C. Lin, L. Wang, K. Tsai, “Hybrid Real-Time Matrix Factorization for Implicit Feedback Recommendation Systems,” IEEE Access, Vol. 6, pp. 21369-21380, 2018.

[13]	X. Luo, M. Zhou, Y. Xia and Q. Zhu, “An Efficient Non-Negative Matrix-Factorization-Based Approach to Collaborative Filtering for Recommender Systems,” IEEE Transactions on Industrial Informatics, Vol. 10, Issue 2, pp. 1273-1284, 2014.

[14]	W. Ma, Y. Wu, M. Gong, C. Qin and S. Wang, “Local Probabilistic Matrix Factorization for Personal Recommendation,” The 13th International Conference on Computational Intelligence and Security (CIS), pp. 97-101, 2017.

[15]	H. Ma, H. Yang, M. Lyu and I. King, “SoRec: social recommendation using probabilistic matrix factorization,” The 17th ACM conference on Information and knowledge management (CIKM), pp. 931-940, 2008.

[16]	R. Mehta and K. Rana, “A review on matrix factorization techniques in recommender systems,” The 2nd International Conference on Communication Systems, Computing and IT Applications (CSCITA), pp. 269-274, 2017.

[17]	Q. Meng, H. Zhu, K. Xiao and H. Xiong, “Intelligent Salary Benchmarking for Talent Recruitment: A Holistic Matrix Factorization Approach,” IEEE 18th International Conference on Data Mining (ICDM), pp. 337-346, 2018.

[18]	N. Nghe, L. Drumond, T. Horváth, A. Nanopoulos, and L. Thieme, “Matrix and Tensor Factorization for predicting Student Performance,” The 3rd International Conference on Computer Supported Education (CSEDU), pp. 69-78, 2011.

[19]	H. Park, J. Jung and U. Kang, “A Comparative Study of Matrix Factorization and Random Walk with Restart in Recommender Systems,” IEEE International Conference on Big Data (IEEE BigData), pp. 756-765, 2017.

[20]	R. Salakhutdinov and A. Mnih, “Probabilistic Matrix Factorization,” ACM 20th International Conference on Neural Information Processing Systems (NIPS), pp. 1257-1264, 2007.

[21]	N. Sorkunlu, D. Luong and V. Chandola, “dynamicMF: A Matrix Factorization Approach to Monitor Resource Usage in High Performance Computing Systems,” IEEE International Conference on Big Data (IEEE BigData), pp. 1302-1307, 2018.

[22]	T. Tran, K. Lee, Y. Liao and D. Lee, “Regularizing Matrix Factorization with User and Item Embeddings for Recommendation,” The 27th ACM International Conference on Information and Knowledge Management (CIKM), pp. 687-696, 2018.

[23]	G. Trigeorgis, K. Bousmalis, S. Zafeiriou and B. Schuller, “A Deep Matrix Factorization Method for Learning Attribute Representations,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, Issue 3, pp. 417-429, 2017.

[24]	G. Winata, A. Madotto, J. Shin,and E. Barezi, “Low-Rank Matrix Factorization of LSTM as Effective Model Compression,” The 7th International Conference on Learning Representations (ICLR), 2019.

[25]	J. Tu, G. Yu, C. Domeniconi, J. Wang, G. Xiao and M. Guo, “Multi-Label Answer Aggregation based on Joint Matrix Factorization,” IEEE 18th International Conference on Data Mining (ICDM), pp. 517-526, 2018.

[26]	T. Wallace, C. Godwin, J. Thomson, L. Shepherd, A. Tjernlund, B. Bowman, E. Lanxner, K. Rizer, G. Carragher, C. Watson, and L. Liberman, “The Definitive Guide to Selling on Amazon,” BigCommerce, pp. 1-224, 2019.

[27]	C. Wang, Q. Liu, R. Wu, E. Chen, C. Liu, X. Huang and Z. Huang, “Confidence-Aware Matrix Factorization for Recommender Systems,” The 32nd AAAI Conference on Artificial Intelligence (AAAI), pp. 434-442, 2018.

[28]	Q. Wang, P. Tan and J. Zhou, “Imputing Structured Missing Values in Spatial Data with Clustered Adversarial Matrix Factorization,” IEEE 18th International Conference on Data Mining (ICDM), pp. 1284-1289, 2018.

[29]	Q. Wu and C. Pu, “Modeling and implementing collaborative editing systems with transactional techniques,” The 6th International ICST Conference on Collaborative Computing: Networking, Applications, Worksharing (CollaborateCom), pp. 1-10, 2010.

[30]	Z. Wu, H. Tian, X. Zhu, and S. Wang, “Optimization Matrix Factorization Recommendation Algorithm Based on Rating Centrality,” The 3rd International Conference on Data Mining and Big Data (DMBD), LNCS Vol. 10943, pp. 114-125, 2018.

[31]	L. Xiong, X. Chen, T. Huang, J. Schneider, and J. Carbonell, “Temporal Collaborative Filtering with Bayesian Probabilistic Tensor Factorization,” The 10th SIAM International Conference on Data Mining (SDM), pp. 211-222, 2010.

[32]	J. Yoo and S. Choi, “Probabilistic matrix tri-factorization,” The 34th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 1553-1556, 2009.

[33]	H. Yu, H. Huang, I. Dihillon and C. Lin, “A Unified Algorithm for One-Cass Structured Matrix Factorization with Side Information,” The 31st AAAI Conference on Artificial Intelligence (AAAI), pp. 2845-2851, 2017.

[34]	V. Yuvaraj and N. SivaKumar, “A Semi- Non-Negative Matrix Factorization and Principal Component Analysis Unified Framework for Data Clustering,” International Journal of Advanced Research in Science, Engineering and Technology (IJARSET), Vol. 5, Issue 1, 2018.

[35]	G. Zeng, H. Zhu, Q. Liu, P. Luo, E. Chen and T. Zhang, “Matrix Factorization with Scale-Invariant Parameters,” The 24th International Joint Conference on Artificial Intelligence (IJCAI), pp. 4017-4024, 2015.

[36]	J. Zhang and C. Chow, “CRATS: An LDA-Based Model for Jointly Mining Latent Communities, Regions, Activities, Topics, and Sentiments from Geosocial Network Data,” IEEE Transactions on Knowledge and Data Engineering (TKDE), Vol. 28, No. 11, pp. 2895-2909, 2016.

[37]	S. Zhao, M. Lyu, and I. King, “STELLAR: Spatial-Temporal Latent Ranking for Successive Point-of-Interest Recommendation,” The 13th AAAI Conference on Artificial Intelligence (AAAI), pp. 315-321, 2016.

[38]	MovieLens Datasets: collected by GroupLens Research: https://grouplens.org/datasets/movielens/