Top-N興趣順序推薦已成為人們日常生活的常用服務。該服務的目標是對過去的登記行動序列進行建模，以預測用戶在可預見的未來可能訪問的排名Top-N的位置。但是，由於這些數據的複雜性，應仔細挖掘用戶的順序模式，以便發現最近去的地點對下一步行動的影響。這一發現代表了用戶的規律性，通常包含用戶的長期興趣或一般偏好以及用戶的短期興趣或順序模式。此外，流行的POI模式應該包含在推理過程中，其中可能會在用戶的順序模式中跳過一些步驟。為了滿足這一要求，在本文中，我們提出了一種新的雙LSTM，稱為UP-LSTM，作為解決用戶短期和長期利益以及流行POI模式一致性的解決方案。 UP-LSTM包括U-LSTM和P-LSTM模型的同時學習。通過融合雙LSTM與不同權重的影響來推斷最終的Top-N預測。在具有各種度量的兩個移動性數據集上進行實驗。結果表明，與順序POI推薦任務中的其他基線相比，UP-LSTM模型具有優越的性能。這表明UP-LSTM提供了一種通用且靈活的網絡結構，用於分析複雜用戶移動下的近期和遠程推斷。

Top-N Point Of Interest Sequential recommendation has become a common service for people everyday life. The goal of this service is to model the sequence of check-in actions in the past in order to predict top-N ranked locations that a user is likely to visit within the foreseeable future. However, due to the complex nature of these data, users’ sequential patterns should be mined carefully in order to discover the impact of recent locations on the next move. This finding represents users regularity that normally contains both users’ long-term interest or general preference and users’ short-term interest or sequential patterns. In addition, the popular POI patterns should be included in the inferencing process where there might be a few step skipping in the users’ sequential patterns. To address this requirement, in this thesis, we propose a novel dual LSTM called UP-LSTM as a solution to cope with the users’ short-and -long term interests and the uniformity of popular POI patterns. UP-LSTM consists of the simultaneous learning of U-LSTM and P-LSTM models. The final Top-N prediction is inferenced by fusing the impact of both dual LSTM with a different weight. The experiments are conducted on the two mobility datasets with various metrics. The results have shown that UP-LSTM model has a superior performance compared with other baselines in a sequential POI recommendation task. This indicates that UP-LSTM provides a general and flexible network structure for analyzing both  a recent and distant inferences under the complicated users’ movement.

Table of Contents
Chinese Abstract	I
Abstract	II
Table of Contents 	IV
List of Figures	V
List of Tables	VI
Chapter 1 Introduction	1
Chapter 2 Related Work	7
2.1	Conventional Recommendation approaches	7
2.2	Deep Learning Recommendation approaches	10
Chapter 3	12
Preliminaries	12
Chapter 4	13
4.1 Feature Extraction and Embedding	13
    4.1.1 POI representation learning	13
    4.1.2 Check-in popularity pattern learning	15
    4.1.3 Embedding Look-up:	16
4.2 UP-LSTM model learning and Training	17
    4.2.1 U-LSTM:	18
    4.2.2 P-LSTM:	19
    4.2.3 U-LSTM and P-LSTM model training	20
    4.2.4 UP-LSTM Recommendation module	20
Chapter 5 Experiments	22
5.1	Experiment Settings	22
5.2	Analysis on Overall Performance	25
5.3    Comparing Model Performance on Precision and Recall	27
5.4    The Effect of Patterns on Recommendation	27
5.5    The Effect of Dimension on The Recommendation Model	30
Chapter 6 Conclusions	31
Reference	32



List of Figures

Fig. 1 : The Motivation of our proposed UP-LSTM model	2
Fig. 2 : The architecture of UP-LSTM	13
Fig. 3 : The concept of POI representation learning	14
Fig. 4 : The detail of UP-LSTM Model Training and Testing	17

List of Tables

Table 1: Brightkite datasets	22
Table 2: Preprocessed Brightkite datasets	23
Table 3: ACC@N Performance of different models	25
Table 4: The comparison of precision	26
Table 5: The comparison of recall	26
Table 6: The comparison of f1-measure 	26
Table 7: The error of different α in dimension 30 	28
Table 8: The error of different α in dimension 40	29
Table 9: Comparison between different dimensions 	30

[1]	D. Agarwal and B.C. Chen, “Regression-based latent factor models,” The 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 19-28, 2009.
[2]	R. Agrawal and R. Srikant, “Mining sequential patterns,” In International Conference on Data Engineering, pp. 3-14, 1995.
[3]	M. Chen, Y. Liu, and X. Yu, “NLPMM: A Next Location Predictor with Markov Modeling,” Advances in Knowledge Discovery and Data Mining Lecture Notes in Computer Science, pp. 186–197, 2014.
[4]	C. Cheng, H. Yang, I. King, and M. R. Lyu. “Fused matrix factorization with geographical and social influence in location-based social networks,” The 26th AAAI Conference on Artificial Intelligence, pp. 17-23, 2012.
[5]	C. Cheng, H. Yang, M. R. Lyu, and I. King, “Where You Like to Go Next: Successive Point-of-Interest Recommendation,” The 13th International Joint Conference on Artificial Intelligence, pp.2605-2611, 2013.
[6]	N. Du, H. Dai, R. Trivedi, U. Upadhyay, M. Gomez-Rodriguez, and L. Song, “Recurrent Marked Temporal Point Processes,” The 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1555-1564, 2016.
[7]	S. Feng, X. Li, Y. Zeng, G. Cong, Y. M. Chee, and Q. Yuan, “Personalized ranking metric embedding for next new poi recommendation,” The 25th International Joint Conference on Artificial Intelligence, 2015.
[8]	S. Gambs, M. Killijian, and M. Cortez, “Next place prediction using mobility Markov chains,” The First Workshop on Measurement, Privacy, and Mobility, NO. 3, pp. 1-6, 2012.
[9]	H. Gao, J. Tang, X. Hu, and H. Liu, “Exploring temporal effects for location recommendation on location-based social networks,” The 13th ACM Conference on Recommender Systems, pp. 93-100, 2013.
[10]	H. Gao, J. Tang, X. Hu, and H. Liu, “Modeling temporal effects of human mobile behavior on location-based social networks,” The 22nd ACM international conference on Conference on information & knowledge management, pp. 1673-1678, 2013.
[11]	J. Han, J. Pei, and M. Kamber, “Data mining: concepts and techniques,” Morgan Kaufmann Publishers Inc, 2011.
[12]	J. Han, J. Pei, Y. Yin, and R. Mao, “Mining Frequent Patterns without Candidate Generation,” Data Mining and Knowledge Discovery, vol.8, pp. 53-87, 2004.
[13]	J. He, X. Li, L. Liao, D. Song, and K. Cheung, “Inferring a personalized next point-of-interest recommendation model with latent behavior patterns,” The 13th AAAI Conference on Artificial Intelligence, pp. 137-143, 2016.
[14]	R. He and J. McAuley, “Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation,” In International Conference on Data Mining, 2016.
[15]	Y. Koren, “Factorization meets the neighborhood: a multifaceted collaborative filtering model,” The 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 426-434, 2008.
[16]	Y. Koren, R. Bell, and C. Volinsky, “Matrix factorization techniques for recommender systems,” IEEE Computer, NO. 8, pp. 30-37, 2009.
[17]	D. R. Liu, C. H. Lai, and W. J. Lee, “A hybrid of sequential rules and collaborative filtering for product recommendation,” Information Sciences, VOL. 179, NO. 20, pp. 3505-3519, 2009.
[18]	X. Liu, Y. Liu, and X. Li, “Exploring the context of locations for personalized location recommendations,” The 16th International Joint Conference on Artificial Intelligence, 2016.
[19]	Q. Liu, S. Wu, L. Wang, and T. Tan, “Predicting the next location: a recurrent model with spatial and temporal contexts,” The 13th AAAI Conference on Artificial Intelligence, pp.194-200, 2016.
[20]	J. Manotumruksa, C. Macdonald, and I. Ounis, “A Contextual Attention Recurrent Architecture for Context-Aware Venue Recommendation,” The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 555-564, 2018.
[21]	J. McInerney, J. Zheng, A. Rogers, and N. R. Jennings, "Modelling heterogeneous location habits in human populations for location prediction under data sparsity," The 2013 ACM international joint conference on Pervasive and ubiquitous computing, pp. 469-478, 2013.
[22]	R. Mehta and K. Rana, “A review on matrix factorization techniques in recommender systems,” The 2nd International Conference on Communication Systems, Computing and IT Applications, pp. 269-274, 2017.
[23]	X. Meng, S. Wang, Waqas Ahmad, Y. Chen, “A sequential multi-type POI recommendation service based on location,” International Conference on Identification, Information and Knowledge in the Internet of Things, 2016.
[24]	R. Pan, Y. Zhou, B. Cao, N. N. Liu, R. Lukose, M. Scholz, and Q. Yang, “One-class collaborative filtering,” In International Conference on Data Mining, pp. 502-511, 2008.
[25]	S. Rendle, C. Freudenthaler, and L. S. Thieme, “Factorizing personalized markov chains for next-basket recommendation,” In International Conference on World Wide Web, pp. 811-820, 2010.
[26]	R. Salakhutdinov and A. Mnih, “Bayesian probabilistic matrix factorization using Markov chain Monte Carlo,” The 25th international conference on Machine learning, pp. 880-887, 2008.
[27]	R. Salakhutdinov and A. Mnih, “Probabilistic matrix factorization,” Advances in Neural Information Processing Systems, VOL. 20, pp.1257-1264, 2008.
[28]	B. Sarwar, G. Karypis, J. Konstan, and John Riedl, “Item-based collaborative ltering recommendation algorithms,” In International Conference on World Wide Web, pp. 285-295, 2001.
[29]	S. Sedhain, A. K. Menon, S. Sanner, and L. Xie, “Autorec: Autoencoders meet collaborative filtering,” In International Conference on World Wide Web, pp. 111-112, 2015.
[30]	V. Sridhar, “Unsupervised Text Normalization Using Distributed Representations of Words and Phrases,” The 1st Workshop on Vector Space Modeling for Natural Language Processing, pp. 8-16, 2015.
[31]	P. Wang, J. Guo, Y. Lan, J. Xu, S. Wan, and X. Cheng, “Learning hierarchical representation model for next basket recommendation,” In International ACM SIGIR conference on Research and Development in Information Retrieval, pp. 403-412, 2015.
[32]	F. Wang, Y. Qu, L. Zheng, C. T. Lu, and P. S. Yu, “Deep and Broad Learning on Content-aware POI Recommendation,” IEEE 3rd International Conference on Collaboration and Internet Computing, 2017.
[33]	Y. Wang, N. J. Yuan, D. Lian, L. Xu, X. Xie, E. Chen, and Y. Rui, “Regularity and conformity: Location prediction using heterogeneous mobility data,” The 21th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1275-1284, 2015.
[34]	Y. Wu, C. DuBois, A. X. Zheng, and M. Ester, “Collaborative denoising auto-encoders for top-n recommender systems,” In International Conference on Web Search and Data Mining, pp. 153-162, 2016.
[35]	Q. Wu and C. Pu, “Modeling and implementing collaborative editing systems with transactional techniques,” The 6th International ICST Conference on Collaborative Computing: Networking, Applications, Worksharing, pp. 1-10, 2010.
[36]	L. Xiong, X. Chen, T. Huang, J. Schneider, and J. Carbonell, “Temporal Collaborative Filtering with Bayesian Probabilistic Tensor Factorization,” The 2010 SIAM International Conference on Data Mining, pp. 211-222, 2010.
[37]	D. Yao, C. Zhang, J. Huang, and J. Bi, “SERM: A Recurrent Model for Next Location Prediction in Semantic Trajectories,” The 2017 ACM on Conference on Information and Knowledge Management, pp. 2411-2414, 2017.
[38]	J. Yoo and S. Choi, “Probabilistic matrix tri-factorization,” The 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1553-1556, 2009.
[39]	Q. Yuan, G. Cong, Z. Ma, A. Sun, and N. M. Thalmann, “Time-aware point-of-interest recommendation,” In ACM SIGIR, pp. 363-372, 2013.
[40]	Q. Yuan, G. Cong, Z. Ma, A. Sun, and N. M. Thalmann, “Who, where, when and what: discover spatiotemporal topics for twitter users,” The 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 605-613,2013.
[41]	J. Zeng, Y. Li, “A Point-of-Interest Recommendation Method Using Location Similarity,” The 6th IIAI International Congress on Advanced Applied Informatics, 2017.
[42]	J. Zhang and C. Chow, “Spatiotemporal Sequential Influence Modeling for Location Recommendations,” ACM Transactions on Intelligent Systems and Technology, VOL. 7, NO. 1, pp. 1-25, 2015.
[43]	H. Zhang, Y. Yang, H. Luan, S. Yang, and T. Chua, “Start from Scratch: Towards Automatically Identifying, Modeling, and Naming Visual Attributes,” The ACM International Conference on Multimedia, pp. 187-196, 2014.
[44]	S. Zhao, M. Lyu, and I. King, “Geo-Teaser: Geo-Temporal Sequential Embedding Rank for POI Recommendation,” SpringerBriefs in Computer Science Point-of-Interest Recommendation in Location-Based Social Networks, pp. 57-78, 2018.
[45]	L. Zheng, V. Noroozi, and P. S. Yu. “Joint Deep Modeling of Users and Items Using Reviews for Recommendation,” In International Conference on Web Search and Data Mining, pp. 425-434, 2017.