本篇論文提出利用新穎的3D卷積神經網路(3D-CNN, 3D Convolutional Neural Network)學習包含時間及空間的特徵，並將該類神經網路分類器與技術發展純熟的 Cascade架構結合。可以對於不同複雜度的個別區域(如前景與背景)，動態學習出一個至數個分類器，級聯成獨立相應的偵測系統。方法上利用將訓練樣本集合細分為典型和非典型兩個子集合，模擬正負樣本；並以增加不同比例高斯雜訊的方式擴充資料量、增加變化性，解決卷積神經網路較難應用於異常事件偵測領域的問題。使用常見的資料集來測試偵測系統之精確度與召回率，由實驗結果顯示我們所提出的方法有不錯的表現，經過後處理後能與其他先進的演算法匹敵。

In this paper we present a neural network (NN) architecture for abnormal events detection in a surveillance system. When training such system, it is a challenge that only normal samples are available for training. In addition, there are various contents inside a surveillance video frame, such like lawn, pedestrian walking area, trees, etc., and each has its own “typical/atypical” pattern. In solving the first problem, we propose a scheme to separate training data into “typical” and “atypical” that serve the roles of negative and positive training samples. In the second problem, we first divide a frame into several blocks. Then, for each block, an adaptive cascaded 3D-CNN classifier is trained. In this way, most of irrelevant blocks (normal) are discarded and only those abnormal or confusing normal blocks are kept for further computations. We evaluate our approach on popular dataset and show that our approach is competitive to state-of-the-art methods.

第一章　緒論	1
1.1	研究背景與目的	1
1.2	論文架構	4
第二章　相關文獻回顧	5
第三章　本文方法與網路模型	8
3.1	資料前處理 Data Preprocessing	8
3.2	訓練資料分為典型/非典型樣本	8
I.	經pretrained P3D model 提取特徵	8
II.	由Principal Components Analysis降維	9
III.	計算Mahalanobis Distance	10
IV.	透過Elbow Method分典型/非典型	11
3.3	3D 卷積神經網路(3D Convolutional Neural Network)	12
I.	網路架構 Net	13
II.	空洞卷積 Dilated Convolution	14
III.	損失函數 Loss Function	15
第四章　系統訓練及測試流程	17
4.1	資料擴增 Data Augmentation	17
4.2	訓練 Training	18
I.	Stop Criteria for Training a Classifier	18
4.3	測試 Testing	23
4.4	演算法 Algorithm	26
第五章	實驗結果	27
5.1	決定Typical Score Threshold	29
5.2	Typical Accuracy Threshold的必要性	30
5.3	False Negative Rate的必要性	31
5.4	其他相關文獻比較	33
第六章 結論	36
6.1	小結	36
6.2	未來精進方向	36
參考文獻	37
附錄:英文論文	41

圖目錄
圖1異常事件範例	1
圖2級聯分類器示意圖	3
圖3 場景影像作為3D-CNN-Classifier的輸入	8
圖4 馬氏距離計算標準差	10
圖5 Elbow Method應用取分界線	11
圖6 Dilated convolution實例	15
圖7 訓練資料擴增示意圖	17
圖8 非典型訓練樣本增加雜訊後示例	18
圖9 3D-CNN-Classifier訓練架構圖	20
圖10 Adaboost訓練示意圖	21
圖11 異常區域偵測實例	25
圖12 UCSD資料庫	27
圖13 偵測結果實例	28
圖14 區域ROC曲線比較	30
圖15 原始模型與增加準確率門檻值的比較	31
圖16 Original Model 與Consecutive Frame 的ROC比較	34

表目錄
表 1 本篇3D ConvNet詳細結構	14
表 2 以Ped1數據比較加上FNR Threshold條件前後的系統表現	32
表 3 原始系統與兩種後處理方式的Frame-level Confusion Matrix比較	33
表 4 與其他實驗數據比較	35

[1]	Yuan Yuan, Yachuang Feng, and Xiaoqiang Lu. Structured dictionary learning for abnormal event detection in crowded scenes. In Pattern Recognition, vol. 73, pp. 99-110, 2018.
[2]	Cewu Lu, Jianping Shi, and Jiaya Jia. Abnormal Event Detection at 150 FPS in MATLAB. In ICCV, pp. 2720-2727, 2013.
[3]	Mahdyar Ravanbakhsh, Moin Nabi, Enver Sangineto, Lucio Marcenaro, Carlo Regazzoni, and Nicu Sebe. Abnormal event detection in videos using generative adversarial nets. In ICIP, 2017.
[4]	Dan Xu, Elisa Ricci, Yan Yan, Jingkuan Song, and Nicu Sebe. Learning Deep Representations of Appearance and Motion for Anomalous Event Detection. In BMVC, 2015.
[5]	Radu Tudor Ionescu, Sorina Smeureanu, Marius Popescu, and Bogdan Alexe. Detecting Abnormal Events in Video Using Narrowed Normality Clusters. In IEEE Winter Conference on Applications of Computer Vision, 2018.
[6]	Raghavendra Chalapathy, Aditya Krishna Menon, and Sanjay Chawla. Anomaly Detection using One-Class Neural Networks”,in arXiv preprint arXiv:1802.06360, 2018.
[7]	Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning Spatiotemporal Features with 3D Convolutional Networks. In ICCV, 2015.
[8]	Zhaofan Qiu, Ting Yao, and Tao Mei. Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks. In ICCV, 2017.
[9]	Tian Wang & Hichem Snoussi. Detection of Abnormal Visual Events via Global Optical Flow Orientation Histogram. In TIFS, vol. 9, no. 6, pp. 988-998, June 2014.
[10]	A. Adam, E. Rivlin, I. Shimshoni, and D. Reinitz. Robust Real-Time Unusual Event Detection using Multiple Fixedlocation Monitors. In PAMI, March 2008.
[11]	J. Kim & K. Grauman. Observe Locally, Infer Globally: a Space-Time MRF for Detecting Abnormal Activities with Incremental Updates. In CVPR, 2009. 
[12]	R. Mehran, A. Oyama, and M. Shah. Abnormal Crowd Behavior Detection using Social Force Model. In CVPR, 2009.
[13]	Bosi Yu, Yazhou Liu, and Quansen Sun. A Content-Adaptively Sparse Reconstruction Method for Abnormal Events Detection With Low-Rank Property. In IEEE IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017.
[14]	Radu Tudor Ionescu, Sorina Smeureanu, Bogdan Alexe, and Marius Popescu. Unmasking The Abnormal Events in Video. In ICCV, 2017.
[15]	L. Kratz & K. Nishino. Anomaly Detection in Extremely Crowded Scenes Using Spatio-Temporal Motion Pattern Models. In CVPR, 2009.
[16]	N. Dalal & B. Triggs. Histograms of oriented gradients for human detection. In CVPR, June 2005.
[17]	Fisher Yu & Vladlen Koltun. Multi-Scale Context Aggregation by Dilated Convolutions. In CVPR, 2016.
[18]	Karen Simonyan & Andrew Zisserman. Very Deep Convolutional Networks For Large-Scale Image Recognition. In ICLR, 2015. 
[19]	Patrick Schlachter, Yiwen Liao and Bin YangDeep. One-Class Classification Using Intra-Class Splitting. In IEEE Data Science Workshop, 2019.
[20]	Poojan Oza & Vishal M. Patel. One-Class Convolutional Neural Network. In CVPR, 2019.
[21]	T.B. Moeslund, A. Hilton and V. Kru¨ger. A Survey of Advances in Vision-Based Human Motion Capture and Analysis. In Computer Vision and Image Understanding, 2006.
[22]	P.K. Turaga, R. Chellappa, V.S. Subrahmanian and O. Udrea. Machine Recognition of Human Activities: A Survey. In IEEE Transactions on Circuits and Systems for Video Technology, 2008.
[23]	O.P. Popoola and K. Wang. Video-Based Abnormal Human Behavior Recognition—A Review. In IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 2012.
[24]	Cui, X., Liu, Q., Gao, M., and Metaxas, D. N. Abnormal Detection Using Interaction Energy Potentials. In CVPR, June 2011.
[25]	Zhang, Y., Qin, L., Yao, H., and Huang, Q. Abnormal Crowd Behavior Detection Based on Social Attribute-Aware Force Model. In ICIP, September 2012.
[26]	Kai-Wen Cheng and Yie-Tarng Chen and Wen-Hsien Fang. Video Anomaly Detection and Localization Using Hierarchical Feature Representation and Gaussian Process Regression. In CVPR, 2015.
[27]	Cong, Y., Yuan, J., and Tang, Y. Video Anomaly Search in Crowded Scenes via Spatio-Temporal Motion Context. In Information Forensics and Security, IEEE Transactions on, 8(10), 2013.
[28]	Krizhevsky, A., Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, 2012.
[29]	Perera, Pramuditha, and Vishal M. Patel. Learning Deep Features for One-Class Classification. arXiv preprint arXiv:1801.05365, 2018.
[30]	Kiran, B. Ravi, Dilip Mathew Thomas, and Ranjith Parakkal. An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos. In Journal of Imaging, 2018.
[31]	S. Ji, W. Xu, M. Yang, and K. Yu. 3D Convolutional Neural Networks for Human Action Recognition. In IEEE TPAMI, 2013.
[32]	V. Jain, B. Bollmann, M. Richardson, D. Berger, M. Helmstaedter, K. Briggman, W. Denk, J. Bowden, J. Mendenhall, W. Abraham, K. Harris, N. Kasthuri, K. Hayworth, R. Schalek, J. Tapia, J. Lichtman, and H. Seung. Boundary Learning By Optimization with Topological Constraints. In CVPR, 2010.
[33]	Wenqing Chu , Hongyang Xue , Chengwei Yao , and Deng Cai. Sparse Coding Guided Spatiotemporal Feature Learning for Abnormal Event Detection in Large Videos. In IEEE Transactions on Multimedia, 2019.
[34]	Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun. Deep Residual Learning for Image Recognition. In CVPR, 2015.
[35]	USCD database http://www.svcl.ucsd.edu/projects/anomaly/dataset.htm
[36]	V. Saligrama and Z. Chen. Video Anomaly Detection Based on Local Statistical Aggregates. In Proc. IEEE Conf. Comput. Vision Pattern Recognit., 2012.
[37]	V. Mahadevan, W. Li, V. Bhalodia, and N. Vasconcelos. Anomaly Detection in Crowded Scenes. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2010.
[38]	M. Hasan, J. Choi, J. Neumann, A. K. Roy-Chowdhury, and L. S. Davis. Learning Temporal Regularity in Video Sequences. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016.