目前常被應用的深度學習方法，無論是MLP, CNN, RNN等由學習觀點來看大都以最大概似估計法 (MLE) 來最小化資料與模型之間的差異。其基本假設在於相信世界的本質是穩定的，所有現象背後都有一個穩定的母體，所以我們只要透過來自母體大量隨機且可重複實驗的事件，就可以透過期望值估算各種統計量。然而實際上這種假設不一定正確。在資料無法大量取得的狀況下，產生的模型也較容易出錯。近年來貝氏深度學習方法愈來愈受到重視，此類方法旨在估計權重的後驗分佈而非估計值。 而後驗本身可以用於量化網絡模型的不確定性，進而可避免深度學習領綠中常見的過度配適問題。
本研究將利用變分推論方法，對機器學習領域中常使用的開源網路庫keras修改程式碼，建置貝氏循環類神經網路層來處理飛行中操作不確定性的問題。並利用此一新開發的技術來偵測飛安事件中重落地發生的可能性。解決分析重落地事件肇因效率不佳的問題。

Most of the deep learning methods (MLP, CNN, RNN, etc.) use the maximum likelihood to estimate the weights of network models. The basic hypothesis is that the nature of the world is stable, and there is a stable matrix behind all phenomena. So we can estimate various statistics through expected values as long as we use a large number of random and repeatable experiments from the matrix. In fact, this assumption is not necessarily correct. In situations where a large amount of data cannot be obtained, the resulting model is also prone to errors. In recent years, Bayesian deep learning methods have received more and more attention. Such methods are designed to estimate the posterior distribution of weights rather than estimates. The posterior itself can be used to quantify the uncertainty of the network model, thereby avoiding the common overfitting problem in deep learning models.
This project will use variational inference methods to modify the code of the open source library keras commonly used in deep learning, and build a Bayesian recurrent neural network layer to handle uncertainty of operation problems in flying. We will use this newly developed technology to detect hard landing of aviation event occurrence possibility. Solve the problem of poor efficiency of analysing hard landing causes

第一章 緒論	1
1.1研究背景與動機	1
1.2研究目的	4
第二章 文獻探討	5
2.1飛安事件	5
2.1.1飛行參數超限偵測	5
2.1.2重落地事件預防	6
2.2貝氏學習	9
2.3變分推論	11
2.4變分推論與神經網路	16
第三章 研究方法	20
3.1模型框架	21
3.2重落地事件資料集	21
3.2.1 重落地肇因分類	21
3.2.2 影響重落地之參數	23
3.3模型建置	24
3.3.1類別變數初始化	24
3.3.2改寫長短期記憶網路權重	26
3.3.3計算模型損失函數	28
3.4模型評估方式	30
第四章 實驗與結果	31
4.1模型訓練	31
4.2實驗結果	33
4.2.1傳統長短期記憶網路	34
4.2.2貝氏長短期記憶網路—以一般高斯建模	36
4.2.3貝氏長短期記憶網路—以複合高斯建模	40
4.2.4重落地肇因預測	44
第五章 結論與展望	46
5.1結論	46
5.2未來展望	49
參考文獻	50
附錄—貝氏長短期記憶網路模型程式碼	56

表目錄
表 1 重落地事件分類	22
表 2 平飄推力中不同分類之個數	23
表 3 影養飛行駕駛之因素	24
表 4 模型類別變數初始化整理	25
表 5 改寫輸入權重所需向量整理表	28
表 6 B777資料集之模型建立設定	32
表 7 B777模型編譯與評估參數設定	32
表 8 B777模型訓練參數設定	33
表 9 LSTM不同參數之預測準確率與KAPPA值	35
表 10 一般高斯建模BAYSELSTM不同參數平均準確率與最佳KAPPA值	38
表 11 LSTM與BAYSELSTM藉由一般高斯建模成效比較	39
表 12 複合高斯建模BAYSELSTM不同參數平均準確率與最佳KAPPA值	42
表 13 LSTM與BAYSELSTM藉由複合高斯建模成效比較	43
表 14 LSTM預測錯誤前五筆在BAYSELSTM之不同分類機率	44
表 15 LSTM預測正確前五筆在BAYSELSTM之不同分類機率	45

圖目錄
圖 1 變分推論求解示意圖	13
圖 2 研究方法架構圖	20
圖 3 LSTMCELL架構圖	26
圖 4 輸入LSTMCELL變數示意圖	27
圖 5傳統長短期記憶網路訓練過程準確率變化圖	34
圖 6貝氏長短期記憶網路(一般高斯)訓練過程準確率變化圖	36
圖 7貝氏長短期記憶網路(複合高斯)訓練過程準確率變化圖	40

Benbassat, D., and Abramson, C. I. (2002). Landing flare accident reports and pilot perception analysis. The International Journal of Aviation Psychology, 12(2),137- 152.
Benbassat, D., Abramson, C. I., and Williams, K. W.(2002).Comparative approach to pilot error and effective landing flare instructions. International Journal of Comparative Psychology, 15(2).
Bernardo, J. M., and Smith, A. F. (2009).Bayesian theory(Vol. 405). John Wiley and Sons.
Bishop, C. M. (1997). Bayesian neural networks. Journal of the Brazilian Computer Society, 4(1).
Blei, David, Rajesh Ranganath, and Shakir Mohamed. "Variational inference: Foundations and modern methods." NIPS Tutorial (2016).
Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. (2017). Variational inference: A review for statisticians.Journal of the American statistical Association, 112(518), 859-877.
Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D. (2015, June). Weight uncertainty in neural network. In International Conference on Machine Learning (pp. 1613-1622). PMLR.
Buntine, W. L. (1991). Bayesian backpropagation. Complex systems, 5, 603-643.
Chen, C., Lin, X., and Terejanu, G. (2018, August). An approximate bayesian long short-term memory algorithm for outlier detection. In 2018 24th International Conference on Pattern Recognition (ICPR) (pp. 201-206). IEEE.
Chung, J., Kastner, K., Dinh, L., Goel, K., Courville, A., and Bengio, Y. (2015). A recurrent latent variable model for sequential data. arXiv preprint arXiv:1506.02216.
de Freitas, J. F. G. (2003). Bayesian methods for neural networks (Doctoral dissertation, University of Cambridge).
Efron, B. (2013). Bayes' theorem in the 21st century. Science, 340(6137), 1177-1178.
Fabius, O., and Van Amersfoort, J. R. (2014). Variational recurrent auto-encoders. arXiv preprint arXiv:1412.6581.
Fortunato, M., Blundell, C., and Vinyals, O. (2017). Bayesian recurrent neural networks.arXiv preprint arXiv:1704.02798.
Fox, C. W., and Roberts, S. J. (2012). A tutorial on variational Bayesian inference. Artificial intelligence review, 38(2), 85-95.
Gal, Y., & Ghahramani, Z. (2016). A theoretically grounded application of dropout in recurrent neural networks. Advances in neural information processing systems, 29, 1019-1027.
Gal, Y., & Ghahramani, Z. (2016, June). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on 
machine learning (pp. 1050-1059). PMLR.
Graves, A. (2011). Practical variational inference for neural networks. In Advances in neural information processing systems (pp. 2348-2356).
Graves, A., and Jaitly, N. (2014, June). Towards end-to-end speech recognition with recurrent neural networks. In International conference on machine learning (pp. 1764-1772). PMLR.
Hinton, G. E., and Van Camp, D. (1993, August). Keeping the neural networks simple by minimizing the description length of the weights. In Proceedings of the sixth annual conference on Computational learning theory (pp. 5-13)
Huan-Jyh Shyur, Chi-Bin Cheng, Yulin Hsiao (2020) Using Deep Learning Approach in Flight Exceedance Event Analysis.
Hoffman, M. D., Blei, D. M., Wang, C., and Paisley, J.(2013) . Stochastic variational inference. Journal of Machine Learning Research, 14(5).
International Air Transport Association, Safety Report 2020 (2021)
Ibrahim, M. R., Haworth, J., Lipani, A., Aslam, N.,Cheng, T., and Christie, N. (2021). Variational-LSTM Autoencoder to forecast the spread of coronavirus across the globe. PloS one, 16(1), e0246120.
Janic, M. (2000). An assessment of risk and safety in civil aviation. Journal of Air Transport Management, 6(1), 43-50.
Jasra, S., Gauci, J., Muscat, A., Valentino, G., Zammit-Mangion, D., and Camilleri, R. (2018). Literature review of machine learning techniques to analyse flight data.
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., and Saul, L. K. (1999). An Introduction to variational methods for graphical models. Machine learning, 37(2),183-233
Joyce, J. (2003). Bayes' theorem.
Kingma, D. P., and Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114.
Kullback, S., and Leibler, R. A. (1951). On information and sufficiency. The annals of mathematical statistics, 22(1), 79-86.
Landings, A. H. (2004). Flight Safety
Lee, W. K. (2006). Risk assessment modeling in aviation safety management. Journal of Air Transport Management, 12(5), 267-273.
MacKay, D. J. (1992). A practical Bayesian framework for backpropagation networks. Neural computation, 4(3), 448-472.
MacKay, D. J. (1995). Bayesian methods for neural networks: Theory and applications.
Mullachery, V., Khera, A., and Husain, A. (2018). Bayesian neural networks. arXiv preprint arXiv:1801.07710.
Nanduri, A., and Sherry, L. (2016, April). Anomaly detection in aircraft data using Recurrent Neural Networks (RNN). In 2016 Integrated Communications Navigation and Surveillance (ICNS) (pp. 5C2-1). Ieee.
Neal, R. M. (2012). Bayesian learning for neural networks (Vol. 118). Springer Science & Business Media.
Oehling, J., and Barry, D. J. (2019). Using machine learning methods in airline flight data monitoring to generate new operational safety knowledge from existing data. Safety science, 114, 89-104.
Padmanaban, S., and Mahendran, S. M. E. (2015). Enhancing flight data monitoring and analysis can increase flight safety. Journal of Aeronautics and Aerospace Engineering, 4(2), 1000143.
Park, D., Hoshi, Y., and Kemp, C. C. (2018). A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder. IEEE Robotics and Automation.
Peng, W., Ye, Z. S., & Chen, N. (2019). Bayesian deep-learning-based health prognostics toward prognostics uncertainty. IEEE Transactions on Industrial Electronics, 67(3), 2283-2293.
Qi, J., Liu, X., and Tejedor, J. (2020). Variational Inference-Based Dropout in Recurrent Neural Networks for Slot Filling in Spoken Language Understanding. arXiv preprint arXiv:2009.01003.
Shyur, H. J. (2008). A quantitative model for aviation safety risk assessment. Computers &Industrial Engineering, 54(1), 34-44.
Sun, R., and Li, C. (2021, April). A Risk Prediction Model of Hard Landing Based on Random Forest Algorithm. In E3S Web of Conferences (Vol. 257, p. 02080).
Tong, C., Yin, X., Li, J., Zhu, T., Lv, R., Sun, L., and Rodrigues, J. J. (2018). An innovative deep architecture for aircraft hard landing prediction based on time-series sensor data. Applied Soft Computing, 73, 344-349.
Uehara, A. F., and Niedermeier, D. (2015). Influence of coupled sidesticks on the pilot monitoring's awareness during flare. In AIAA Modeling and Simulation Technologies Conference (p. 2338).
van der Westhuizen, J., and Lasenby, J. (2017). Bayesian LSTMs in medicine. arXiv preprint arXiv:1706.01242.
Wainwright, M. J., and Jordan, M. I. (2008). Graphical models, exponential families,and variational inference. Now Publishers Inc.
Wang, H., and Yeung, D. Y. (2020). A survey on Bayesian deep learning. ACM Computing Surveys (CSUR), 53(5), 1-37.
Wang, L., Wu, C., and Sun, R. (2013, July). Pilot operating characteristics analysis of long landing based on flight QAR data. In International Conference on Engineering Psychology and Cognitive Ergonomics (pp. 157-166). Springer, Berlin, Heidelberg.
Wang, L., Wu, C., and Sun, R. (2014). An analysis of flight Quick Access Recorder (QAR) data and its applications in preventing landing incidents. Reliability Engineering & System Safety, 127, 86-96.
Wang, L., Wu, C., Sun, R., and Cui, Z. (2014, June). An analysis of hard landing incidents based on flight QAR data. In International Conference on Engineering Psychology and Cognitive Ergonomics (pp. 398-406). Springer, Cham.
Wang, L., Ren, Y., and Wu, C. (2018). Effects of flare operation on landing safety: A study based on ANOVA of real flight data. Safety science, 102, 14-25.
Yang, Y., Li, W., Gulliver, T. A., & Li, S. (2019). Bayesian deep learning-based probabilistic load forecasting in smart grids. IEEE Transactions on Industrial Informatics, 16(7), 4703-4713.
Zhang, C., Bütepage, J., Kjellström, H., and Mandt, S. (2018). Advances in variational inference. IEEE transactions on pattern analysis and machine intelligence, 41(8), 2008-2026.
Zhang, H., and Zhu, T. (2018, September). Aircraft Hard Landing Prediction Using LSTM Neural Network. In Proceedings of the 2nd International Symposium on Computer Science and Intelligent Control (pp. 1-5).
Zhang, X., Fang, F., & Wang, J. (2020). Probabilistic solar irradiation forecasting based on variational Bayesian inference with secure federated learning. IEEE Transactions on Industrial Informatics