書法是東方民族的一個傳統文化，由於現代紙筆等傳統工具越來越少需求，在現今的環境下書法變成是一種文字藝術，而不再是作為書寫的工具。本研究將建立一個空中書寫系統，透過深度相機獲取即時深度影像搭配 MediaPipe Hands 來完成空間中的書法揮毫。本研究提出具門檻值之局部加權迴歸散佈平滑法來偵測影像中由深度相機偵測到的異常筆畫資訊，並排除這些錯誤的資訊，解決深度相機的破洞或偽影問題導致書法系統的錯誤筆觸，門檻值之設定透過測量拇指指尖與食指指尖開合之最大速度以及拇指指尖與食指指尖全開之距離決定。透過局部加權迴歸散佈平滑法加上門檻值之設定排除由於深度相機獲取深度資訊不完整導致書法系統的錯誤。

Calligraphy is a traditional culture of the Eastern peoples. As traditional tools such as modern paper and pen are less and less in demand, in today's environment calligraphy has become an art of writing, rather than a writing tool. This research will build a mid-air writing system that uses depth cameras to acquire real-time depth images and use MediaPipe Hands to complete calligraphy strokes in space. This study proposes a modification of locally weighted scatterplot smoothing with threshold to detect abnormal stroke information detected by the depth camera in the image, and eliminate these erroneous information to solve the problem of holes or artifacts in the depth camera causing erroneous strokes in the calligraphy system, the threshold value is set by measuring the maximum opening and closing speed of the thumb tip and the index finger tip and the distance between the thumb tip and the index finger tip when they are fully opened. Through locally weighted scatterplot smoothing and threshold setting, errors in the calligraphy system caused by incomplete depth information obtained by the depth camera are eliminated.

目錄
目錄	1
圖目錄	3
表目錄	4
緒論	5
1.1 研究背景與動機	5
1.2 研究目的	6
1.3 論文架構	7
第二章 文獻探討	8
2.1 影像辨識	8
2.1.1 MediaPipe系統介紹	8
2.1.2 RealSense深度相機及雙目立體視覺	9
2.2繪圖	11
2.2.1接觸式繪圖	12
2.2.2非接觸式繪圖	12
2.3局部加權迴歸散佈平滑法	13
第三章 電腦視覺為基礎之空中書法	18
3.1空中書法系統運作流程	18
3.1.1影像捕捉與預處理	18
3.1.2三維距離計算	19
3.1.3資料前處理	19
3.1.4影像輸出與展示	20
3.1.5空中書法系統處理流程	20
3.2 TLOWESS	21
3.3門檻值設計	22
3.4 LSTM	24
第四章 實驗結果	26
4.1實驗環境	26
4.2排除異常數值優化筆跡	26
4.2.1不同迭帶次數的LOWESS比較	27
4.2.2 TLOWESS與其他迴歸方式之比較	30
4.3 TLOWESS與LSTM比較	31
第五章 結論與未來討論方向	33
參考文獻	34
 
圖目錄
圖 1、Medipipe hands的手部關節點位置[2]	9
圖 2、Intel® RealSense™ D455	10
圖 3、雙目測距原理 [1]	11
圖 4、影像展示	13
圖 5、迴歸殘差	16
圖 6、多種方法的迴歸曲線比較	16
圖 7、LOWESS無法有效排除所有異常數值	17
圖 8、空中書法系統運作流程圖	21
圖 9、測量拇指指尖與食指指尖支開合速度	24
圖 10、TLOWESS偵測與修正	24
圖 11、LSTM訓練模型架構[18]	25
圖 12、書法系統的筆劃粗細程度為4號跟8號的連線線段	26
圖 13、不同迭帶次數的殘差圖比較1	27
圖 14、不同迭帶次數的殘差圖比較2	28
圖 15、不同迭帶次數的殘差圖比較3	29
圖 16、不同迴歸方式優化之結果以【少】為例	30
圖 17、不同迴歸方式優化之結果以【大】為例	31
圖 18、TLOWESS與LSTM進行筆畫修正之結果	32
表目錄
表 1、圖13之MAE評分	28
表 2、圖14之MAE評分	29
表 3、圖15之MAE評分	30

參考文獻
[1]	N. K. Lu, N. X. Wang, N. Z. Wang, and N. L. Wang, “Binocular Stereo Vision based on OpenCV,” IET International Conference on Smart and Sustainable City (ICSSC 2011), Jan. 2011.
[2]	V. Kriznar, M. Leskovsek, and B. Batagelj, “Use of Computer Vision Based Hand Tracking in Educational Environments,” 2021 44th International Convention on Information, Communication and Electronic Technology (MIPRO), Sep. 2021.
[3]	F. Zhang, V. Bazarevsky, A. Vakunov, A. Tkachenka, G. Sung, C.L. Chang, M. Grundmann, "MediaPipe hands: On-device real-time hand tracking." arXiv.org, Jun. 2020.
[4]	W.M. Yang and W. Choi, “Verification of Noise Reduction by applying a Smoothing Algorithm in the MediaPipe Hand Tracking System,” Journal of Digital Contents Society, vol. 25, no. 5, pp. 1217–1224, 2024.
[5]	S. Wang and X. Wang, “Feature pyramid-based convolutional neural network image inpainting,” Signal Image and Video Processing, vol. 18, no. 1, pp. 437–443, 2023.
[6]	Y. Li, Q. Fan, H. Huang, Z. Han, and Q. Gu, “A modified YOLOV8 detection network for UAV aerial image recognition,” Drones, vol. 7, no. 5, pp. 304, 2023.
[7]	V.T. Pham, T.L. Le, T.-H. Tran, and T. P. Nguyen, “Hand detection and segmentation using multimodal information from Kinect,” 2020 International Conference on Multimedia Analysis and Pattern Recognition (MAPR), Oct. 2020.
[8]	V. Pterneas, Mastering the Microsoft Kinect. Apr. 2022.
[9]	Z. Jiang, L. Seo, and C. Roh, “Study of KINECT based 3D Holographic and Gesture,” Journal of Digital Contents Society, vol. 14, no. 4, pp. 411–417, 2013.
[10]	L. Keselman, J. I. Woodfill, A. Grunnet-Jepsen, and A. Bhowmik, “Intel(R) RealSense(TM) Stereoscopic Depth Cameras,” 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Jul. 2017.
[11]	Z. Zhang, “Microsoft Kinect Sensor and its effect,” IEEE MultiMedia, Feb. 2012.
[12]	M. Servi, A. Profili, R. Furferi, and Y. Volpe, “Comparative evaluation of Intel RealSense D415, D435i, D455 and Microsoft Azure Kinect DK sensors for 3D Vision applications,” IEEE Access, vol. 12, pp. 111311-111321, 2024.
[13]	A. Kumar and E. Rajan, “3D Image Edge Detection Algorithm for depth detection,” International Journal of Recent Technology and Engineering (IJRTE), vol. 8, no. 2S11, pp. 3555–3557, 2019.
[14]	L. Zhang, H. Xia, and Y. Qiao, “Texture Synthesis Repair of RealSense D435i Depth Images with Object-Oriented RGB Image Segmentation,” Sensors, vol. 20, no. 23, p. 6725, 2020.
[15]	J. Li and Z. Wang, “Local Regression Based Hourglass Network for Hand Pose Estimation from a Single Depth Image,” 2018 24th International Conference on Pattern Recognition (ICPR), Aug. 2018.
[16]	M. Sladekova and A. P. Field, “Quantifying Heteroscedasticity in Linear Models Using Quantile LOWESS Intervals,” PsyArXiv, Jun. 2024.
[17]	Y. Dai, Y. Wang, M. Leng, X. Yang, and Q. Zhou, “LOWESS smoothing and Random Forest based GRU model: A short-term photovoltaic power generation forecasting method,” Energy, vol. 256, p. 124661, 2022.
[18]	F. Sherratt, A. Plummer, and P. Iravani, “Understanding LSTM network behaviour of IMU-Based locomotion mode recognition for applications in prostheses and wearables,” Sensors, vol. 21, no. 4, p. 1264, 2021