§ 瀏覽學位論文書目資料
  
系統識別號 U0002-1008202309562900
DOI 10.6846/tku202300560
論文名稱(中文) 應用立體視覺於空中書寫之研究
論文名稱(英文) Research of applying computer stereo vision to air writing
第三語言論文名稱
校院名稱 淡江大學
系所名稱(中文) 資訊工程學系碩士班
系所名稱(英文) Department of Computer Science and Information Engineering
外國學位學校名稱
外國學位學院名稱
外國學位研究所名稱
學年度 111
學期 2
出版年 112
研究生(中文) 林愷蔚
研究生(英文) Kai-Wei Lin
學號 610410135
學位類別 碩士
語言別 繁體中文
第二語言別
口試日期 2023-06-12
論文頁數 31頁
口試委員 指導教授 - 陳建彰(ccchen34@mail.tku.edu.tw)
口試委員 - 許哲銓
口試委員 - 蔡耀弘
關鍵字(中) 雙目測距
肢體偵測
單應矩陣
關鍵字(英) Binocular stereo range-measuring
Limb detection
Homographic matrix
第三語言關鍵字
學科別分類
中文摘要
現今書法推廣主要透過展覽、比賽、教學和講座等相關活動進行。然而,隨著時代的演進,政治和社會環境都發生了巨大變化,傳統書法文化在吸引外部族群方面面臨困難,其影響力將逐漸減弱。因此,本研究旨在將書法與現代科技相結合,讓使用者只需用手指在空中揮舞,即可讓電腦模擬出逼真的書法書寫效果。這項研究採用兩台便利取得的網路攝影機,使用張正友標定法,實現雙目標定測距,從中獲取手的深度絕對座標。我們利用深度座標模擬書寫力道,當使用者離攝影機越近時,畫面上呈現的點越大,反之,則越小。本研究透過對於捕捉座標的優化,減少筆跡繪出時產生的誤差,同時也使用 MediaPipe Hand 技術抓取手的座標,無需任何物體輔助辨識。本研究提出之方法,藉由低成本的網路攝影機,實現空中寫字的技術。
英文摘要
Nowadays, calligraphy promotion is mainly carried out through related activities such as exhibitions, competitions, teaching, and lectures. However, with the evolution of the times, the political and social environment has undergone tremendous changes and traditional calligraphy culture faces difficulties in attracting external groups. The influence of calligraphy still gradually declines. Therefore, this research aims to improve calligraphy with modern technology. It allows users to simulate realistic calligraphy writing effects by simply waving their fingers in the air. This research uses two webcameras to achieve dual-target ranging, from which the depths of the hand are obtained. We use the depth coordinates to simulate the strength of writing. When the hand is near the camera, the points on the screen will be larger. In contrast, the points will be smaller when the hand is farther. By using the joint points coordinates extracted from the MediaPipe, we can write calligraphy on the screen by mathematical calculation. Therefore, the proposed system,based on two fundamental webcameras and mathematical computation, can implement Chinese calligraphy on the screen efficiently.
第三語言摘要
論文目次
目錄

第一章 緒論................................................................................................1

1.1 研究背景與動機...........................................................................1
1.2 研究目的.......................................................................................2
1.3 論文架構.......................................................................................2

第二章 文獻探討 .......................................................................................4

2.1 繪圖方式.......................................................................................4
2.1.1 接觸式繪圖.................................................................................4
2.1.2 非接觸式繪圖.............................................................................4
2.2 肢體辨識.......................................................................................6
2.2.1 OpenPose ...................................................................................6
2.2.2 MediaPipe ..................................................................................7
2.3 手勢控制.......................................................................................9
2.4 距離感測.....................................................................................10
2.5 立體視覺(Computer stereo vision)..........................................11
2.5.1 影像擷取...................................................................................11
2.5.2 相機標定...................................................................................11
2.5.3 特徵提取...................................................................................13

4

2.5.4 立體匹配...................................................................................14
2.5.5 深度資訊計算...........................................................................15

第三章 空中寫字系統 .............................................................................17

3.1 前置設定步驟.............................................................................17
3.2 辨識和繪畫步驟.........................................................................18

第四章 實驗結果 .....................................................................................21

4.1 實驗環境.....................................................................................21
4.2 比較單和雙網路攝影機的差別.................................................22
4.3 優化筆跡.....................................................................................25
4.4 空中寫字.....................................................................................26

第五章 結論與未來研究方向 .................................................................29

參考文獻...................................................................................................30

5

圖目錄

圖 1、OpenPose 抓取關節解說圖[3]..........................................6
圖 2、21 個手部座標[11]............................................................8
圖 3、手掌偵測模型架構[2].......................................................8
圖 4、、我們手部座標模型架構[2]...........................................9
圖 5、極線約束示意圖[20].......................................................15
圖 6、三角測量圖......................................................................16
圖 7、矯正步驟..........................................................................17
圖 8、MediaPipe 抓取手的關節點位置[22].............................19
圖 9、誤差點辨識步驟..............................................................20
圖 10、固定網路攝影機圖........................................................22
圖 11、矯正時所拍攝畫面........................................................22
圖 12、未辨別誤差點結果圖....................................................25
圖 13、使用辨別誤差點結果圖................................................25
圖 14、未使用兩點連接結果圖................................................26
圖 15、使用兩點連接結果圖....................................................26
圖 16、最大筆跡和最小筆跡圖................................................27
圖 17、空中寫字成果展示........................................................28

6

表目錄
表 1、距離感測比較表..............................................................10
表 2、單攝影機和雙攝影機比較表..........................................24
參考文獻
[1] V. Kriznar, M. Leskovsek, B. Batagelj, “Use of Computer Vision Based Hand
Tracking in Educational Environments.” 2010 IEEE Computer Society Conference
on Computer Vision and Pattern Recognition, Budapest, November, 2021.
[2] Zhang, Fan, et al. "Mediapipe hands: On-device real-time hand tracking." arXiv
preprint arXiv:2006.10214 (2020).
[3] Abdlkarim, Diar, et al. "A methodological framework to assess the accuracy of
virtual reality hand-tracking systems: A case study with the oculus quest 2."
BioRxiv (2022): 2022-02.
[4] https://www.uploadvr.com/facebook-wrist-based-hand-tracking-haptics/
[5] Zhang, Zhengyou. "Microsoft kinect sensor and its effect." IEEE multimedia 19.2
(2012): 4-10.
[6] Cao, Zhe, et al. "Realtime multi-person 2d pose estimation using part affinity
fields." IEEE conference on computer vision and pattern recognition. 2017.
[7] Chua, Leon O., and Tamas Roska. "The CNN paradigm." IEEE Transactions on
Circuits and Systems I: Fundamental Theory and Applications 40.3 (1993): 147-
156.
[8] Theeuwes, Jan. "Top–down and bottom–up control of visual selection." Acta
psychologica 135.2 (2010): 77-99.
[9] Križnar, V., M. Leskovšek, and Borut Batagelj. "Use of Computer Vision Based
Hand Tracking in Educational Environments." 2021 44th International Convention
on Information, Communication and Electronic Technology (MIPRO). IEEE, 2021.
[10] Lin, Tsung-Yi, et al. "Feature pyramid networks for object detection." IEEE
conference on computer vision and pattern recognition. 2017.
[11] Praditasari, Wibby AA, Ria Aprilliyani, and Ikhwannul Kholis. "Design and
implementation of interactive virtual museum based on hand tracking OpenCV in
indonesia." 2021 8th International Conference on Electrical Engineering, Computer
Science and Informatics (EECSI). IEEE, 2021.
[12] Suarez, Jesus, and Robin R. Murphy. "Hand gesture recognition with depth images:
A review." 2012 IEEE RO-MAN: the 21st IEEE international symposium on robot
and human interactive communication. IEEE, 2012.
[13] LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep learning." nature 521.7553 (2015): 436-444.
[14] Voulodimos, Athanasios, et al. "Deep learning for computer vision: A brief
review." Computational intelligence and neuroscience 2018 (2018).
[15] Maurice, Xavier, Pierre Graebling, and Christophe Doignon. "A pattern framework
driven by the hamming distance for structured light-based reconstruction with a
single image." CVPR 2011. Ieee, 2011.
[16] Hussain, Syed Ausaf, and Najeed Ahmed Khan. "Face-to-camera distance
estimation using machine learning." 2022 3rd International Conference on
Innovations in Computer Science & Software Engineering (ICONICS). IEEE, 2022.
[17] Wiley, W. C., and Ii H. McLaren. "Time‐of‐flight mass spectrometer with
improved resolution." Review of scientific instruments 26.12 (1955): 1150-1157.
[18] Zhang, Zhengyou. "A flexible new technique for camera calibration." IEEE
Transactions on pattern analysis and machine intelligence 22.11 (2000): 1330-1334.
[19] Zhou, Qiang, and Xin Li. "Stn-homography: estimate homography parameters
directly." arXiv preprint arXiv:1906.02539 (2019).
[20] Hirschmuller, Heiko, and Daniel Scharstein. "Evaluation of cost functions for
stereo matching." 2007 IEEE Conference on Computer Vision and Pattern
Recognition. IEEE, 2007.
[21] Xu, Zewen, Zheng Rong, and Yihong Wu. "A survey: which features are required
for dynamic visual simultaneous localization and mapping?." Visual Computing
for Industry, Biomedicine, and Art 4.1 (2021): 1-16.
[22] Lu, Ke, et al. "Binocular stereo vision based on OpenCV." IET International
Conference on Smart and Sustainable City (ICSSC 2011). IET, 2011.
[23] Di Leo, G., C. Liguori, and A. Paolillo. "Propagation of uncertainty through stereo
triangulation." 2010 IEEE Instrumentation & Measurement Technology
Conference Proceedings. IEEE, 2010.
[24] Chunduru, Vaishnav, Mrinalkanti Roy, and Rajeevlochana G. Chittawadigi. "Hand
tracking in 3d space using mediapipe and pnp method for intuitive control of virtual
globe." 2021 IEEE 9th Region 10 Humanitarian Technology Conference (R10-
HTC). IEEE, 2021.
論文全文使用權限
國家圖書館
不同意無償授權國家圖書館
校內
校內紙本論文立即公開
同意電子論文全文授權於全球公開
校內電子論文立即公開
校外
同意授權予資料庫廠商
校外電子論文立即公開

如有問題,歡迎洽詢!
圖書館數位資訊組 (02)2621-5656 轉 2487 或 來信