投籃動作需要用到的是全身的協調性，不僅僅在於手的姿勢與力量。本研究利用攝影機錄製的影片，轉換得到每張影格的人體關節點資訊，進而利用其分析投籃姿勢的優劣和命中率之關係，並透過條件式生成對抗網路（cGAN）提供姿勢的改進方向。
本研究將影片轉為人體關節點資訊後，依跳投的特性計算每次投籃的區間，再將關節點在投籃過程的節點資訊繪製成貝茲曲線（Bézier curve），再以曲線相似度演算法比較與其他次投籃的相似度，最後使用最近鄰居分類法（K-Nearest Neighbor, KNN）進行投進和投失的姿勢判斷，並以混淆矩陣（Confusion matrix）分析其研究結果。研究結果顯示一般籃球課之學生，穩定度較差，能由研究顯示出其投進和投失的姿勢差異；而校隊球員已有相當穩定的投籃姿勢，因此難以從投籃姿勢判斷進球與否，需要考量更多的投籃資訊。

A good basketball shooting is the coordination of the whole body, not only on the posture and strength of the hand. This study adopts calculated human joint points to analyze the relationship between the pros and cons of shooting posture and the hit rate. The posture improvement suggestion, estimated from cGAN, is then provided.
In this study, a shooting period is first estimated from human joint points, extracted from the OpenPose system, in a shooting video. The Bézier curve formed from these human joint points is then calculated for the further matching process. By using curve fitting method, K-NN classification method, and confusion matrix analysis, the Discrete Frechet distance method with 3-NN outperforms other combinations. Moreover, experimental results show that students in general basketball classes have poor stability and the proposed structure can estimate the shooting with accuracy more than 85%. However, the university team players have a stable degree of shooting posture. Therefore, it is difficult to estimate the shooting accuracy from the shooting posture.

致謝	i
摘要	ii
Abstract	iii
目錄	iv
圖目錄	vii
表目錄	x
第一章 緒論	1
1.1 研究背景與動機	1
1.2 研究目的	2
1.3 論文架構	3
第二章 文獻探討	4
2.1 動作識別相關研究	4
2.2 OpenPose介紹	4
2.2.1 OpenPose演算法Part Affinity Fields簡介	7
2.2.2 基於OpenPose相關研究	10
2.3 其他人體姿態評估相關研究	11
2.4 生成對抗網路	13
2.4.1 Conditional GAN	15
2.4.2 圖像轉換問題	16
2.5 投籃姿勢的校正	19
第三章 投籃姿勢修正建議系統	22
3.1 系統流程架構	22
3.2 資料集	25
3.2.1 投籃過程資料集	25
3.2.2 投籃錯誤姿勢資料集	26
3.3 姿勢相似度分類方法	29
3.3.1 曲線相似度演算法	30
3.3.2 投籃姿勢分類方法	31
3.4 投籃姿勢修正方法	33
第四章 姿勢分析實驗結果	36
4.1 投籃姿勢分析	36
4.1.1 一般體育課學生姿勢分析結果	39
4.1.2 校隊投籃姿勢分析結果	43
4.1.3 曲線相似度演算法結果之比較	47
4.2 錯誤投籃姿勢修正結果	54
4.3 討論	56
第五章 結論與建議	58
5.1 結論	58
5.2 未來研究方向	58
參考文獻	60
附錄一 英文論文	64


圖目錄
圖 1：OpenPose的兩種關節點[1]5
圖 2：OpenPose手指及臉部關節點[1]5
圖 3：DeepCut流程示意圖[29]6
圖 4：OpenPose 模型架構圖[7]8
圖 5：手肘置信圖（confidence heat map）9
圖 6：手腕和手肘的關係圖（PAFs）9
圖 7：完善姿態評估的校正過程(a)-(e)、(f)-(j) [8]12
圖 8：HRNet架構圖[36]13
圖 9：GAN架構圖15
圖 10：cGAN網路架構圖[26]16
圖 11：encoder-decoder之網路架構[22]17
圖 12：U-Net網路架構[22]17
圖 13：pix2pix判別器D的輸入[22]19
圖 14：pix2pix使用不同目標函數之比較[22]19
圖 15：(a)蹲下時標準姿勢；(b)伸展時標準姿勢；(c)出手時標準姿勢[20]	20
圖 16：(a)投籃前；(b)投籃後提高投球進球機率之特徵[27]21
圖 17：系統流程架構23
圖 18：由系統擷取之投籃過程24
圖 19：(a) 修正前錯誤姿勢；(b)修正後姿勢建議24
圖 20：五個關節之投籃過程25
圖 21：標準開始投藍之特徵[27]26
圖 22：標準結束投籃之特徵[27]27
圖 23：(a)正規化前原始影像；(b)正規化後訓練用影像29
圖 24：圖18經由手動修正後之姿勢建議29
圖 25：五個關節點之貝茲曲線	30
圖 26：生成器G之神經網路架構圖34
圖 27：判別器D之神經網路架構圖35
圖 28：一般體育課學生實驗結果41
圖 29：一般體育課學生實驗結果之混淆矩陣42
圖 30：10名一般學生使用C3相似度算法預測投籃情形的混淆矩陣43
圖 31：校隊實驗結果45
圖 32：校隊實驗結果之混淆矩陣46
圖 33：10名校隊球員使用C1曲線相似度演算法預測投籃情形的混淆矩陣47
圖 34：四次實驗在五種曲線相似度演算法上的表現48
圖 35：一般體育課學生個別差異實驗之預測表現49
圖 36：一般體育課學生在整體差異實驗之預測表現50
圖 37：校隊球員在個別差異實驗之預測表現50
圖 38：校隊球員在整體差異實驗之預測表現51
圖 39：GAN三種訓練回合之比較55
圖 40：GAN訓練500回後之效果56
圖 41：GAN錯誤產生的例子56

表目錄
表 1：兩組資料集之有效球數37
表 2：各曲線相似度演算法之代號37
表 3：混淆矩陣對應其結果之對照表38
表 4：一般體育課學生在個別差異實驗之混淆矩陣52
表 5：一般體育課學生在整體差異實驗之混淆矩陣52
表 6：校隊球員在個別差異實驗之混淆矩陣53
表 7：校隊球員在整體差異實驗之混淆矩陣53

參考文獻
[1]	https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/ output.md

[2]	H. Alt and M. Godau, “Computing the Fréchet Distance Between Two Polygonal Curves,” In International Journal of Computational Geometry & Applications, vol. 05, pp. 75-91, 1995.

[3]	A. Andrade-Campos, R. de-Carvalho, R. A. F. Valente, “Novel Criteria for Determination of Material Model Parameters,” In International Journal of Mechanical Sciences, vol. 54, pp. 294-305, 2012.

[4]	D. J. Berndt and J. Clifford, “Using Dynamic Time Warping to Find Patterns in Time Series,” In International Conference on Knowledge Discovery and Data Mining, pp. 359-370, 1994.

[5]	K. Bringmann, “Why Walking the Dog Takes Time: Fréchet Distance Has No Strongly Subquadratic Algorithms Unless SETH Fails,” In IEEE 55th Annual Symposium on Foundations of Computer Science, vol. 1, pp. 661-670, 2014.

[6]	J. Cao and J. Lin, “A Study on Formulation of Objective Functions for Determining Material Models,” In International Journal of Mechanical Sciences, vol. 50, pp. 193-204, 2008.

[7]	Z. Cao, T. Simon, S. E. Wei, Y. Sheikh, “Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields,” In Conference on Computer Vision and Pattern Recognition, pp. 7291-7299, 2017.

[8]	J. Carreira, P. Agrawal, K. Fragkiadaki, J. Malik, “Human Pose Estimation with Iterative Error Feedback,” In Conference on Computer Vision and Pattern Recognition, pp. 4733-4742, 2016.

[9]	Y. J. Chang, S.-F. Chen, and J. D. Huang, “A Kinect-based System for Physical Rehabilitation: A Pilot Study for Young Adults with Motor Disabilities,” In Research in developmental disabilities, vol. 32, pp. 2566-2570, 2011.

[10]	Q. de-Smedt, H. Wannous, J. P. Vandeborre, J. Guerry, B. L. Saux, D. Filliat, “3D Hand Gesture Recognition Using a Depth and Skeletal Dataset: SHREC'17 track,” In 10th Eurographics Workshop on 3D Object Retrieval, pp. 33-38, 2017.

[11]	G. Devineau, W. Xi, F. Moutarde, J. Yang, “Convolutional Neural Networks for Multivariate Time Series Classification using both Inter- and Intra- Channel Parallel Convolutions,” In Reconnaissance des Formes, Image, Apprentissage et Perception, 2018.

[12]	A. Driemel, S. Har-Peled, C. Wenk, “Approximating the Fréchet Distance for Realistic Curves in Near Linear Time,” In Discrete & Computational Geometry, vol. 48, pp. 94-127, 2012.
[13]	T. Eiter and H. Mannila, “Computing discrete Fréchet distance,” Technical report, 1994.

[14]	K. Evans, S. A. Horan, R. J. Neal, R. S. Barrett, P. M. Mills, “Repeatability of Three-Dimensional Thorax and Pelvis Kinematics in the Golf Swing Measured Using a Field-Based Motion Capture System,” In Sports Biomechanics, vol. 11, pp. 262-272, 2012.

[15]	F. Petitjean, A. Ketterlin, P. Gançarski, “A Global Averaging Method for Dynamic Time Warping, with Applications to Clustering,” In Pattern Recognition, vol. 44, pp. 678-693, 2011.

[16]	T. Giorgino, “Computing and Visualizing Dynamic Time Warping Alignments in R: The dtw Package,” In Journal of Statistical Software, vol. 31, pp. 1-24, 2009.

[17]	I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, “Generative Adversarial Nets,” In Conference on Neural Information Processing Systems, 2014.

[18]	G. Hidalgo, Y. Raaj, H. Idrees, D. Xiang, H. Joo, T. Simon, Y. Sheikh, “Single-Network Whole-Body Pose Estimation,” In International Conference on Computer Vision, 2019.

[19]	J. Hou, G. Wang, X. Chen, J. H. Xue, R. Zhu, H. Yang, “Spatial-Temporal Attention Res-TCN for Skeleton-based Dynamic Hand Gesture Recognition,” In European Conference on Computer Vision, pp. 273-286, 2018.

[20]	C. H. Hsia, C. H. Chien, H. W. Hsu, Y. F. Chang, J. S. Chiang, “Analyses of Basketball Player Field Goal Shooting Postures for Player Motion Correction Using Kinect Sensor,” In International Symposium on Intelligent Signal Processing and Communication Systems, pp. 1-4, 2014.

[21]	L. H. Hsu and Y. S. Tsai, “Real-time Depth Estimation for Multiple Persons Using Single Camera and OpenPose,” Master thesis in NTOU, 2019.

[22]	P. Isola, J. Y. Zhu, T. Zhou, A. A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks,” In Conference on Computer Vision and Pattern Recognition, pp. 1125-1134, 2018.

[23]	C. F. Jekel, G. Venter, M. P. Venter, N. Stander, R. T. Haftka, “Similarity Measures for Identifying Material Parameters from Hysteresis Loops using Inverse Analysis,” In International Journal of Material Forming, vol. 12, pp. 355-378, 2019.

[24]	S. L Seyler, A. Kumar, M. F. Thorpe, O. Beckstein, “Path Similarity Analysis: A Method for Quantifying Macromolecular Pathways,” In PLOS Computational Biology, vol. 11, pp. 1-37, 2015.

[25]	L. Liu, X. Wu, L. Wu, T. Guo, “Static Human Gesture grading based on Kinect,” In 5th International Congress on Image and Signal Processing, 2012.

[26]	M. Mirza and S. Osindero, “Conditional Generative Adversarial Nets,” In Conference on Computer Vision and Pattern Recognition, 2014.

[27]	M. Nakai, Y. Tsunoda, H. Hayashi, H. Murakoshi, “Prediction of Basketball Free Throw Shooting by OpenPose,” In Japanese Society for Artificial Intelligence International Symposia on Artificial Intelligence: New Frontiers in Artificial Intelligence, pp. 435-446, 2018.

[28]	A. Newell, K. Yang, J. Deng, “Stacked Hourglass Networks for Human Pose Estimation,” In European Conference on Computer Vision, pp 483-499, 2016.

[29]	L. Pishchulin, E. Insafutdinov, S. Tang, B. Andres, M. Andriluka, P. V. Gehler, B. Schiele, “DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation,” In Conference on Computer Vision and Pattern Recognition, pp. 4929-4937, 2016.

[30]	S. Qiao, Y. Wang, J. Li, “Real-Time Human Gesture Grading Based on OpenPose,” In 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics, 2017.

[31]	V. Ramakrishna, D. Munoz, M. Hebert, J. A. Bagnell, Y. Sheikh, “Pose Machines: Articulated Pose Estimation via Inference Machines,” In European Conference on Computer Vision, pp. 33-47, 2014.

[32]	C. Reinschmidt, A. J. van-den-Bogert, B. M. Nigg, A. Lundberg, N. Murphy, “Effect of Skin Movement on the Analysis of Skeletal Knee Joint Motion During Running,” In Journal of Biomechanics, vol. 30, pp. 729-732, 1997.

[33]	Z. Ren, J. Meng, J. Yuan, Z. Zhang, “Robust hand gesture recognition with kinect sensor,” In 19th ACM international conference on Multimedia, pp. 759-760, 2011.

[34]	S. Salvador and P. K. Chan, “Toward Accurate Dynamic Time Warping in Linear Time and Space,” In Intelligent Data Analysis, pp. 561-580, 2007.

[35]	P. Senin, “Dynamic Time Warping Algorithm Revie,” In Information and Computer Science Department University of Hawaii at Manoa Honolulu, pp. 1-23, 2008.

[36]	K. Sun, B. Xiao, D. Liu, J. Wang, “Deep High-Resolution Representation Learning for Human Pose Estimation,” In Conference on Computer Vision and Pattern Recognition, pp. 5693-5703, 2019.

[37]	J. Tompson, R. Goroshin, A. Jain, Y. LeCun, C. Bregler, “Efficient Object Localization Using Convolutional Networks,” In Conference on Computer Vision and Pattern Recognition, pp. 648-656, 2015.

[38]	P. Tormene, T. Giorgino, S. Quaglini, M. Stefanelli, “Matching Incomplete Time Series With Dynamic Time Warping: An Algorithm and an Application to Post-Stroke Rehabilitation,” In Artificial Intelligence in Medicine, vol. 45, pp. 11-34, 2009.
[39]	A. Toshev and C. Szegedy, “DeepPose: Human Pose Estimation via Deep Neural Networks,” In Conference on Computer Vision and Pattern Recognition, pp. 1653-1660, 2014.

[40]	C. E. Tsai and Y. T. Ching, “A Human Identification System Based on Skeleton Key Points Data in The Scenario of People Running on a Treadmill,” Master thesis in NCTU, 2019.

[41]	S. E. Wei, N. C. Tang, Y. Y. Lin, M. F. Weng, H. Y. M. Liao, “Skeleton-augmented Human Action Understanding by Learning with Progressively Refined Data,” In 1st ACM International Workshop on Human Centered Event Understanding from Multimedia, pp. 7-10, 2014.

[42]	S. E. Wei, V. RamaKrishna, T. Kanade, Y. Sheikh, “Convolutional Pose Machines,” In Conference on Computer Vision and Pattern Recognition, pp. 4724-4732, 2016.

[43]	K. Witowski and N. Stander, “Parameter Identification of Hysteretic Models Using Partial Curve Mapping,” In 12th AIAA Aviation Technology, Integration, and Operations (ATIO) Conference and 14th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, 2012.

[44]	B. Xiao, H. Wu, Y. Wei, “Simple Baselines for Human Pose Estimation and Tracking,” In European Conference on Computer Vision, pp. 466-481, 2018.

[45]	Y. Fujimori, Y. Ohmura, T. Harada, Y. Kuniyoshi, “Wearable motion capture suit with full-body tactile sensors,” In IEEE International Conference on Robotics and Automation, 2009.