隨著IEEE 802.11be標準（Wi-Fi 7）的推出，無線通訊技術實現了資料傳輸速率和頻譜利用率的顯著提高。特別地，協調空間重用（C-SR）技術對於提升多接入點（AP）環境中的網路效能具有關鍵作用。然而，優化C-SR策略需要精確調整傳輸功率和選擇合適的調製編碼方案（MCS），這對傳統網路管理構成挑戰。本研究提出了一種基於強化學習的方法，動態調整傳輸功率和MCS選擇，以提高網路的信號品質和輸送量。通過模擬實驗，我們驗證了該方法能顯著優化高密度無線區域網路(Wireless LAN, WLAN)環境下的網路效能，相較於傳統策略，顯著提高了網路的總輸送量，並證明瞭強化學習在動態網路環境中調整功率和MCS的有效性。本研究為WLAN的智慧管理和自動化提供了有效的解決方案，推動了無線通訊技術的發展，並為實現更高效、可靠的無線連接提供了實際參考。

With the launch of the IEEE 802.11be standard (Wi-Fi 7), wireless communication technology has achieved significant improvements in data transmission rates and spectrum utilization. Specifically, Coordinated Spatial Reuse (C-SR) technology plays a key role in enhancing network efficiency in multi-access point (AP) environments. However, optimizing C-SR strategies requires precise adjustment of transmission power and selection of appropriate Modulation and Coding Schemes (MCS), which pose challenges for traditional network management. This study proposes a reinforcement learning-based method to dynamically adjust transmission power and MCS selection to improve network signal quality and throughput. Through simulation experiments, we validated that this method can significantly optimize network performance in high-density Wireless Lan environments, substantially increasing the total network throughput compared to traditional strategies, and proving the effectiveness of reinforcement learning in adjusting power and MCS in dynamic network environments. This research provides effective solutions for the intelligent management and automation of WLAN, advancing the development of wireless communication technology, and offering practical references for achieving more efficient and reliable wireless connections.

目錄
第一章	簡介	1
1.1簡述802.11be	2
1.2簡述C-SR	5
1.3研究目的	8
1.4相關研究	9
1.5論文貢獻	11
第二章	預備知識	13
第三章	系統模型	23
3.1場景與路徑損耗模型	23
3.2 C-TDMA/SR	25
3.3 Q-Learning	26
3.4 System Overview	28
第四章	Q-Learning Enhance Coordinated Spatial Reuse (QE-CSR)	30
4.1 QE-CSR	32
4.2 Transmission Scheduling Order Algorithm	42
第五章	Bianchi Model	44
第六章	Simulation Results	46
6.1 Data Rate比較	48
6.2 執行時間比較圖	52
6.3 QE-CSR不同節點密度與場景大小比較	53
第七章	結論	56
參考文獻	57

 
圖目錄
圖一、Wi-Fi各代版本功能比較圖	3
圖二、MLO示意圖	4
圖三、MRU示意圖	5
圖四、傳統SR運作示意圖。	6
圖五、傳統SR於高密度環境示意圖	7
圖六、C-SR於高密度環境示意圖	7
圖七、傳統無線區域網路的AP傳輸模式	14
圖八、OBSS示意圖	15
圖九、相鄰BSS的節點使用不同的BSS Color發送訊號	16
圖十、Intra-BSS示意圖，對STA1和STA2該BSS為Intra-BSS。	17
圖十一、Inter-BSS示意圖，對STA1來說BSS2為Inter-BSS。	18
圖十二、雙AP與雙STA的場景下，AP發射功率與STA接收SNR變化圖	20
圖十三、MCS與SNR關係圖	21
圖十四、MCS與輸送量關係圖	22
圖十五、企業密集WLAN場景。	23
圖十六、C-TDMA/SR方案	25
圖十七、MAP-TF封包內容	26
圖十八、Q-Learning的運作流程	27
圖十九、傳輸範例圖	31
圖二十、論文簡易架構圖	32
圖二十一、預設功率下傳輸範圍示意圖	33
圖二十二、調整功率後傳輸範圍示意圖	34
圖二十三、初始化Q-Table示意圖	35
圖二十四、動作(Action)以及狀態(State)初始化示意圖	36
圖二十五、初始狀態與Q-Table關係圖	37
圖二十六、可選擇動作示意圖	38
圖二十七、調整前與調整後最小SNR示意圖	38
圖二十八、Q-Table更新示意圖	40
圖二十九、論文架構圖	46
圖三十、所有傳輸可能性的輸送量變化圖	50
圖三十一、不重複傳輸組合的輸送量比較圖	51
圖三十二、Data Rate增益圖	52
圖三十三、執行時間比較圖	53
圖三十四、不同節點數量輸送量比較圖	54
圖三十五、不同場景大小輸送量比較圖	55

表目錄
表1、動作參照表	36
表2、網路模擬參數表格	47

I. AlShourbaji, "An overview of wireless local area network (WLAN)," arXiv preprint arXiv:1303.1882, 2013.
[2]	E. Mozaffariahrar, F. Theoleyre, and M. Menth, "A Survey of Wi-Fi 6: Technologies, Advances, and Challenges," Future Internet, vol. 14, no. 10, p. 293, 2022. [Online]. Available: https://www.mdpi.com/1999-5903/14/10/293.
[3]	A. Garcia-Rodriguez, D. López-Pérez, L. Galati-Giordano, and G. Geraci, "IEEE 802.11be: Wi-Fi 7 Strikes Back," IEEE Communications Magazine, vol. 59, no. 4, pp. 102-108, 2021, doi: 10.1109/MCOM.001.2000711.
[4]	S. Joshi, R. Roy, R. V. Bhat, P. Hathi, and N. Akhtar, "Dynamic Distributed Threshold Control for Spatial Reuse in IEEE 802.11ax," in 2022 National Conference on Communications (NCC), 24-27 May 2022 2022, pp. 373-378, doi: 10.1109/NCC55593.2022.9806744. 
[5]	F. Wilhelmi, S. Barrachina-Muñoz, C. Cano, I. Selinis, and B. Bellalta, "Spatial Reuse in IEEE 802.11ax WLANs," Computer Communications, vol. 170, pp. 65-83, 2021/03/15/ 2021, doi: https://doi.org/10.1016/j.comcom.2021.01.028.
[6]	A. Krotov, A. Kiryanov, and E. Khorov, "Rate Control With Spatial Reuse for Wi-Fi 6 Dense Deployments," IEEE Access, vol. 8, pp. 168898-168909, 2020, doi: 10.1109/ACCESS.2020.3023552.
[7]	C. Deng et al., "IEEE 802.11 be Wi-Fi 7: New challenges and opportunities," IEEE Communications Surveys & Tutorials, vol. 22, no. 4, pp. 2136-2166, 2020.
[8]	A. Garcia-Rodriguez, D. López-Pérez, L. Galati-Giordano, and G. Geraci, "IEEE 802.11 be: Wi-Fi 7 strikes back," IEEE Communications Magazine, vol. 59, no. 4, pp. 102-108, 2021.
[9]	H. Cai, B. Li, M. Yang, and Z. Yan, "Coordinated tdma mac scheme design and performance evaluation for the next generation wlan: Ieee 802.11 be," in Smart Grid and Internet of Things: 4th EAI International Conference, SGIoT 2020, TaiChung, Taiwan, December 5–6, 2020, Proceedings, 2021: Springer, pp. 297-306. 
[10]	W. Ahn, "Novel multi-AP coordinated transmission scheme for 7th generation WLAN 802.11 be," Entropy, vol. 22, no. 12, p. 1426, 2020.
[11]	D. Nunez, F. Wilhelmi, S. Avallone, M. Smith, and B. Bellalta, "TXOP sharing with Coordinated Spatial Reuse in Multi-AP Cooperative IEEE 802.11be WLANs," in 2022 IEEE 19th Annual Consumer Communications & Networking Conference (CCNC), 8-11 Jan. 2022 2022, pp. 864-870, doi: 10.1109/CCNC49033.2022.9700500. 
[12]	E. Haghani, M. N. Krishnan, and A. Zakhor, "Adaptive Carrier-Sensing for Throughput Improvement in IEEE 802.11 Networks," in 2010 IEEE Global Telecommunications Conference GLOBECOM 2010, 6-10 Dec. 2010 2010, pp. 1-6, doi: 10.1109/GLOCOM.2010.5683231. 
[13]	M. Park, "IEEE 802.11ac: Dynamic Bandwidth Channel Access," in 2011 IEEE International Conference on Communications (ICC), 5-9 June 2011 2011, pp. 1-5, doi: 10.1109/icc.2011.5963089. 
[14]	W. Wang, F. Zhang, and Q. Zhang, "Managing channel bonding with clear channel assessment in 802.11 networks," in 2016 IEEE International Conference on Communications (ICC), 22-27 May 2016 2016, pp. 1-6, doi: 10.1109/ICC.2016.7511455. 
[15]	I. Jamil, L. Cariou, and J. F. Hélard, "Efficient MAC protocols optimization for future high density WLANS," in 2015 IEEE Wireless Communications and Networking Conference (WCNC), 9-12 March 2015 2015, pp. 1054-1059, doi: 10.1109/WCNC.2015.7127615. 
[16]	M. S. Afaqui, E. Garcia-Villegas, E. Lopez-Aguilera, and D. Camps-Mur, "Dynamic sensitivity control of access points for IEEE 802.11ax," in 2016 IEEE International Conference on Communications (ICC), 22-27 May 2016 2016, pp. 1-7, doi: 10.1109/ICC.2016.7511025. 
[17]	J. Jung, J. Baik, Y. Kim, H. S. Park, and J. M. Chung, "OTOP: Optimized Transmission Power Controlled OBSS PD-Based Spatial Reuse for High Throughput in IEEE 802.11be WLANs," IEEE Internet of Things Journal, vol. 10, no. 19, pp. 17110-17123, 2023, doi: 10.1109/JIOT.2023.3275544.
[18]	Y. Wang, N. Liu, Z. Pan, and X. You, "AI-Based Resource Allocation in E2E Network Slicing with Both Public and Non-Public Slices," Applied Sciences, vol. 13, no. 22, p. 12505, 2023. [Online]. Available: https://www.mdpi.com/2076-3417/13/22/12505.
[19]	S. Merlin et al., "TGax Simulation Scenarios," Nov. 2015.
[20]	G. Bianchi, "Performance analysis of the IEEE 802.11 distributed coordination function," IEEE Journal on Selected Areas in Communications, vol. 18, no. 3, pp. 535-547, 2000, doi: 10.1109/49.840210.
[21]	C. J. C. H. Watkins and P. Dayan, "Q-learning," Machine Learning, vol. 8, no. 3, pp. 279-292, 1992/05/01 1992, doi: 10.1007/BF00992698.
[22]	M. Talukder and J. Xie, "Enhanced Coordinated Spatial Reuse: Bidirectional Multiple AP Coordination for IEEE 802.11 be," in ICC 2023-IEEE International Conference on Communications, 2023: IEEE, pp. 660-665. 
[23]    C. J. C. H. Watkins and P. Dayan, "Q-learning," Machine Learning, vol. 8, no. 3, pp. 279-292, 1992.
[24]    J. Clifton and E. Laber, "Q-learning: Theory and applications," Annual Review of Statistics and Its Application, vol. 7, no. 1, pp. 279-301, 2020.