隨著物聯網技術的發展，智慧農業對長期穩定的環境數據監測需求日益增加，但感測器數量龐大且電源受限，造成能源管理上的挑戰。本論文針對此問題，提出一套基於 Q-learning 的自主喚醒與休眠控制機制，應用於無線供能感測網路中。感測器能依據自身的感測值變化、時間段與休眠間隔等狀態，自行學習最佳的喚醒策略，以動態平衡資料即時性與能耗效率。
論文首先建構智慧農業場景的系統架構，結合超低功耗藍牙（BLE）、LoRa 及多接取點（AP）無線供能設計，並採用 Poisson Disk 佈建演算法確保感測器均勻分布。模擬實驗中，本論文以台北、台中與高雄之氣象數據為基礎，透過 Q-learning 與週期性喚醒策略進行比較。結果顯示，Q-learning 機制能有效減少不必要的喚醒次數，在環境穩定時展現顯著節能效果，且在劇烈變化情境下仍能維持資料即時性。
綜合而言，本論文驗證了 Q-learning 應用於智慧農業感測器喚醒機制的可行性，能同時提升能源利用效率與系統適應性，為大規模物聯網感測網路的實務應用提供參考。

With the development of IoT technology, smart agriculture is increasingly demanding long-term, stable environmental data monitoring. However, the large number of sensors and limited power supply pose energy management challenges. This paper addresses this issue by proposing an autonomous wake-up and sleep control mechanism based on Q-learning for wireless sensor networks. Sensors can autonomously learn optimal wake-up strategies based on their own sensing value changes, time periods, and sleep intervals, dynamically balancing data immediacy and energy efficiency.

The paper first constructs a system architecture for smart agriculture scenarios, combining ultra-low-power Bluetooth (BLE), LoRa, and a multi-access point (AP) wireless power design. A Poisson Disk deployment algorithm is employed to ensure even sensor distribution. In simulation experiments, Q-learning is compared with a periodic wake-up strategy based on meteorological data from Taipei, Taichung, and Kaohsiung. Results demonstrate that the Q-learning mechanism effectively reduces unnecessary wake-ups, significantly reducing energy consumption in stable environments, and maintaining data immediacy even under volatile conditions.
In summary, this paper demonstrates the feasibility of applying Q-learning to the wake-up mechanism of smart agricultural sensors, which can simultaneously improve energy efficiency and system adaptability, providing a reference for the practical application of large-scale IoT sensor networks.

目錄
第 1 章　Introduction.....................................................................1
第 2 章　Background and Literature Review...........................................................................8
2.1　相關技術背景介紹..............................................................8
2.2　強化學習與Q-learning原理 ....................................................12
2.3　感測器部署演算法與 LoRa 通訊技術..............................................15
2.4　現有喚醒與休眠控制策略回顧....................................................18
2.5　本研究與現有方法差異..........................................................22
2.6　本章小結.....................................................................26
第 3 章　系統架構與感測器佈建......................................................27
3.1　系統整體架與應用場景說明......................................................27
3.2　感測器與AP佈建策略..........................................................30
第 4 章　Q-learning 自適應喚醒機制設計............................................35
4.1　狀態、行動與獎勵設計.........................................................35
4.2　Q-learning 演算法與參數設定..................................................42
4.3　模擬實現與資源管理流程.......................................................49
第 5 章　模擬結果與分析............................................................53
5.1　模擬設定與資料來源（台北／台中／高雄）.........................................53
5.2　不同策略下之能耗與喚醒效能比較................................................59
5.3　不同情境下自適應學習機制的表現................................................64
第 6 章　結論與未來工作............................................................69
6.1　研究結論與貢獻 ..............................................................69
6.2　研究限制與未來方向 ..........................................................71
6.3　全文總結....................................................................74
參考文獻.........................................................................75

圖目錄
圖一、無線供能通訊網路（WPCN）架構示意圖.............................................2
圖二、裝置根據環境轉換之睡眠與喚醒示意圖.............................................3
圖三、週期性喚醒與Q-learning自主喚醒比較.............................................4
圖四、 Q-learning自主學習示意圖.....................................................6
圖五、環境感測與無線供能物聯網（WPCN）整合應用概念圖..................................8
圖六、Poisson Disk演算法示意圖：有效點（綠）、無效點（橘）與最小間距半徑...............16
圖七、本研究於感測網路系統四大核心面向之改進方向示意圖................................25
圖八、高理想化數據通訊流程圖.......................................................28
圖九、線性內插法示意圖............................................................54
圖十、AP與感測器部屬之結果.........................................................56
圖十一、Q-learning和Duty cycle的能耗比較結果圖.....................................62
圖十二、學習到午後雷陣雨最為明顯的一天.............................................63
圖十三、台北地區三種感測器每日喚醒無傳次數（Q-learning，五次訓練平均）................65
圖十四、台中地區三種感測器每日喚醒無傳次數（Q-learning，五次訓練平均）................65
圖十五、高雄地區三種感測器每日喚醒無傳次數（Q-learning，五次訓練平均）................65
圖十六、Duty Cycle 策略下每日固定喚醒並回傳次數（每 15 分鐘喚醒）....................66
圖十七、台北地區三種感測器每日喚醒並回傳次數（Q-learning，五次訓練平均）..............67
圖十八、台中地區三種感測器每日喚醒並回傳次數（Q-learning，五次訓練平均）..............67
圖十九、高雄地區三種感測器每日喚醒並回傳次數（Q-learning，五次訓練平均）..............67

表目錄
表一、喚醒與休眠控制方式比較.....................................................20
表二、執行Q-learning需考慮的能耗表...............................................52
表三、使用者根據三種感測器的ΔV輸入Moderate的值後系統自動推倒結果....................55
表四、使用者根據三種感測器的V輸入Normal的值後系統自動推倒結.果......................55

1)	A comparative study of LPWAN technologies for large-scale IoT deployment(2019)
2)	Energy Efficiency Trade-Off Between Duty-Cycling and Wake-Up Radio Techniques in IoT Networks
3)	Energy-efficient power scheduling and allocation scheme
4)	Tiny Machine Learning and On-Device Inference: A Survey of Applications, Challenges, and Future Directions
5)	Cisco Report
6)	A reinforcement learning based sleep scheduling algorithm for compressive data gathering in wireless sensor networks (RLSSA CDG) (2023)
7)	Q-Learning and Efficient Low-Quantity Charge Method for Nodes to Extend the Lifetime of Wireless Sensor Networks(2023)
8)	Energy Aware Deep Reinforcement Learning Scheduling for Sensors Correlated in Time and Space(2020)
9)	Deep Reinforcement Learning for Task Offloading in UAV Aided Smart Farm Networks(2022)
10)	Research on rechargeable agricultural wireless sensor network based on ZigBee immune routing repair algorithm(2024)
11)	A smart agriculture IoT system based on deep reinforcement learning(2019)
12)	Energy-Efficient Wireless Sensor Networks for Precision Agriculture, (2017)
13)	Wireless Technologies for Agricultural Monitoring using IoT Devices with Energy Harvesting Capabilities(2020)
14)	An adaptive hexagonal deployment model for resilient wireless sensor networks in precision agriculture(2024)
15)	Energy Aware Deep Reinforcement Learning Scheduling for Sensors(2020)
16)	A Reinforcement Learning-Based Dynamic Clustering of Sleep Scheduling(2024)
17)	Optimizing QoS and security in agriculture IoT deployments using Q-Learning(2024)
18)	Environmental Perception Q-Learning to Prolong the Lifetime of Poultry Farm Monitoring Networks(2021)
19)	A comprehensive review of sensor node deployment strategies for wireless sensor networks(2024)
20)	A Delay-Tolerant low-duty cycle scheme in wireless sensor networks for IoT applications(2023)
21)	Energy-Efficient Wireless Sensor Networks for Precision Agriculture: A Review(2017)
22)	Wireless Technologies for Agricultural Monitoring using Internet of Things Devices with Energy Harvesting Capabilities(2020)
23)	Design and Implementation of a Wireless SensorNetwork for Agricultural Applications(2019)
24)	Sustainable Smart Farm Networks: Enhancing Resilience and Efficiency with Decision Theory-Guided Deep Reinforcement Learning(2024)
25)	Autonomous Management of Energy-Harvesting IoT Nodes Using Deep Reinforcement Learning(2019)
26)	ICs afford accurate temperature sensing(2020)
27)	How to Reduce Power Consumption in IoT Devices for Extended Battery Life(2023)
28)	Sustainable Agriculture with Self-Powered Wireless Sensing(2024)
29)	Attack-Resistant, Energy-Adaptive Monitoring for Smart Farms: Uncertainty-Aware Deep Reinforcement Learning Approach(2021)
30)	ACES -- Automatic Configuration of Energy Harvesting Sensors with Reinforcement Learning(2019)