|Table of Contents|

Citation:
　Haiyan Zhao,Jing Yan,Tao Wu,et al.Energy-Efficient Underwater Data Collection: A Q-Learning Based Approach[J].Journal of Marine Science and Application,2022,(3):204-218.[doi:10.1007/s11804-022-00285-8]
Click and Copy

Energy-Efficient Underwater Data Collection: A Q-Learning Based Approach

Info

Title:
Energy-Efficient Underwater Data Collection: A Q-Learning Based Approach
Author(s):
Haiyan Zhao1; 2;  Jing Yan1; 2;  Tao Wu3;  Aihong Li1; 2;  Xiaoyuan Luo1
Affilations:
Author(s):
Haiyan Zhao1; 2;  Jing Yan1; 2;  Tao Wu3;  Aihong Li1; 2;  Xiaoyuan Luo1
1 Department of Automation, Yanshan University, Qinhuangdao 066004, China;
2 Key Laboratory of Ocean Observation Technology Ministry of Natural Resources, National Marine Technology Center, Tianjin 300110, China;
3 Department of Frontier and Innovation Research, Wuhan Second Ship Design and Research Institute, Wuhan 430205, China
Keywords:
Underwater data collection|Q-learning|Energy efficient|Path planning|Autonomous underwater vehicle
分类号:
-
DOI:
10.1007/s11804-022-00285-8
Abstract:
Underwater data collection is an importance part in the process of network monitoring, network management and intrusion detection. However, the limited-energy of nodes is a major challenge to collect underwater data. The solution of this problem are not only in the hands of network topology but in the hands of path of autonomous underwater vehicle (AUV). With the problem in hand, an energy-efficient data collection scheme is designed for mobile underwater network. Especially, the data collection scheme is divided into two phases, i.e., routing algorithm design for sensor nodes and path planing for AUV. With consideration of limited-energy and network robustness, Q-learning based dynamic routing algorithm is designed in the first phase to optimize the routing selection of nodes, through which a potential-game based optimal rigid graph method is proposed to balance the trade-off between the energy consumption and the network robustness. With the collected data, Q-learning based path planning strategy is proposed for AUV in the second phase to find the desired path to gather the data from data collector, then a mode-free tracking controller is developed to track the desired path accurately. Finally, the performance analysis and simulation results reveal that the proposed approach can guarantee energy-efficient and improve network stability.

References:

Ali T, Jung L, Faye I (2014) End-to-end delay and energy efficient routing protocol for underwater wireless sensor networks. Wireless Personal Communications 79(1):339-361. https://doi.org/10.1007/s11277-014-1859-z
Bhadauria D, Tekdas O, Isler V (2011) Robotic data mules for collecting data over sparse sensor fields. Journal of Field Robotics 28(3):388-404. https://doi.org/10.1002/rob.20384
Caccia M, Veruggio G (2000) Guidance and control of a reconfigurable unmanned underwater vehicle. Control Engineering Practice 8(1):21-37. https://doi.org/10.1016/S0967-0661(99)00125-2
Cai S, Zhu Y, Wang T, Xu G, Liu A, Liu X (2019) Data collection in underwater sensor networks based on mobile edge computing. IEEEAccess 7:65357-65367. https://doi.org/10.1109/ACCESS.2019.2918213
Cashmore M, Fox M, Long D, Magazzeni D, Ridder B (2018) Opportunistic planning in autonomous underwater missions. IEEE Transactions on Automation Science and Engineering 15(2):519-530. https://doi.org/10.1109/TASE.2016.2636662
Chen J, Xu Y, Wu Q, Zhang Y, Chen X, Qi N (2019) Interference-aware online distributed channel selection for multicluster fanet:A potential game approach. IEEE Transactions on Vehicular Technology 68(4):3792-3804. https://doi.org/10.1109/TVT.2019.2902177
Dhurandher S, Khairwal S, Obaidat M, Misra S (2009) Efficient data acquisition in underwater wireless sensor ad hoc networks. IEEE Wireless Communications 16(6):70-78. https://doi.org/10.1109/MWC.2009.5361181
Duan R, Du J, Jiang C, Ren Y (2020) Value-based hierarchical information collection for auv-enabled internet of underwater things. IEEE Internet of Things Journal 7(10):9870-9883. https://doi.org/10.1109/JIOT.2020.2994909
Gjanci P, Petrioli C, Basagni S, Phillips C, Bölöni L, Turgut D (2018) Path finding for maximum value of information in multimodal underwater wireless sensor networks. IEEE Transactions on Mobile Computing 17(2):404-418. https://doi.org/10.1109/TMC.2017.2706689
Gong Z, Li C, Jiang F (2020) A machine learning-based approach for auto-detection and localization of targets in underwater acoustic array networks. IEEE Transactions on Vehicular Technology 69(12):15857-15866. https://doi.org/10.1109/TVT.2020.3036350
Han G, Li S, Zhu C, Jiang J, Zhang W (2017) Probabilistic neighborhood-based data collection algorithms for 3D underwater acoustic sensor networks. Sensors 17(2):1-16. https://doi.org/10.3390/s17020316
Han G, Shen S, Song H, Yang T, Zhang W (2018) A stratification-based data collection scheme in underwater acoustic sensor networks. IEEE Transactions on Vehicular Technology 67(11):10671-10682. https://doi.org/10.1109/TVT.2018.2867021
Han G, Shen S, Wang H, Jiang J, Guizani M (2019) Prediction-based delay optimization data collection algorithm for underwater acoustic sensor networks. IEEE Transactions on Vehicular Technology 68(7):6926-6936. https://doi.org/10.1109/TVT.2019.2914586
Jin Z, Ma Y, Su Y, Li S, Fu X (2017) A Q-learning-based delay-aware routing algorithm to extend the lifetime of underwater sensor networks. Sensors 17(7):1-15. https://doi.org/10.3390/s17071660
Khan J, Cho H (2014) A data gathering protocol using AUV in underwater sensor networks. Proceedings of OCEANS, Taipei, 1-6. https://doi.org/10.3390/s150819331
Khan J, Cho H (2016) Data-gathering scheme using auvs in large-scale underwater sensor networks:A multihop approach. Sensors 16(10):1-20. https://doi.org/10.3390/s16101626
Kim J, Joe H, Yu S, Lee J, Kim, M (2016) Time-delay controller design for position control of autonomous underwater vehicle under disturbances. IEEE Transactions on Industrial Electronics 63(2):1052-1061. https://doi.org/10.1109/TIE.2015.2477270
Lee S, Park S, Choi H (2018) Potential game-based non-myopic sensor network planning for multi-target tracking. IEEE Access 6:79245-79257. https://doi.org/10.1109/ACCESS.2018.2885027
Li C, Xu Y, Diao B, Wang Q, An Z (2016a) DBR-MAC:A depth-based routing aware mac protocol for data collection in underwater acoustic sensor networks. IEEE Sensors Journal 16(10):3904-3913. https://doi.org/10.1109/JSEN.2016.2530815
Li G, Peng S, Wang C, Niu J, Yuan Y (2019) An energy-efficient data collection scheme using denoising autoencoder in wireless sensor networks. Tsinghua Science and Technology 24(1):86-96. https://doi.org/10.26599/TST.2018.9010002
Li N, Martínez J, Meneses Chaus J, Eckert M (2016b) A survey on underwater acoustic sensor network routing protocols. Sensors 16(3):1-28. https://doi.org/10.3390/s16030414
Lin C, Han G, Guizani M, Bi Y, Du J (2019) A scheme for delay-sensitive spatiotemporal routing in sdn-enabled underwater acoustic sensor networks. IEEE Transactions on Vehicular Technology 68(9):9280-9292. https://doi.org/10.1109/TVT.2019.2931312
Lin C, Deng D, Wang S (2016) Extending the lifetime of dynamic underwater acoustic sensor networks using multi-population harmony search algorithm. IEEE Sensors Journal 16(11):4034-4042. https://doi.org/10.1109/JSEN.2015.2440416
Liu Z, Meng X, Liu Y, Yang Y, Wang Y (2022) AUV-aided hybrid data collection scheme based on value of information for internet of underwater things. IEEE Internet of Things Journal 9(9):6944-6955. https://doi.org/10.1109/JIOT.2021.3115800
Luo X, Li X, Wang J, Guan X (2018) Potential-game based optimally rigid topology control in wireless sensor networks. IEEE Access 6:16599-16609. https://doi.org/10.1109/ACCESS.2018.2814079
Luo X, Yan Y, Li S, Guan X (2013) Topology control based on optimally rigid graph in wireless sensor networks. Computer Networks 57(4):1037-1047. https://doi.org/10.1016/j.comnet2012.12.002
McMahon J, Plaku E (2017) Autonomous data collection with limited time for underwater vehicles. IEEE Robotics and Automation Letters 2(1):112-119. https://doi.org/10.1109/LRA.2016.2553175
Ren J, Zhang Y, Zhang K, Liu A, Chen J, Shen X (2016) Lifetime and energy hole evolution analysis in data-gathering wireless sensor networks. IEEE Transactions on Industrial Informatics 12(2):788-800. https://doi.org/10.1109/TII.2015.2411231
Sehgal A, David C, Schönwälder J (2011) Energy consumption analysis of underwater acoustic sensor networks. Proceedings of OCEANS, Waikoloa, USA, 1-6. https://doi.org/10.23919/OCEANS.2011.6107287
Shames I, Summers T (2015) Rigid network design via submodular set function optimization. IEEE Transactions on Network Science and Engineering 2(3):84-96. https://doi.org/10.1109/TNSE.2015.2480247
Su R, Zhang D, Li C, Gong Z, Venkatesan R, Jiang F (2019) Localization and data collection in auv-aided underwater sensor networks:Challenges and opportunities. IEEE Network 33(6):86-93. https://doi.org/10.1109/MNET.2019.1800425
Thenozhi S, Yu W (2014) Stability analysis of active vibration control of building structures using PD/PID control. Engineering Structures 81:208-218. https://doi.org/10.1016/j.engstruct.2014.09.042
Wang Z, Song H, Watkins D, Ong K, Xue P, Yang Q, Shi X (2015) Cyber-physical systems for water sustainability:challenges and opportunities. IEEE Communications Magazine 53(5):216-222. https://doi.org/10.1109/MCOM.2015.7105668
Wei B, Luo Y, Jin Z, Wei J, Su Y (2013) ES-VBF:an energy saving routing protocol. Proceedings of the 2012 International Conference on Information Technology and Software Engineering, Beijing, 87-97. https://doi.org/10.1007/978-3-642-34528-9-10
Wu J, Sun X, Wu J, Han G (2021) Routing strategy of reducing energy consumption for underwater data collection. Intelligent and Converged Networks 2(3):163-176. https://doi.org/10.23919/ICN.2021.0012
Xia N, Ou Y, Wang S, Zheng R, Du H, Xu C (2017) Localizability judgment in uwsns based on skeleton and rigidity theory. IEEE Transactions on Mobile Computing 16(4):980-989. https://doi.org/10.1109/TMC.2016.2586051
Xie P, Cui J, Lao L (2006) VBF:Vector-based forwarding protocol for underwater sensor networks. Proceeding of International Conference on Research in Networking, Springer, Berlin, Heidelberg, 1216-1221. https://doi.org/10.1007/11753810-111
Yan J, Chen C, Luo X, Yang X, Hua C, Guan X (2016) Distributed formation control for teleoperating cyber-physical system under time delay and actuator saturation constrains. Information Sciences 370:680-694. https://doi.org/10.1016/j.ins.2016.02.019
Yan J, Zhao H, Meng Y, Guan X (2021a) Localization in underwater sensor networks. ser. Wireless Networks, Singapore Springer. https://doi.org/10.1007/978-981-16-4831-1
Yan J, Yang X, Zhao H, Luo X, Guan X (2021b) Autonomous underwater vehicles:Localization, tracking, and formation. ser. Cognitive Intelligence and Robotics, Singapore Springer. https://doi.org/10.1007/978-981-16-6096-2
Yan J, Yang X, Luo X, Chen C (2018) Energy-efficient data collection over auv-assisted underwater acoustic sensor network. IEEE Systems Journal 12(4):3519-3530. https://doi.org/10.1109/JSYST.2017.2789283
Yan J, Zhao H, Luo X, Wang Y, Chen C, Guan X (2020a) Asynchronous localization of underwater target using consensus-based unscented kalman filtering. IEEE Journal of Oceanic Engineering 45(4):1466-1481. https://doi.org/10.1109/JOE.2019.2923826
Yan J, Zhao H, Pu B, Luo X, Chen C, Guan X (2020b) Energy-efficient target tracking with uasns:A consensus-based bayesian approach. IEEE Transactions on Automation Science and Engineering 17(3):1361-1375. https://doi.org/10.1109/TASE.2019.2950702
Yu H, Yao N, Liu J (2015) An adaptive routing protocol in underwater sparse acoustic sensor networks. Ad Hoc Networks 34:121-143. https://doi.org/10.1016/j.adhoc.2014.09.016
Zhang P, de Queiroz M, Cai X (2015) Three-dimensional dynamic formation control of multi-agent systems using rigid graphs. Journal of Dynamic Systems, Measurement, and Control 137(11):1-7. https://doi.org/10.1115/L4030973
Zhao C, Guo L (2017) PID controller design for second order nonlinear uncertain systems. Science China Information Sciences 60(2):1-13. https://doi.org/10.1007/s11432-016-0879-3

Memo

Memo:
Received date:2022-05-21;Accepted date:2022-07-07。
Foundation item:Supported by the National Natural Science Foundation of China (61873345, 62222314), the Distinguished Young Foundation of Hebei Province (F2022203001), the Central Guidance Local Foundation of Hebei Province (226Z3201G), the three-three-three Foundation of Hebei Province (C20221019), and the Open Fund Project of Key Laboratory of Ocean Observation Technology, MNR (2021klootA02).
Corresponding author:Jing Yan,E-mail:jyan@ysu.edu.cn
Last Update: 2022-10-09