|Table of Contents|

Citation:
　Kohyar Bolvary Zadeh Dashtestani,Reza Javidan,Reza Akbari.Dynamic Clustering Method for Underwater Wireless Sensor Networks based on Deep Reinforcement Learning[J].Journal of Marine Science and Application,2025,(4):864-876.[doi:10.1007/s11804-025-00647-y]
Click and Copy

Dynamic Clustering Method for Underwater Wireless Sensor Networks based on Deep Reinforcement Learning

Info

Title:
Dynamic Clustering Method for Underwater Wireless Sensor Networks based on Deep Reinforcement Learning
Author(s):
Kohyar Bolvary Zadeh Dashtestani;  Reza Javidan;  Reza Akbari
Affilations:
Author(s):
Kohyar Bolvary Zadeh Dashtestani;  Reza Javidan;  Reza Akbari
Department of Computer Engineering and Information Technology, Shiraz University of Technology, Shiraz 7155713876, Iran
Keywords:
Underwater wireless sensor network; Clustering; Cluster head selection; Deep reinforcement learning
分类号:
-
DOI:
10.1007/s11804-025-00647-y
Abstract:
Underwater wireless sensor networks (UWSNs) have emerged as a new paradigm of real-time organized systems, which are utilized in a diverse array of scenarios to manage the underwater environment surrounding them. One of the major challenges that these systems confront is topology control via clustering, which reduces the overload of wireless communications within a network and ensures low energy consumption and good scalability. This study aimed to present a clustering technique in which the clustering process and cluster head (CH) selection are performed based on the Markov decision process and deep reinforcement learning (DRL). DRL algorithm selects the CH by maximizing the defined reward function. Subsequently, the sensed data are collected by the CHs and then sent to the autonomous underwater vehicles. In the final phase, the consumed energy by each sensor is calculated, and its residual energy is updated. Then, the autonomous underwater vehicle performs all clustering and CH selection operations. This procedure persists until the point of cessation when the sensor’s power has been reduced to such an extent that no node can become a CH. Through analysis of the findings from this investigation and their comparison with alternative frameworks, the implementation of this method can be used to control the cluster size and the number of CHs, which ultimately augments the energy usage of nodes and prolongs the lifespan of the network. Our simulation results illustrate that the suggested methodology surpasses the conventional low-energy adaptive clustering hierarchy, the distance- and energy-constrained K-means clustering scheme, and the vector-based forward protocol and is viable for deployment in an actual operational environment.

References:

[1] Akyildiz IF, Su W, Sankarasubramaniam Y, Cayirci E (2002) Wireless sensor networks: a survey. Computer Networks 38(4): 393-422. https://doi.org/10.1016/S1389-1286(01)00302-4
[2] Alsalman L, Alotaibi E (2021) A balanced routing protocol based on machine learning for underwater sensor networks. IEEE Access 9: 152082-152097. https://doi.org/10.1109/ACCESS.2021.3126107
[3] Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA (2017) Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine 34(6): 26-38. https://doi.org/10.1109/MSP.2017.2743240
[4] Balhara S, Gupta N, Alkhayyat A, Bharti I, Malik RQ, Mahmood SN, Abedi F (2022) A survey on deep reinforcement learning architectures, applications and emerging trends. IET Communications 2022: 1-16. https://doi.org/10.1049/cmu2.12447
[5] Bhattacharjya K, Alam S, De D (2019) CUWSN: energy efficient routing protocol selection for cluster based underwater wireless sensor network. Microsystem Technologies 28(2): 1-17. https://doi.org/10.1007/s00542-019-04583-0
[6] Bholowalia P, Kumar A (2014) EBK-means: A clustering technique based on elbow method and k-means in WSN. International Journal of Computer Applications 105(9): 17-24
[7] Das D, Ameer PM (2017) Energy efficient geographic clustered multi-hop routing for underwater sensor networks. TENCON 2017-2017 IEEE Region 10 Conference, 409-414. https://doi.org/10.1109/TENCON.2017.8227899
[8] Fattah S, Gani A, Ahmedy I, Idris MYI, Targio Hashem IA (2020) A survey on underwater wireless sensor networks: requirements, taxonomy, recent advances, and open research challenges. Sensors 20(18): 5393. https://doi.org/10.3390/s20185393
[9] Gupta S, Singh NP (2024) Underwater wireless sensor networks: a review of routing protocols, taxonomy, and future directions. The Journal of Supercomputing 80(4): 5163-5196. https://doi.org/10.1007/s11227-023-05646-w
[10] Han G, Shen S, Song H, Yang T, Zhang W (2018) A stratification-based data collection scheme in underwater acoustic sensor networks. IEEE Transactions on Vehicular Technology 67(11): 10671-10682. https://doi.org/10.1109/TVT.2018.2867021
[11] Heinzelman WB, Chandrakasan AP, Balakrishnan H (2002) An application-specific protocol architecture for wireless microsensor networks. IEEE Transactions on Wireless Communications 1(4): 660-670. https://doi.org/10.1109/TWC.2002.804190
[12] Hu T, Fei Y (2010) QELAR: A machine-learning-based adaptive routing protocol for energy-efficient and lifetime-extended underwater sensor networks. IEEE Transactions on Mobile Computing 9(6): 796-809. https://doi.org/10.1109/TMC.2010.28
[13] Javaid N, Hafeez T, Wadud Z, Alrajeh N, Alabed MS, Guizani N (2017) Establishing a cooperation-based and void node avoiding energy-efficient underwater WSN for a cloud. IEEE Access 5: 11582-11593. https://doi.org/10.1109/ACCESS.2017.2707531
[14] Jawhar I, Mohamed N, Al-Jaroodi J, Zhang S (2018) An architecture for using autonomous underwater vehicles in wireless sensor networks for underwater pipeline monitoring. IEEE Transactions on Industrial Informatics 15(3): 1329-1340. https://doi.org/10.1109/TII.2018.2848290
[15] Khalid M, Ullah Z, Ahmad N, Arshad M, Jan B, Cao Y, Adnan A (2017) A survey of routing issues and associated protocols in underwater wireless sensor networks. Journal of Sensors 7539751: 1-17. https://doi.org/10.1155/2017/7539751
[16] Khan A, Javaid N, Latif G, Jatta L, Fatima A, Khan W (2018) Cluster based and adaptive power controlled routing protocol for underwater wireless sensor networks. 21st Saudi Computer Society National Computer Conference (NCC), 1-6. https://doi.org/10.1109/NCG.2018.8592955
[17] Khan G, Gola KK, Ali W (2015) Energy efficient routing algorithm for UWSN-A clustering approach. Second International Conference on Advances in Computing and Communication Engineering, 150-155. https://doi.org/10.1109/ICACCE.2015.42
[18] Khan W, Wang H, Anwar MS, Ayaz M, Ahmad S, Ullah I (2019) A multi-layer cluster based energy efficient routing scheme for UWSNs. IEEE Access 7: 77398-77410. https://doi.org/10.1109/ACCESS.2019.2922060
[19] Khisa S, Moh S (2021) Survey on recent advancements in energy-efficient routing protocols for underwater wireless sensor networks. IEEE Access 9: 55045-55062. https://doi.org/10.1109/ACCESS.2021.3071490
[20] Kozlowski M, McConville R, Santos-Rodriguez R, Piechocki R (2018) Energy efficiency in reinforcement learning for wireless sensor networks. ArXiv Preprint ArXiv: 1812.02538. https://doi.org/10.48550/arXiv.1812.02538
[21] Li Y (2017) Deep reinforcement learning: An overview. ArXiv Preprint ArXiv: 1701.07274. https://doi.org/10.48550/arXiv.1701.07274
[22] Lu Y, He R, Chen X, Lin B, Yu C (2020) Energy-efficient depth-based opportunistic routing with q-learning for underwater wireless sensor networks. Sensors 20(4): 1025. https://doi.org/10.3390/s20041025
[23] Luo J, Chen Y, Wu M, Yang Y (2021) A survey of routing protocols for underwater wireless sensor networks. IEEE Communications Surveys & Tutorials 23(1): 137-160. https://doi.org/10.1109/COMST.2020.3048190
[24] Majid A, Azam I, Waheed A, Zain-ul-Abidin M, Hafeez T, Khan ZA, Qasim U, Javaid N (2016) An energy efficient and balanced energy consumption cluster based routing protocol for underwater wireless sensor networks. IEEE 30th International Conference on Advanced Information Networking and Applications (AINA), 324-333. https://doi.org/10.1109/AINA.2016.165
[25] Mazinani SM, Yousefi H, Mirzaie M (2018) A vector-based routing protocol in underwater wireless sensor networks. Wireless Personal Communications 100(4): 1569-1583. https://doi.org/10.1007/s11277-018-5654-0
[26] Nasir YS, Guo D (2019) Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks. IEEE Journal on Selected Areas in Communications 37(10): 2239-2250. https://doi.org/10.1109/JSAC.2019.2933973
[27] Nazareth P, Chandavarkar BR (2024) Cluster-based multi-attribute routing protocol for underwater acoustic sensor networks. Wireless Personal Communications 134(2): 781-808. https://doi.org/10.1007/s11277-024-10926-6
[28] Nguyen NT, Le TTT, Nguyen HH, Voznak M (2021) Energy-efficient clustering multi-hop routing protocol in a UWSN. Sensors 21(2): 627. https://doi.org/10.3390/s21020627
[29] Omeke KG, Mollel MS, Ozturk M, Ansari S, Zhang L, Abbasi QH, Imran MA (2021) DEKCS: A dynamic clustering protocol to prolong underwater sensor networks. IEEE Sensors Journal 21(7): 9457-9464. https://doi.org/10.1109/JSEN.2021.3054943
[30] Rahmani M, Dehghani MJ, Xiao P, Bashar M, Debbah M (2022) Multi-agent reinforcement learning-based pilot assignment for cell-free massive MIMO systems. IEEE Access 10: 120492-120502. https://doi.org/10.1109/ACCESS.2022.3221935
[31] Rowshanrad S, Keshtgary M, Javidan R (2014) MBC: A multi-hop balanced clustering routing protocol for wireless sensor networks. International Journal of Artificial Intelligence and Mechatronics 2(6): 164-170
[32] Shelar PA, Mahalle PN, Shinde GR, Bhapkar HR, Tefera MA (2022) Performance-aware green algorithm for clustering of underwater wireless sensor network based on optical signal-to-noise ratio. Mathematical Problems in Engineering 2022: 1-18. https://doi.org/10.1155/2022/1647028
[33] Souiki S, Hadjila M, Feham M (2015) Fuzzy based clustering and energy efficient routing for underwater wireless sensor networks. Int. J. of Comp. Netw. & Commun. (IJCNC) 7(2): 33-44. https://doi.org/10.5121/ijcnc.2015.7203
[34] Stojanovic M, Preisig J (2009) Underwater acoustic communication channels: Propagation models and statistical characterization. IEEE Communications Magazine 47(1): 84-89. https://doi.org/10.1109/MCOM.2009.4752682
[35] Sun Y, Zheng M, Han X, Li S, Yin J (2022) Adaptive clustering routing protocol for underwater sensor networks. Ad Hoc Networks 136: 102953. https://doi.org/10.1016/j.adhoc.2022.102953
[36] Sutton RS, Barto AG (2018) Reinforcement learning: An introduction. MIT Press
[37] Taheri R, Parsaei MR, Javidan R (2016) A new method for optimizing energy consumption in wireless sensor networks using enhanced LEACH protocol. Journal of Engineering and Applied Sciences 100(3): 576-581. https://doi.org/10.3923/jeasci.2016.576.581
[38] Wang C, Shen X, Wang H, Zhang H, Mei H (2023) Reinforcement learning-based opportunistic routing protocol using depth information for energy-efficient underwater wireless sensor networks. IEEE Sensors Journal 23(15): 17771-17783. https://doi.org/10.1109/JSEN.2023.3285751
[39] Wang K, Gao H, Xu X, Jiang J, Yue D (2015) An energy-efficient reliable data transmission scheme for complex environmental monitoring in underwater acoustic sensor networks. IEEE Sensors Journal 16(11): 4051-4062. DOI: 10.1109/JSEN.2015.2428712
[40] Wang M, Chen Y, Sun X, Xiao F, Xu X (2020) Node energy consumption balanced multi-hop transmission for underwater acoustic sensor networks based on clustering algorithm. IEEE Access 8: 191231-191241. https://doi.org/10.1109/ACCESS.2020.3032019
[41] Zhu F, Wei J (2018) An energy efficient routing protocol based on layers and unequal clusters in underwater wireless sensor networks. Journal of Sensors 2018: 1-10. https://doi.org/10.1155/2018/5835730

Memo

Memo:
Received date:2023-11-20;Accepted date:2024-10-14。<br>Corresponding author:Reza Javidan,E-mail:javidan@sutech.ac.ir
Last Update: 2025-08-27