|Table of Contents|

Citation:
　Yifan Qiu,Xiaoyu Yang,Feng Tong,et al.Evaluation of Reinforcement Learning-Based Adaptive Modulation in Shallow Sea Acoustic Communication[J].Journal of Marine Science and Application,2026,(1):292-299.[doi:10.1007/s11804-025-00613-8]
Click and Copy

Evaluation of Reinforcement Learning-Based Adaptive Modulation in Shallow Sea Acoustic Communication

Info

Title:
Evaluation of Reinforcement Learning-Based Adaptive Modulation in Shallow Sea Acoustic Communication
Author(s):
Yifan Qiu1; 2;  Xiaoyu Yang1; 2;  Feng Tong1; 2;  Dongsheng Chen1; 2
Affilations:
Author(s):
Yifan Qiu1; 2;  Xiaoyu Yang1; 2;  Feng Tong1; 2;  Dongsheng Chen1; 2
1. College of Ocean and Earth Sciences, Xiamen University, Xiamen 361100, China;
2. National and Local Joint Engineering Research Center for Navigation and Location Service Technology, Xiamen 361100, China
Keywords:
Adaptive modulation; Shallow sea underwater acoustic modulation; Reinforcement learning
分类号:
-
DOI:
10.1007/s11804-025-00613-8
Abstract:
While reinforcement learning-based underwater acoustic adaptive modulation shows promise for enabling environment-adaptive communication as supported by extensive simulation-based research, its practical performance remains underexplored in field investigations. To evaluate the practical applicability of this emerging technique in adverse shallow sea channels, a field experiment was conducted using three communication modes: orthogonal frequency division multiplexing (OFDM), M-ary frequency-shift keying (MFSK), and direct sequence spread spectrum (DSSS) for reinforcement learning-driven adaptive modulation. Specifically, a Q-learning method is used to select the optimal modulation mode according to the channel quality quantified by signal-to-noise ratio, multipath spread length, and Doppler frequency offset. Experimental results demonstrate that the reinforcement learning-based adaptive modulation scheme outperformed fixed threshold detection in terms of total throughput and average bit error rate, surpassing conventional adaptive modulation strategies.

References:

[1] Abdallah S, Kaisers M (2016) Addressing environment non-stationarity by repeating Q-learning updates. Journal of Machine Learning Research 17(46): 1-31
[2] Barua S, Rong Y, Nordholm S, Chen P (2022) Real-time adaptive modulation schemes for underwater acoustic OFDM communication. Sensors 22(9): 3436. https://doi.org/10.3390/s22093436
[3] Benson A, Proakis J, Stojanovic M (2000) Towards robust adaptive acoustic communications. In: OCEANS 2000 MTS/IEEE Conference and Exhibition. Conference Proceedings (Cat. No. 00CH37158). Piscataway: IEEE, 1243-1249
[4] Bhopale P, Kazi F, Singh N (2019) Reinforcement learning based obstacle avoidance for autonomous underwater vehicle. Journal of Marine Science and Application 18: 228-238. https://doi.org/10.1007/s11804-019-00089-3
[5] Bulut S, Ergin S (2021) Effects of temperature, salinity, and fluid type on acoustic characteristics of turbulent flow around circular cylinder. Journal of Marine Science and Application 20: 213-228. https://doi.org/10.1007/s11804-021-00197-z
[6] Fan C, Wang Z (2021) Adaptive switching for multimodal underwater acoustic communications based on reinforcement learning. In: The 15th International Conference on Underwater Networks & Systems. ACM, Shenzhen 1-2
[7] Fu Q, Song A (2018) Adaptive modulation for underwater acoustic communications based on reinforcement learning. In: OCEANS 2018 MTS/IEEE Charleston. Piscataway: IEEE, 1-8
[8] Gussen CMG, Diniz PSR, Campos MLR, Martins WA, Costa FM, Gois JN (2016) A survey of underwater wireless communication technologies. Journal of Communication and Information Systems 31(1): 242-255. https://doi.org/10.14209/jcis.2016.22
[9] Huang L, Wang Y, Zhang Q, Han J, Tan W, Tian Z (2022) Machine learning for underwater acoustic communications. IEEE Wireless Communications 29(3): 102-108. https://doi.org/10.1109/MWC.2020.2000284
[10] Huang J, Diamant R (2020) Adaptive modulation for long-range underwater acoustic communication. IEEE Transactions on Wireless Communications 19(10): 6844-6857. https://doi.org/10.1109/TWC.2020.3006230
[11] Huang JG, Wang H, He CB, Zhang QF, Jing LY (2018) Underwater acoustic communication and the general performance evaluation criteria. Frontiers of Information Technology & Electronic Engineering 19: 951-971. https://doi.org/10.1631/FITEE.1700775
[12] Li B, Zheng S, Tong F (2019a) Bit-error rate based Doppler estimation for shallow water acoustic OFDM communication. Ocean Engineering 182: 203-210. https://doi.org/10.1016/j.oceaneng.2019.04.045
[13] Li G, Wu J, Tang T, Chen Z, Chen J, Liu H (2019b) Underwater acoustic time delay estimation based on envelope differences of correlation functions. Sensors 19(5): 1259. https://doi.org/10.3390/s19051259
[14] Mani S, Duman TM, Hursky P (2008) Adaptive coding/modulation for shallow-water UWA communications. Journal of the Acoustical Society of America 123: 3749-3749. https://doi.org/10.1121/1.2935305
[15] Radosevic A, Ahmed R, Duman TM, Proaki JG, Stojanovic M (2014) Adaptive OFDM modulation for underwater acoustic communications: design considerations and experimental results. IEEE Journal of Oceanic Engineering 39(2): 357-370. https://doi.org/10.1109/JOE.2013.2253212
[16] Stojanovic M, Preisig J (2009) Underwater acoustic communication channels: Propagation models and statistical characterization. IEEE Communications Magazine 47(1): 84-89. https://doi.org/10.1109/MCOM.2009.4752682
[17] Su W, Lin J, Chen K, Xiao L, Em C (2019) Reinforcement learning-based adaptive modulation and coding for efficient underwater communications. IEEE Access 7: 67539-67550. https://doi.org/10.1109/ACCESS.2019.2918506
[18] Su Y, Liu Y, Fan R, et al (2022) A cooperative jamming scheme based on node authentication for underwater acoustic sensor networks. Journal of Marine Science and Application 21: 197-209. https://doi.org/10.1007/s11804-022-00277-8
[19] Tang N, Zeng Q, Luo D, Xu Q, Hu H (2020) Research on development and application of underwater acoustic communication system. Journal of Physics: Conference Series 1617: 012036. https://doi.org/10.1088/1742-6596/1617/1/012036
[20] Sweta T, Ruthrapriya S, Sneka J, John SRA, Rohith G, Mangal D (2024) Reinforcement learning-based automated modulation switching algorithm for an enhanced underwater acoustic communication. Results in Engineering 23: 102791. https://doi.org/10.1016/j.rineng.2024.102791
[21] Wan L, Wang Z, Zhou S, Yang TC, Shi Z (2012) Performance comparison of doppler scale estimation methods for underwater acoustic OFDM. Journal of Electrical and Computer Engineering 2012: 1-11. https://doi.org/10.1155/2012/703243
[22] Wan L, Zhou H, Xu X, Huang Y, Zhou S, Shi Z (2015) Adaptive modulation and coding for underwater acoustic OFDM. IEEE Journal of Oceanic Engineering 40(2): 327-336. https://doi.org/10.1109/JOE.2014.2323365
[23] Wang C, Wang Z, Sun W, Fuhrmann DR (2018) Reinforcement learning-based adaptive transmission in time-varying underwater acoustic channels. IEEE Access 6: 2541-2558. https://doi.org/10.1109/ACCESS.2017.2784239
[24] Zhang Y, Zhang Z, Chen L, Wang X (2021) Reinforcement Learning-Based Opportunistic Routing Protocol for Underwater Acoustic Sensor Networks. IEEE Transactions on Vehicular Technology 70(3): 2756-2770. https://doi.org/10.1109/TVT.2021.3058282
[25] Zhu Z, Tong F, Zhou Y, Zhang Z, Zhang F (2023) Deep learning prediction of time-varying underwater acoustic channel based on LSTM with attention mechanism. Journal of Marine Science and Application 22: 650-658. https://doi.org/10.1007/s11804-023-00347-5

Memo

Memo:
Received date:2024-10-10;Accepted date:2024-12-2。<br>Foundation item:The authors are grateful for funding from the National Key Research and Development Program of China (No. 2018YFE0110000), the National Natural Science Foundation of China (No. 11274259, No. 11574258), and the Science and Technology Commission Foundation of Shanghai (21DZ1205500) in support of the present research.<br>Corresponding author:Feng Tong,Email:E-mail:ftong@xmu.edu.cn
Last Update: 2026-03-09