Atricle

Info

Title:: Automated Ship Berthing Guidance Method Based on Three-dimensional Target Measurement

Author(s):: Yiming Ma¹; Chao Mi¹; 2; Lei Yao²; Yi Liu¹; Weijian Mi¹; 2

Affilations:

Keywords:: Automated ship; Automatic berthing; Berthing guidance; 3D measurement; Neural networks; Deep learning; Position estimation

DOI:: 10.1007/s11804-023-00336-8

Abstract:: Automatic berthing guidance is an important aspect of automated ship technology to obtain the ship-shore position relationship. The current mainstream measurement methods for ship-shore position relationships are based on radar, multisensor fusion, and visual detection technologies. This paper proposes an automated ship berthing guidance method based on three-dimensional (3D) target measurement and compares it with a single-target recognition method using a binocular camera. An improved deep object pose estimation (DOPE) network is used in this method to predict the pixel coordinates of the two-dimensional (2D) keypoints of the shore target in the image. The pixel coordinates are then converted into 3D coordinates through the camera imaging principle, and an algorithm for calculating the relationship between the ship and the shore is proposed. Experiments were conducted on the improved DOPE network and the actual ship guidance performance to verify the effectiveness of the method. Results show that the proposed method with a monocular camera has high stability and accuracy and can meet the requirements of automatic berthing.

References:

Akbar J, Shahzad M, Malik MI, Ul-Hasan A, Shafait F (2019) Runway detection and localization in aerial images using deep learning.In 2019 IEEE Digital Image Computing:Techniques and Applications (DICTA), 1-8.https://doi.org/10.1109/DICTA 47822.2019.8945889
Chang JR, Chen YS (2018) Pyramid stereo matching network.In Proceedings of the IEEE conference on computer vision and pattern recognition, 5410-5418.https://doi.org/10.1109/CVPR.2018.00567
Cheng Y, Li B (2021) Image segmentation technology and its application in digital image processing.In 2021 IEEE Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC), 1174-1177.https://doi.org/10.1109/IPEC51340.2021.9421206
Dickmann J, Appenrodt N, Klappstein J, Bloecher HL, Muntzinger M, Sailer A, Brenk C (2015) Making bertha see even more:Radar contribution.IEEE Access, 3:1233-1247.https://doi.org/10.1109/ACCESS.2015.2454533
Dwina N, Arnia F, Munadi K (2018) Skin segmentation based on improved thresholding method.In 2018 IEEE International ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI-NCON), 95-99.https://doi.org/10.1109/ECTI-NCON.2018.8378289
Fu MY, Xu YJ, Wang YH (2015) Cooperation and collision avoidance for multiple DP ships with disturbances.In 2015 IEEE 34th Chinese Control Conference (CCC), 4208-4213.https://doi.org/10.1109/ChiCC.2015.7260288
Girshick R, Donahue J, Darrell T, Malik J (2015) Region-based convolutional networks for accurate object detection and segmentation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(1):142-158.https://doi.org/10.1109/TPAMI.2015.2437384
Hui Y, Han Y (2019) Advanced density peak and K-means clustering on image segmentation.In 2019 IEEE 4th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), 1:2740-2744.https://doi.org/10.1109/IAEAC47372.2019.8997758
Lee SD, Tzeng CY, Kehr YZ, Huang CC, Kang CK (2010) Autopilot system based on color recognition algorithm and internal model control scheme for controlling approaching maneuvers of a small boat.IEEE Journal of Oceanic Engineering, 35(2):376-387.https://doi.org/10.1109/JOE.2010.2043753
Lepetit V, Moreno-Noguer F, Fua P (2009) EP n P:An accurate O (n) solution to the P n P problem.International Journal of Computer Vision, 81:155-166.https://doi.org/10.1007/s11263-008-0152-6
Li TY, Zhu HB (2018) Research on model control of binocular robot vision system.In 2018 IEEE Chinese Automation Congress(CAC), 1794-1797.https://doi.org/10.1109/CAC.2018.8623756
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD:Single shot multibox detector.In Proceedings of Computer Vision-ECCV 2016:14th European Conference, Amsterdam, Part I, 21-37.Springer International Publishing.https://doi.org/10.48550/arXiv.1512.02325
Mazenc F, Pettersen K, Nijmeijer H (2002) Global uniform asymptotic stabilization of an underactuated surface vessel.IEEE Transactions on Automatic Control, 47(10):1759-1762.https://doi.org/10.1109/TAC.2002.803554
Mi C, Huang Y, Fu C, Zhang Z, Postolache O (2021) Vision-based measurement:actualities and developing trends in automated container terminals.IEEE Instrumentation & Measurement Magazine, 24(4):65-76.https://doi.org/10.1109/MIM.2021.9448257
Morrical N, Tremblay J, Lin Y, Tyree S, Birchfield S, Pascucci V, Wald I (2021) NViSII:A scriptable tool for photorealistic image generation.arXiv preprint arXiv:2105.13962.https://doi.org/10.48550/arXiv.2105.13962
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once:Unified, real-time object detection.In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 779-788.https://doi.org/10.1109/CVPR.2016.91
Shao G, Ma Y, Malekian R, Yan X, Li Z (2019) A novel cooperative platform design for coupled USV-UAV systems.IEEE Transactions on Industrial Informatics, 15(9):4913-4922.https://doi.org/10.1109/TII.2019.2912024
Shuo J, Yonghui Z, Wen R, Kebin T (2017) The unmanned autonomous cruise ship for water quality monitoring and sampling.In 2017 IEEE International Conference on Computer Systems, Electronics and Control (ICCSEC), 700-703.https://doi.org/10.1109/ICCSEC.2017.8447040
Song Q, Lin GY, Ma JQ, Zhang HM (2016) An edge-detection method based on adaptive canny algorithm and iterative segmentation threshold.In IEEE 2016 2nd International Conference on Control Science and Systems Engineering (ICCSSE), 64-67.https://doi.org/10.1109/CCSSE.2016.7784354
Sun ZY, Wang LH, Liu LQ (2020) Three-dimensional reconstruction algorithm based on inverse perspective transformation.In 2020 IEEE International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), 221-225.https://doi.org/10.1109/ICBAIE49996.2020.00053
Tateno K, Tombari F, Laina I, Navab N (2017) CNN-SLAM:Real-time dense monocular slam with learned depth prediction.In Proceedings of the IEEE conference on computer vision and pattern recognition, 6243-6252.https://doi.org/10.1109/CVPR.2017.695
Tekin B, Sinha SN, Fua P (2018) Real-time seamless single shot 6d object pose prediction.In Proceedings of the IEEE conference on computer vision and pattern recognition, 292-301.https://doi.org/10.1109/CVPR.2018.00038
Tremblay J, To T, Sundaralingam B, Xiang Y, Fox D, Birchfield S (2018) Deep object pose estimation for semantic robotic grasping of household objects.arXiv preprint arXiv:1809.10790.https://doi.org/10.48550/arXiv.1809.10790
Xiang Y, Schmidt T, Narayanan V, Fox D (2017) Posecnn:A convolutional neural network for 6d object pose estimation in cluttered scenes.arXiv preprint arXiv:1711.00199.https://doi.org/10.48550/arXiv.1711.00199
Yu R, Zhu Q, Xia G, Liu Z (2012) Sliding mode tracking control of an underactuated surface vessel.IET Control Theory & Applications, 6(3):461-466.https://doi.org/10.1049/iet-cta.2011.0176
Zhao W, Zhang S, Guan Z, Zhao W, Peng J, Fan J (2020) Learning deep network for detecting 3d object keypoints and 6d poses.In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14134-14142.https://doi.org/10.1109/CVPR42600.2020.01414

Journal of Marine Science and Application[ISSN:1002-2848/CN:61-1400/f]

Info

References:

Memo

Commonly used function

Navigate

Tools

Statistics