Abstract:
Reliable recognition of vessel mooring operations remains challenging because crew actions vary considerably, mooring lines are flexible and non-rigid, and floating bollards move with changes in water level. These factors lead to substantial variations in the relative positions and motion patterns of crew members, mooring lines, and bollards, making it difficult to determine the actual state of the mooring process through conventional visual monitoring. To address this problem, this study proposes a full-process vessel mooring recognition approach based on machine vision and human skeletal keypoint detection. The method combines spatial relationship analysis, crew action recognition, and mooring-line state detection to determine whether mooring conditions are satisfied and to determine the operational stage and completion status of the operation. First, the spatial relationships among the floating bollard, crew members, and mooring lines are dynamically detected and analyzed to determine whether the vessel has met the conditions required for mooring. By focusing on the relative spatial relationships among the main operating objects rather than on fixed absolute positions, the method can better accommodate the movement of vessels and the positional changes of floating bollards caused by water-level fluctuations. The entire mooring process is then divided into four stages: mooring preparation, mooring in progress, mooring completion, and unmooring. Human skeletal keypoints are detected for crew members during operation, and changes in keypoint positions and motion patterns are used to characterize crew actions. These motion features are compared with the criteria for the corresponding stage to determine whether the observed behavior is consistent with the expected operating procedure. In addition to crew actions, the state of the mooring line is incorporated into the recognition process. The system detects whether the mooring line is taut or slack and combines this information with the spatial relationships among the bollard, crew members, and mooring lines, as well as the changes in human skeletal keypoints. This joint analysis provides a more comprehensive basis for judging whether the vessel has properly completed the mooring operation in a standardized manner. By integrating multiple visual cues, the method reduces the limitations associated with relying on a single target or a single action feature and is better suited to practical waterway environments in which both human behavior and flexible objects exhibit large temporal and spatial variations. The approach also enhances the continuity of operation-state assessment throughout the vessel mooring and unmooring process. The results show that the proposed method significantly improves the recognition performance of vessel mooring operations. Compared with the original manual identification approach, the precision of mooring-recognition precision increased from 65.0% to 95.7%, corresponding to a relative improvement of 47.2%. Meanwhile, the average recognition time was reduced by 83.3%. When deployed on an edge-computing device, the method achieved an inference frame rate of 45 frames per second, indicating that it can meet the real-time processing requirements of field monitoring. These results demonstrate that the integrated use of spatial relationships, human skeletal keypoints, and mooring-line states can effectively improve the reliability and efficiency of full-process mooring recognition under complex operating conditions. A scenario-based analysis was further conducted using the 2022 operational data of Changzhou Ship Lock to estimate the potential operational benefits of the proposed method. The results indicate that navigation efficiency could increase by approximately 10.4%, while the average daily cargo throughput could increase by approximately 44,200 t. These estimates suggest that reducing manual identification time and improving the timeliness of mooring-state determination may contribute to higher lock operating efficiency and greater vessel-handling capacity. The proposed method can therefore provide technical support for intelligent recognition and safety supervision of vessel mooring and unmooring operations at ship locks and ports, and it also offers a reference for the coordinated recognition of human behavior and non-rigid objects in complex waterway environments.