Abstract: Landslide rockfall poses a significant threat as a common natural disaster, necessitating the development of efficient and accurate monitoring methods for slope safety. Conventional rockfall monitoring techniques based on video imagery often overlook the dynamic movement of fallen rocks. To address this issue, this study focuses on enhancing the YOLOv5s and DeepSort models using the DBT (Dynamic Background Template) framework, leading to the proposal of a rockfall tracking algorithm based on the M-DBT (Modified DBT) framework. At the framework level, an innovative motion detector is constructed by integrating the Background Matting algorithm into the original DBT framework. This integration forms the foundation of the M-DBT target tracking algorithm framework, which dynamically invokes the rockfall tracking algorithm based on motion detection results. The computational efficiency of the monitoring device is significantly improved. At the algorithm level, the YOLOv5s and DeepSort algorithms are optimized by incorporating the CBAM (Convolutional Block Attention Module) attention mechanism, ASPP (Atrous Spatial Pyramid Pooling) spatial pyramid pooling, SIoU (Scale-Invariant Intersection over Union) loss function, and Swin Transformer network. These enhancements augment the algorithms' capacity to extract rockfall features, enhance the accuracy of prediction boxes, and improve the precision of ID allocation for detected targets. The model is trained and evaluated using custom-built target detection and image classification datasets. Experimental results demonstrate the effectiveness of the rockfall tracking algorithm based on the M-DBT framework in significantly reducing computational resource consumption by up to 21.5% compared to the DBT-based algorithm. The improved YOLOv5s model exhibits an impressive 8.4% increase in detection accuracy, with the highest achieved rock detection accuracy reaching 87%. Similarly, the enhanced DeepSort model achieves an 8.9% improvement in tracking accuracy, with the highest attained tracking accuracy reaching 84%. The proposed rockfall tracking algorithm showcases strong practical value by enabling the detection and tracking of landslide rockfalls.