Abstract: Developing a highly precise deformation prediction model for concrete dams is crucial for assessing the structural integrity of the dam. However, the deformation monitoring data exhibits complex non-linear and non-smooth characteristics that hinder the accuracy and generalization capability of prediction models. To address these challenges, this study introduces the Whale Optimization Algorithm (WOA) and the Envelope Entropy Theory Adaptive Optimization seeking Variational Modal Decomposition (VMD) parameters. These techniques are employed to decompose the deformation data into multiple scales by identifying the optimal combination of parameters, thereby obtaining Intrinsic Mode Functions (IMFs) at different characteristic scales. Subsequently, the decomposed components are reconstructed and utilized as inputs for the Extreme Gradient Boosting (XGBoost) model for individual predictions. The final predicted values are obtained by aggregating the results from each prediction. Using the deformation monitoring data from the Shankouyan crushed concrete arch dam in China, a comparative study is conducted to assess the accuracy and generalization ability of four models: Support Vector Regression Machine (SVR), Random Forest (RF), XGBoost, and WOA-VMD-XGBoost. The findings indicate that the combined model effectively captures the multi-scale features of the deformation signal, mitigates the impact of non-linearity and non-smoothness on model performance, and exhibits superior accuracy and generalization capability compared to individual prediction models. The combined model offers a theoretical foundation and practical guidance for dam deformation monitoring.