Soybean yield estimation and lodging classification based on UAV multi-source data and self-supervised contrastive learning

Unmanned aerial vehicle (UAV) platforms are increasingly used to obtain plant phenotypes in crop breeding for their efficiency and versatility. A lightweight UAV was used to collect high-precision RGB images, multispectral and point cloud data of soybeans (Glycine max (L.) Merr.) across fields at various growth stages, utilizing an innovative cross-circling oblique (CCO) route. A multi-modal data fusion deep learning model was proposed based on the self-supervised contrastive learning strategy with fine-tuning for yield estimation and lodging discrimination in soybean germplasm resources. During the soybean growth stages of flowering (R1) to maturity (R8), the contrastive learning effectively captured the decoupling characteristics of different soybean varieties in the feature space. Higher accuracy in yield estimation was obtained combined contrastive learning with the traditional features. Correlations were significantly reduced between features among varieties (Pearson's mean 0.27–0.62) and feature separations were achieved after dimension reduction (R8: CH = 12.4, DB = 51.8). RMSE of yield estimation was 591.39 kg ha⁻¹ at high density and 532.75 kg ha⁻¹ at low density at R8 growth stages. Lodging discrimination achieved the highest accuracy with an F1-score of 0.57 at high density and 0.64 at low density. The results demonstrated that utilizing contrastive learning for extraction of deep soybean features holds significant potential in supporting traditional features for yield estimation and lodging discrimination.