ScaleMatch: Multi-scale Consistency Enhancement for Semi-supervised Semantic Segmentation

Abstract Semi-supervised learning improves semantic segmentation performance by leveraging unlabeled data, thereby significantly reducing labeling costs. Previous semi-supervised semantic segmentation (S4) methods explored perturbations at the image level but neglected to adequately utilize multi-scale information. When labeled information is insufficient, the scale variation between different objects makes learning instances with extreme scales even more difficult. To address this issue, we propose ScaleMatch, which aims to learn scale-invariant features by obtaining a mixed dual-scale pseudo-label and scale consistency learning. Specifically, the cross-scale interaction fusion (CIF) module enforces interactive information across different scaled-views, allowing for more reliable pseudo-label generation. More importantly, ScaleMatch introduces variable scale branches to utilize scale-invariant supervision. It consists of image-level scale variation consistency (ISVC) and feature-level scale variation consistency (FSVC). Consequently, our ScaleMatch enhances the model's generalization under scale variation, outperforming existing state-of-the-art methods on both the Pascal VOC and Cityscapes datasets under various partition protocols.