Human-efficient annotation system via active semi-supervised learning and selective review

Abstract

Constructing training datasets for perception systems in autonomous driving, underwater exploration, and robotics requires large volumes of pixel-level instance annotations, a cost that grows with scene complexity and dataset size. Recent annotation platforms use foundation models for mask generation and active learning for sample selection, but they train their instance segmentation models only on labeled data. This leaves the unlabeled pool unused and the confidence scores miscalibrated under class imbalance. Addressing both problems requires combining semi-supervised learning (SSL) and calibration, whose interaction is rarely quantified. To reduce human annotation effort while keeping confidence-based review decisions reliable, we propose an iterative annotation system for instance segmentation. The system integrates semi-supervised training, difficulty-aware sample selection, and frequency-aware calibrated selective review. It triages predictions by their calibrated confidence into discard, auto-accept, or human review. In a controlled annotation experiment on Cityscapes with Mask2Former and the Segment Anything Model 2 (SAM 2), the system reduced human annotation time by 1.9× over an active learning baseline and 3.2× over manual labeling under the same 150-image budget. It improved mask Average Precision (AP) by 1.8 and 4.5 points over the two baselines, respectively. At the final round, it auto-accepted 74.3% of the retained predictions at 85.8% precision. An offline ablation showed that SSL and calibration jointly reduced false positives by 40.4%. Auto-accept precision also rose across rounds. An oracle-based test on the underwater dataset UIIS10K reproduced the AP and auto-accept gains. Coupling semi-supervised training with selective review lowers annotation effort while keeping the auto-accepted labels reliable. © 2026