Generalizable Depth Perception via Foundation Model

Abstract

Monocular depth foundation models have recently advanced significantly, yet their application to depth perception tasks utilizing sensor-derived measurements remains underexplored. This paper introduces two methodologies for integrating depth foun- dation models into generalizable depth perception frameworks. The first employs the foundation model as a teacher to generate high-quality pseudo dense labels for depth enhancement, using a robust scale alignment technique to correct inherent scale dis- crepancies between monocular predictions and sensor measurements. The second ap- plies test-time visual prompt tuning, allowing dynamic adaptation to new sensor data without altering pretrained parameters, reducing computational costs while preserv- ing strong generalization. Extensive experiments across multiple datasets demonstrate superior performance of both methods over existing approaches, highlighting their po- tential to enhance depth perception by leveraging the rich knowledge embedded in monocular depth models.