Transformer-Convolution Hybrid U-Shape Neural Network for High-Quality Monte Carlo Denoising

Abstract

In this thesis, we design a U-shape transformer-convolution hybrid neural network for the Monte Carlo denoising task. Our proposed model introduces additional convolutional blocks to the transformer-based network as a hybrid approach. The key motivation of this design is to continuously propagate, activate, and refine local detail information from the rendering features, using convolutional operations. A transformer block and a convolution block in our proposed model form a transformer-convolution hybrid residual block. Additionally, our model adopts a U-shape transformer architecture to exploit inter-window dependencies with local window self-attention while reducing the computational cost. Thanks to these designs, our proposed denoiser can better utilize auxiliary features from Monte Carlo-rendered images, and outperforms state-of-the-art Monte Carlo denoisers for various test scenes, even with faster inference latency.