Robust MMSE video decoding: Theory and practical implementations

Abstract

A novel video decoding algorithm based on the minimum mean square error (MMSE) criterion is investigated in this research. To alleviate the effect of transmission errors, we first develop an error propagation model to estimate and track the mean square error (MSE) of each pixel in the decoder. Then, the proposed video decoding algorithm adjusts the reconstruction of each pixel adaptively according to fluctuating channel conditions. More specifically, the decoder reconstructs a pixel in the kth frame F-k by using a weighted sum of two pixels in frames Fk-1 and Fk-2, respectively, where their weights are adaptively selected to minimize the MSE of the reconstructed pixel by using the error propagation model. Extensive simulation results performed on standard H.263 bit streams demonstrate that the MMSE-based concealment algorithm yields a better performance than the conventional method, even if the encoder transmits a single motion vector per block. Moreover, the proposed MMSE decoding algorithm significantly enhances the error resilient capability of the double-vector motion compensation (DMC) algorithm, where two motion vectors are sent per block.