Layer-Centric Factors of Variation Disentanglement for Task- and Model-Agnostic Generalization

Abstract

Disentanglement learning aims to separate the underlying factors of variation (FoV) to improve generalization. However, most FoV-based latentvector-centric methods impose objective-driven constraints at a bottleneck, and it is difficult to translate disentanglement into consistent gains on downstream tasks without inductive bias. Motivated by architectural approaches complementary to vector-centric objectives for downstream tasks, we propose the Orthogonal Subspaces Projection (OSP) layer, a plug-and-play module that integrates into intermediate layers and promotes FoV separation by projecting latent features into mutually orthogonal subspaces. Across diverse domains and tasks, models equipped with the OSP layer improve disentanglement quality and generalization in downstream tasks, including computer vision (classification, etection, and segmentation), natural language processing (word analogy, and text classification), and fine-tuning settings on large backbones.