Decoder expressivity matters: simpler decoders with latent-space objectives produce better transferable geometric representations than complex pixel-space decoders, even in self-supervised settings.
This paper shows that Novel View Synthesis can learn strong 3D geometric representations if you constrain the decoder and use latent-space reconstruction instead of pixel-level targets. The authors introduce SNAP, which learns viewpoint-invariant features useful for localization, pose estimation, depth, and robot tasks—without needing explicit 3D supervision.