Pretrained tabular models already capture dataset-level structure useful for representation learning; adding a simple readout layer and contrastive training makes them effective dataset encoders without task-specific training.
TabMF repurposes pretrained tabular foundation models as dataset encoders by adding a dataset-level readout and contrastive post-training. This enables creating compact numerical representations of entire datasets for tasks like dataset retrieval and monitoring distribution shifts, outperforming hand-crafted features and other learned approaches.