Successive Capacity Growth: Task-Complexity-Driven Width and Depth Expansion for Vision Transformer Encoders in JEPA World Models

Frederik Berenz|August 27, 2026arXiv

Key Takeaway

Instead of pre-sizing neural network encoders at maximum capacity, you can start small and grow them incrementally as task complexity demands, achieving significant efficiency gains without sacrificing performance.

Summary

This paper introduces Successive Capacity Growth (SCG), a method that automatically expands Vision Transformer encoders in world models from minimal size upward, adding attention heads or layers only when needed to improve prediction accuracy.

architecture efficiency scaling

Key Terms

jepa function-preserving-expansion vision-transformer attention-head