A compact, efficient embedding specialist that punches above its weight for its size. Funnel Transformer's small variant uses a funnel-shaped attention mechanism that progressively compresses sequence representations, making it faster and leaner than traditional transformers of comparable depth. It trades some representational richness for speed and memory efficiency.