Functional Attention replaces token-wise attention with function-space mappings, enabling transformer-like models to handle continuous fields more naturally and work reliably across different input resolutions.
This paper introduces Functional Attention, a new way to process continuous data (like PDEs or 3D shapes) by treating attention as mappings between function spaces rather than discrete tokens. Instead of softmax attention, it uses structured linear operators inspired by geometric functional maps, making the model work consistently across different resolutions and discretizations.