Targeting individual neurons inside the audio encoder at inference time is more effective for improving acoustic perception than post-encoder interventions, suggesting that the encoder itself is where acoustic information gets lost in large audio-language models.
This paper introduces IAAN, a training-free method that identifies and amplifies specific neurons in audio encoders of large audio-language models to improve perception of fine-grained speech attributes like emotion.