Combining contrastive and triplet losses with semi-hard negatives on a lightweight encoder (MobileNetV3) effectively learns to match vocal imitations to sound effects, balancing accuracy with deployment efficiency.
This paper describes winning approaches for a challenge on finding sound effects by imitating them with your voice. The team tested two methods: one using a frozen pre-trained audio encoder with contrastive learning, and another using a mobile-friendly encoder trained with both contrastive and triplet losses. Both approaches help match vocal imitations to actual sound recordings.