Matching Tasks to Objectives: Fine-Tuning and Prompt-Tuning Strategies for Encoder-Decoder Pre-trained Language Models

Ahmad Pouramini, Hesham Faili|June 23, 2026arXiv

Key Takeaway

Matching a task's requirements to the model's pre-training objective—and maintaining that alignment through fine-tuning templates and prompts—dramatically improves performance, especially with limited data.

Summary

This paper shows how to better adapt pre-trained encoder-decoder language models by matching task requirements to pre-training objectives. The authors introduce the Match Task to Objective (MTO) framework, which automatically selects appropriate training objectives and designs aligned prompts/templates.

training

Key Terms

encoder-decoder-architecture prompt-engineering fine-tuning few-shot-learning pre-training