You can extend diffusion models to generate much longer text without retraining by mixing local attention with long-range context retrieval and adaptive denoising strategies.
RangeSWA enables diffusion language models to generate longer text without retraining by combining sliding-window attention with remote summaries and dynamic retrieval. It fixes common problems like repetition and quality loss when generating beyond training length through positional remapping and adaptive scheduling.