You can dramatically improve few-step diffusion inpainting by initializing the noise with semantic information from the input image, rather than random noise—no retraining required.
InverFill speeds up image inpainting by using a smart noise initialization technique that preserves semantic information from the original image. Instead of training new models, it works with existing fast text-to-image models to fill in masked regions with better quality and fewer processing steps.