A transformer model designed to function as a valid language model at every intermediate layer depth, not just the full model.