Homebrewed SLMs: Autoregressive and Masked Diffusion Language Models Built at Home for Portuguese

Key Takeaway

Small language models trained on consumer hardware can match or exceed much larger models on specialized domains, but memorization and domain mismatch remain key challenges—this work provides practical benchmarks and a reusable toolkit for building domain-specific SLMs.