Distributing mixture-of-experts layers across devices so different experts run on different hardware.