Tutel is an implementation of the mixture-of-experts technique for large-scale DNN model training.