While PyTorch Live remained the biggest announcement of the PyTorch Developer Day Conference 2021, there are many important highlights.