In this article, we will take a look at knowledge distillation and will discuss its context briefly.