Knowledge Distillation with Keras*| AI News

  • Overview
  • Transcript

Learn about the concept of knowledge distillation, a process where a large and complex network is trained while extracting important features from the given data. It can produce better predictions, and the ways it can improve deep learning model performance on mobile devices.

Read the article

Subscribe to the Intel Software YouTube Channel

Stay up to date with the AI News YouTube Playlist

I'm David Shaw. In this episode of AI News we look at the concept of knowledge distillation and the ways it can improve deep learning model performance on mobile devices. Knowledge distillation is a process where a large and complex network is trained while extracting important features from the given data. It can produce better predictions. Let's suppose we train a small network with the help of a complex model. This network will be able to produce comparable results and in some cases, even be capable of replicating the results of a more cumbersome network. 

For example, GoogleNet is complex. Its deepness gives the ability to extract features. And it has the power to remain accurate but at a cost. The model is heavy and needs huge amounts of memory and a powerful GPU to perform large calculations. That's why we need to transfer the knowledge learned by this model to a smaller model that can be easily used on a mobile device. Read this article to learn more about how bulky models can be used to create lighter models by using knowledge distillation. 

Thanks for watching. Don't forget to view the links and learn more. And I will see you next week.