ai
Distillation
Knowledge Distillation
Definition
Knowledge distillation is a model compression technique where a smaller student model is trained to mimic the output distribution of a larger teacher model. The student learns from soft probability distributions (teacher logits) rather than hard labels, transferring knowledge more efficiently.
Distillation is used to produce smaller, faster models that retain much of the teacher's accuracy.
Can this ship? Are we exposed? Can you prove it?
Every CISO is asking these three questions. We answer them with facts, not guesses, from repository to runtime.