ACAM-KD: Making Knowledge Distillation Adaptive | Nebius Science Paper Club

How can a smaller model learn more effectively from a larger one? Knowledge distillation transfers knowledge from a teacher model to a compact student, but the features selected for transfer may not reflect what the student currently needs.

Join the Nebius Science Paper Club for a session with Qing Tian, Assistant Professor of Computer Science at the University of Alabama at Birmingham, presenting “ACAM-KD: Adaptive and Cooperative Attention Masking for Knowledge Distillation” (ICCV 2025).

ACAM-KD brings teacher and student signals together through cross-attention, then learns spatial and channel masks that adapt throughout training. We’ll explore how this approach improves knowledge transfer for object detection and semantic segmentation without adding inference cost to the student model.

Talk followed by Q&A and open discussion.

Read the paper before the webinar →

About Nebius Science Paper Club

Nebius Science Paper Club is a webinar series led by Nebius researchers. Each session brings together paper authors and practitioners to discuss new ideas, research, and discoveries in AI.

The session is open to researchers, engineers, students, and anyone interested in knowledge distillation, model compression, and computer vision.