Research: Optimizing Knowledge Distillation Strategies for Efficient On-Device Deployme...

Research question: How can advanced knowledge distillation techniques improve the performance and efficiency of large language models when deployed on resource-constrained devices?

Background & Motivation: Large language models have demonstrated impressive capabilities across a range of tasks, but their computational demands make on-device deployment challenging for edge devices such as smartphones and IoT hardware. Knowledge distillation offers a promising avenue to transfer the capabilities of large models to smaller, more efficient ones.

Research Gap: While several distillation methods exist, there is limited systematic evaluation of their effectiveness specifically for on-device deployment. Key gaps include understanding which distillation strategies best preserve task performance and minimize inference latency under strict resource constraints.

Approach & Expected Contribution: This project will undertake a comparative study of state-of-the-art knowledge distillation techniques, including feature-based, response-based, and task-specific distillation, using benchmark datasets. Experiments will evaluate the resulting distilled models for accuracy, inference speed, and memory footprint on real edge devices. The thesis aims to recommend optimal distillation pipelines for practical deployment scenarios.

Why It Matters: Improving distillation for edge deployment can significantly expand AI accessibility, enabling real-time intelligent applications in healthcare, security, and user experience without reliance on cloud infrastructure.