knowledge-distillation

github:zechenzhangAGI/AI-research-SKILLs · first committed 2025-11-19 · source · badge

Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.

Skill Health: not yet scored

Listed, not yet scored. This skill has no public model test yet. It is in the scoring queue; submitted skills go first, then the most-installed.

Version history

5 commits to this file on GitHub (newest first).

2025-11-20 · commit 0a9ded4 · Standardize skill metadata: Add category tags and update authors
2025-11-20 · commit 51fc3a3 · Trigger upload of remaining 32 skills batch 2
2025-11-20 · commit 81c38fe · Upload all remaining 34 skills to Orchestra marketplace
2025-11-19 · commit bd4461f · Standardize YAML frontmatter for all 54 skills
2025-11-19 · commit 66e31d2 · Add 6 Emerging Techniques skills with comprehensive references

Archived snapshots