Depth-Aware Expert Masking Beats Uniform Pruning for MoE Model Compression
TL;DR: Masking low‑magnitude experts only in the deepest Mixture‑of‑Experts (MoE) layers preserves up to 84 % of output quality while disabling ≈ 6 % of total experts, a far superi