Abstract:Weight pruning is widely advocated for deploying Large Language Models on resource-constrained IoT and edge devices, yet its impact on model fairness remains poorly understood. We conduct a controlled empirical study of three instruction-tuned models (Gemma-2-9b-it, Mistral-7B-Instruct-v0.3, Phi-3.5-mini-instruct) across three pruning methods (Random, Magnitude, Wanda) at four sparsity levels (10-70%) on 12,148 BBQ bias benchmark items with 5 random seeds, totaling 2,368,860 inference records. Our results reveal a Smart Pruning Paradox: activation-aware pruning (Wanda) preserves perplexity nearly perfectly (just 3.5% increase at 50% sparsity for Mistral-7B), yet produces the highest bias amplification, with Stereotype Reliance Score increasing 83.7% and 47-59% of previously unbiased items developing new stereotypical behaviors at 70% sparsity. Random pruning destroys language capability entirely (perplexity exceeding $10^4$ and reaching $10^8$) but produces only random-chance bias. We further show that unstructured pruning provides zero storage savings and zero inference latency reduction on real edge hardware, undermining the primary motivation for its use in IoT deployment. Of 180 dense-vs-pruned comparisons, 141 (78.3%) are significant ($p < 0.05$) with mean $|h| = 0.305$. Published quantization studies report up to 21% of responses flipping between biased and unbiased states; our pruning results show transition rates nearly three times higher (47-59%), suggesting pruning poses a categorically greater risk to alignment than quantization. These findings demonstrate that perplexity-based evaluation provides false assurance of behavioral equivalence, and that IoT deployment pipelines require bias-aware validation before deploying pruned models at the edge.
| Comments: | 8 pages, 7 figures, 8 tables. Accepted at the 7th Annual World AIIoT Congress (AIIoT 2026). This is the author's accepted version; the version of record will appear in IEEE Xplore |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computers and Society (cs.CY) |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2605.08137 [cs.LG] |
| (or arXiv:2605.08137v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.08137 arXiv-issued DOI via DataCite |
Submission history
From: Plawan Kumar Rath [view email]
[v1]
Sat, 2 May 2026 05:27:40 UTC (1,433 KB)