Abstract:Large language models increasingly require specialization across diverse domains, yet existing approaches struggle to balance multi-domain capacities with strict memory and inference constraints. In this work, we introduce SkillWeave, a modular improvement framework that enables LLMs to specialize under fixed memory budgets. SkillWeave partitions full capabilities of a general-purpose model into skillpacks -- lightweight, domain-specific delta modules -- that reorganize and refine the model's internal knowledge. For efficient deployment, SkillWeave integrates SkillZip to compress skillpacks into compact and inference-ready format, enabling strong multi-domain performance with low-latency execution. On multi-task and agentic benchmarks, a 9B SkillWeave model outperforms several baselines and even surpasses a 32B monolithic LLM, while achieving up to 4x speedup.
| Comments: | Accepted by ACL2026 |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.22205 [cs.AI] |
| (or arXiv:2605.22205v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.22205 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Guodong Du [view email]
[v1]
Thu, 21 May 2026 09:12:20 UTC (2,054 KB)