Sparse Fine-tuning for Inference Acceleration of Large Language Models thumbnail
Sparse Fine-tuning for Inference Acceleration of Large Language Models
arxiv.org
We consider the problem of accurate sparse fine-tuning of large language models (LLMs), that is, fine-tuning pretrained LLMs on specialized tasks, while inducing sparsity in their weights. On the accuracy side, we observe that standard loss-based fine-tuning may fail to recover accuracy, especially
1 Users
0 Comments
1 Highlights
1 Notes

Top Highlights

  • We consider the problem of accurate sparse fine-tuning of large language models (LLMs), that is, fine-tuning pretrained LLMs on specialized tasks, while inducing sparsity in their weights. On the accuracy side, we observe that standard loss-based fine-tuning may fail to recover accuracy, especially at high sparsities. To address this, we perform a ...

Tags

AI
LLM
finetuning
sparce fine-tuning

Domain

Ready to highlight and find good content?

Glasp is a social web highlighter that people can highlight and organize quotes and thoughts from the web, and access other like-minded people’s learning.