Publications
OTHERS (INTERNATIONAL) Effect of Model Merging in Domain-Specific Ad-hoc Retrieval
Taiga Sasaki (Hyogo University), Takehiro Yamamoto (Hyogo University), Hiroaki Ohshima (Hyogo University), Sumio Fujita
arXiv.org (arXiv)
September 26, 2025
In this study, we evaluate the effect of model merging in ad-hoc retrieval tasks. Model merging is a technique that combines the diverse characteristics of multiple models to create a new model. We hypothesized that applying model merging to domain-specific ad-hoc retrieval tasks could improve retrieval effectiveness. To verify this hypothesis, we merged the weights of a source retrieval model and a domain-specific (non-retrieval) model using a linear interpolation approach to construct a merged model. A key advantage of our approach is that it requires no additional fine-tuning of the models. We conducted two experiments each in the medical and Japanese domains. The first compared the merged model with the source retrieval model, and the second compared it with a LoRA fine-tuned model under both full and limited data settings for model construction. The experimental results indicate that model merging has the potential to produce more effective domain-specific retrieval models than the source retrieval model, and may serve as a practical alternative to LoRA fine-tuning, particularly when only a limited amount of data is available.
Paper :
Effect of Model Merging in Domain-Specific Ad-hoc Retrieval
(external link)