Publications
カンファレンス (国際) Session-Guided Pre-training for Query Understanding: Constraining Mask Prediction and Improving Query Fine-tunability
Sosuke Nishikawa (The University of Tokyo/LY Corporation), Naoki Yoshinaga (Institute of Industrial Science, The University of Tokyo)
The 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026)
2026.10.24
Large-scale search systems must process vast numbers of diverse, short queries for query understanding (QU) tasks such as named entity recognition (NER) and query classification, under strict latency constraints. Yet general-purpose language models (LMs), trained largely on context-rich open-domain text, are poorly matched to context-poor queries; large generative LMs also incur high serving costs. To achieve accurate and efficient QU, we introduce SEG-QU, a session-guided continual pre-training framework that tailors standard encoder pre-training objectives to search queries. It jointly optimizes token- and query-level objectives over session-derived query pairs: Session-Guided Masked Language Modeling (SG-MLM) reduces uncertainty in single-query MLM, while Same-Session Query Prediction (SSQP) improves downstream fine-tunability. On commercial-search NER and classification tasks, SEG-QU outperforms encoder baselines and fine-tuned generative LMs up to 43× larger, while requiring substantially lower latency and energy. SEG-QU also transfers effectively to a public e-commerce benchmark. Our code is publicly available at https://github.com/lycorp-jp/segqu.
Paper :
Session-Guided Pre-training for Query Understanding: Constraining Mask Prediction and Improving Query Fine-tunability
（外部サイト）
Software :
https://github.com/lycorp-jp/segqu
（外部サイト）