Publications
論文誌 (国際) An empirical study of retrieval-augmented diffusion language models for generative commonsense reasoning
Yubo Fang (University of Tsukuba), Hai-Tao Yu (University of Tsukuba), Hideo Joho (University of Tsukuba), Sumio Fujita, Yan Ge (University of Tsukuba), Chao Lei (University of Tsukuba)
Information Sciences
2026.9.11
Generative commonsense reasoning (GCR) remains challenging for intelligent systems, as large language models (LLMs) rely primarily on parametric knowledge and often lack sufficient commonsense information. While retrieval-augmented generation (RAG) alleviates this limitation by incorporating external knowledge, it has been predominantly studied in the context of autoregressive language models, which suffer from inherent limitations such as exposure bias and limited global refinement. In this work, we propose GRADi4GCR1, a generalized retrieval-augmented diffusion framework for generative commonsense reasoning. GRADi4GCR retrieves relevant commonsense knowledge from an external corpus and integrates it into the iterative diffusion-based generation process, enabling more effective knowledge utilization and global refinement during text generation. Experimental re- sults on two GCR benchmarks show that retrieval augmentation consistently improves generation quality across different diffusion architectures and scales. Additional analyses provide insights into generation diversity, fluency, and the impact of retrieval strategies on generation quality, along with qualitative case studies illustrating how GRADi4GCR performs GCR. These findings suggest that DLMs offer a promising alternative paradigm for retrievalaugmented commonsense reasoning.
Paper :
An empirical study of retrieval-augmented diffusion language models for generative commonsense reasoning
（外部サイト）