Publications
カンファレンス (国際) Overview of the NTCIR-13 Short Text Conversation Task
Lifeng Shang（Huawei）,Tetsuya Sakai（Waseda University）,Hang Li（Toutiao）,Ryuichiro Higashinaka（NTT Media Intelligence Laboratories）,Yusuke Miyao（National Institute of Informatics）,Yuki Arase（Osaka University）,and Masako Nomoto
The 13th NTCIR Conference (NTCIR-13)
2017.12.5
We give an overview of the NII Testbeds and Community for Information access Research (NTCIR)-13 Short Text Conversation (STC) task, which was a core task of NTCIR-13. At NTCIR-12, STC was taken as an IR problem by maintaining a large repository of post-comment pairs then finding a clever method of reusing these existing comments to respond to new posts. At NTCIR-13, besides the retrieval-based method, we focused on a new method called generation-based method to generate ``new'' comments. The generation-based method has gained a great deal of attention in recent years, even though there the problem still remains of whether the retrieval-based method should be wholly replaced with or combined with the generation-based method for the STC task. By organizing this task at NTCIR-13, we provided a transparent platform to compare the two aforementioned methods by conducting comprehensive evaluations. For the Chinese subtask, there were a total of 34 registrations, and 22 teams finally submitted 120 runs. For the Japanese subtask, there were a total of 9 registrations, and 5 teams submitted 15 runs. In this paper, we review the task definition, evaluation measures, test collections, and evaluation results of all teams.
Paper :
Overview of the NTCIR-13 Short Text Conversation Task
（外部サイト）