Publications
CONFERENCE (INTERNATIONAL) Learning Disentangled Document Representations Based on a Classical Shallow Neural Encoder
Yuro Kanada (Shizuoka university), Sumio Fujita, Yoshiyuki Shoji (Shizuoka university)
The 27th International Conference on Information Integration and Web Intelligence (iiWAS 2025)
December 08, 2025
This paper proposes a document embedding method designed to obtain disentangled distributed representations. The resulting representations are expected to satisfy two key criteria: independence across dimensions and semantic interpretability of each dimension. We enhanced a classic shallow neural network-based embedding model with two modifications: 1) guidance task integration, where the network is trained to perform both a simple auxiliary metadata prediction task and a surrounding term prediction task simultaneously, and 2) loss regularization for independence, where the loss function includes both prediction accuracy and the independence across dimensions (i.e., the Kullback- Leibler divergence from a multivariate normal distribution). We evaluated the proposed method through both automatic and human-subject experiments using synthetic datasets and movie review texts. Experimental results show that even shallow neural networks can generate disentangled representations when dimensional independence is explicitly promoted.
Paper :
Learning Disentangled Document Representations Based on a Classical Shallow Neural Encoder
(external link)