Department of Computer Science
Learning topic models by belief propagation
Latent Dirichlet allocation (LDA) is an important hierarchical Bayesian model for probabilistic topic modeling, which attracts worldwide interest and touches on many important applications in text mining, computer vision and computational biology. This paper represents the collapsed LDA as a factor graph, which enables the classic loopy belief propagation (BP) algorithm for approximate inference and parameter estimation. Although two commonly used approximate inference methods, such as variational Bayes (VB) and collapsed Gibbs sampling (GS), have gained great success in learning LDA, the proposed BP is competitive in both speed and accuracy, as validated by encouraging experimental results on four large-scale document datasets. Furthermore, the BP algorithm has the potential to become a generic scheme for learning variants of LDA-based topic models in the collapsed space. To this end, we show how to learn two typical variants of LDA-based topic models, such as author-topic models (ATM) and relational topic models (RTM), using BP based on the factor graph representations.
Bayesian networks, belief propagation, factor graph, Gibbs sampling, hierarchical Bayesian models, Latent Dirichlet allocation, Markov random fields, message passing, topic models, variational Bayes
Source Publication Title
IEEE Transactions on Pattern Analysis and Machine Intelligence
Institute of Electrical and Electronics Engineers
This work is supported by the NSFC (Grant No. 61003154), the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (Grant No. 12KJA520004), and a grant from Baidu to Jia Zeng, and the General Research Fund (HKBU210410) from the Research Grant Council of the Hong Kong Special Administrative Region, China to William K. Cheung.
Link to Publisher's Edition
Zeng, J., Cheung, W., & Liu, J. (2013). Learning topic models by belief propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35 (5), 1121-1134. https://doi.org/10.1109/TPAMI.2012.185