Sum-Product Graphical Models

Sang-Woo Lee and Byoung-Tak Zhang

Abstract

Designing efficient deep probabilistic models with powerful representation is a challenging problem in various academic fields including computational cognitive science. We propose sum-product graphical models (SPGM), which are novel probabilistic models generalizing sum-product networks and other classical probabilistic graphical models. Previous probabilistic graphical models mainly focused on finding only directed connections between nodes (e.g. Bayesian networks, Markov random fields) or only undirected connections (e.g. mixture models, sum-product networks). Our approach considers both directed and undirected interactions on probabilistic graphical models. Sum-product graphical models can be explained as an ensemble of exponential number of classical probabilistic models, whereas sum-product networks consist of naïve Bayes. Allowing leaves in a sum-product network to be directly related to each other, a sum-product graphical model has a powerful representation beyond not only sum-product networks but also Bayesian networks and Markov random fields. In addition, we propose tree-augmented sum-product networks (TASPN), a kind of sum-product graphical models which allows for efficient and exact inference. Tree-augmented sum-product networks are special ensembles of exponential number of Chow-Liu trees (1968). In experiments on various benchmark datasets and a visio-linguistic dataset in cartoon video, we measure performance of our model and discover usefulness of the novel framework.