A Sufficient Statistics Construction of Exponential Family Lévy Measure Densities for Nonparametric Conjugate Models

Conjugate pairs of distributions over infinite dimensional spaces are prominent in machine learning, particularly due to the widespread adoption of Bayesian nonparametric methodologies for a host of models and applications. Much of the existing literature in the learning community focuses on processes possessing some form of computationally tractable conjugacy as is the case for the beta process and the gamma process (and, via normalization, the Dirichlet process). For these processes, conjugacy is proved via statistical machinery tailored to the particular model. We seek to address the problem of obtaining a general construction of prior distributions over infinite dimensional spaces possessing distributional properties amenable to conjugacy. Our result is achieved by generalizing Hjort’s construction of the beta process via appropriate utilization of sufficient statistics for exponential families.