Unified Inference for Variational Bayesian Linear Gaussian State-Space Models

Linear Gaussian State-Space Models are widely used and a Bayesian treatment of parameters is therefore of considerable interest. The approximate Variational Bayesian method applied to these models is an attractive approach, used successfully in applications ranging from acoustics to bioinformatics. The most challenging aspect of implementing the method is in performing inference on the hidden state sequence of the model. We show how to convert the inference problem so that standard Kalman Filtering/Smoothing recursions from the literature may be applied. This is in contrast to previously published approaches based on Belief Propagation. Our framework both simplifies and unifies the inference problem, so that future applications may be more easily developed. We demonstrate the elegance of the approach on Bayesian temporal ICA, with an application to finding independent dynamical processes underlying noisy EEG signals. 1 Linear Gaussian State-Space Models Linear Gaussian State-Space Models (LGSSMs)1 are fundamental in time-series analysis [1, 2, 3]. In these models the observations v1:T 2 are generated from an underlying dynamical system on h1:T according to: v v vt = B ht + t , t N (0V , V ), h h ht = Aht-1 + t , t N (0H , H ) , where N (, ) denotes a Gaussian with mean and covariance , and 0X denotes an X dimensional zero vector. The observation vt has dimension V and the hidden state ht has dimension H . Probabilistically, the LGSSM is defined by: p(v1:T , h1:T |) = p(v1 |h1 )p(h1 ) tT p(vt |ht )p(ht |ht-1 ), =2 with p(vt |ht ) = N (B ht , V ), p(ht |ht-1 ) = N (Aht-1 , H ), p(h1 ) = N (, ) and where = {A, B , H , V , , } denotes the model parameters. Because of the widespread use of these models, a Bayesian treatment of parameters is of considerable interest [4, 5, 6, 7, 8]. An exact implementation of the Bayesian LGSSM is formally intractable [8], and recently a Variational Bayesian (VB) approximation has been studied [4, 5, 6, 7, 9]. The most challenging part of