Exponential Family Predictive Representations of State
David Wingate and Satinder Singh ­ Poster W23

The idea: marry PSRs and graphical models to build models of structured dynamical systems
We assume the distribution of the next n observations in the future is described with an undirected exponential family graphical model ...and that state is the parameters of that distribution

...
Past Present Future n observations

Likelihood

The graph is fully observed, so maximum likelihood learning is straightforward... ...and the states can be used for prediction or in RL algorithms.

Average Reward

True EFPSR

EFPSR

Optimal

Naive