The Recurrent Temporal Restricted Boltzmann Machine

I. Sutskever, G. Hinton, G. Taylor; University of Toronto. Poster Location: T12

The TRBM: a directed sequence of Restricted Boltzmann Machines. The hidden states determine the partition function of the next RBM. Very Intractable.

RTRBM: sample the TRBM as usual (do Gibbs for the RBM in each time step) but in the last step, do a "mean-field" update for the hiddens (so they become continuous). So the RTRBM is very similar to the TRBM, but this modification makes it tractable! (see poster for details).