Poster W10: Expectation Maximization and Posterior Constraints
Intractable Constraints in Tractable Models Jo~o V. Gra¸a , Kuzman Ganchev and Ben Taskar a c L2 F INESC-ID  Department of Computer Science, University of Pennsylvania



Normal EM
1

posterior

-Step arg min KL(q (z | x) || pt (z | x))
0 constraint set Q posterior
q (z|x)

objective

OE jective bl z L() = EX og p (x, z)

-log likelihood E-step bound parameters

1

Constrained EM
Ob jective L()- EX [KL(Q(x)||p (z | x)]
objective -log likelihood + KL(Q || Posterior) -log likelihood Constrained E-step bound parameters

posterior

E-Step arg min KL(q (z | x) || pt (z | x))
q (z|x)Q(x)

0

constraint set Q posterior

Drastic precision/recall improvements in word alignment: EnglishFrench / EnglishSpanish