Relative Performance guarantees for Approximate Inference in Latent Dirichlet Allocation Indraneel Mukherjee, David M. Blei

· Inference: Prob ( latent variables | data ). Intractable · [TehNeWe06] Collapse latent space for better approximation. · Q: Is the improvement worth the extra computation? · Theorem: For LDA, when documents are long, computation not worth it. · Our rigorous mathematical analysis is also verified empirically:
VB vs CVB: per word free energy (10 mov. avgd.)
0.12 2.5 %age change in free energy

VB vs CVB: per word free energy (1000 mov. avgd.)

2.0

0.02

0.04

5 10 25 50

0.08

k

%age change in free energy

0.10

k 5 10 25 50

1.0

0.06

1.5

0

20

40

60

80 #words

100

120

140

2000

4000

6000 #words

8000

10000

M43
1

0.5

0.0

0.00