diff options
author | Yuchen Pei <me@ypei.me> | 2019-02-19 19:22:44 +0100 |
---|---|---|
committer | Yuchen Pei <me@ypei.me> | 2019-02-19 19:22:44 +0100 |
commit | 4eafd0a87884ad2c3b6950d961f746183c7bee82 (patch) | |
tree | f5d8ad2ef491bf9c7c3f06402e6f8303f955e312 | |
parent | b746ee8e3f893bc9153f28ec3d5c85ccb314f358 (diff) |
more plsa2 fix
-rw-r--r-- | posts/2019-02-14-raise-your-elbo.md | 7 |
1 files changed, 6 insertions, 1 deletions
diff --git a/posts/2019-02-14-raise-your-elbo.md b/posts/2019-02-14-raise-your-elbo.md index 8e6cf8c..fb1052a 100644 --- a/posts/2019-02-14-raise-your-elbo.md +++ b/posts/2019-02-14-raise-your-elbo.md @@ -343,6 +343,11 @@ Now the model becomes $$p(x_{\ell, i} | d_{\ell, i} = \ell; \theta) = \sum_k p(x_{\ell, i} | z_{\ell, i} = k; \theta) p(z_{\ell, i} = k | d_{\ell, i} = \ell; \theta).$$ +Since we have regrouped the $x$'s and $z$'s whose indices record the values +of the $d$'s, we can remove $d$ from the equation altogether. + +$$p(x_{\ell, i} | \theta) = \sum_k p(x_{\ell, i} | z_{\ell, i} = k; \theta) p(z_{\ell, i} = k | \theta).$$ + It is effectively a modification of SMM by making $n_d$ copies of $\pi$. More specifically the parameters are $\theta = (\pi_{1 : n_d, 1 : n_z}, \eta_{1 : n_z, 1 : n_x})$, where we @@ -358,7 +363,7 @@ of SMM wherever applicable. The updates at the E-step is -$$r_{\ell i k} = p(z_{\ell i} = k | x_{\ell i}, d_{\ell, i} = \ell) \propto \pi_{\ell k} \eta_{k, x_{\ell i}}.$$ +$$r_{\ell i k} = p(z_{\ell i} = k | x_{\ell i}; \theta) \propto \pi_{\ell k} \eta_{k, x_{\ell i}}.$$ And at the M-step |