aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorYuchen Pei <me@ypei.me>2019-02-19 19:22:44 +0100
committerYuchen Pei <me@ypei.me>2019-02-19 19:22:44 +0100
commit4eafd0a87884ad2c3b6950d961f746183c7bee82 (patch)
treef5d8ad2ef491bf9c7c3f06402e6f8303f955e312
parentb746ee8e3f893bc9153f28ec3d5c85ccb314f358 (diff)
more plsa2 fix
-rw-r--r--posts/2019-02-14-raise-your-elbo.md7
1 files changed, 6 insertions, 1 deletions
diff --git a/posts/2019-02-14-raise-your-elbo.md b/posts/2019-02-14-raise-your-elbo.md
index 8e6cf8c..fb1052a 100644
--- a/posts/2019-02-14-raise-your-elbo.md
+++ b/posts/2019-02-14-raise-your-elbo.md
@@ -343,6 +343,11 @@ Now the model becomes
$$p(x_{\ell, i} | d_{\ell, i} = \ell; \theta) = \sum_k p(x_{\ell, i} | z_{\ell, i} = k; \theta) p(z_{\ell, i} = k | d_{\ell, i} = \ell; \theta).$$
+Since we have regrouped the $x$'s and $z$'s whose indices record the values
+of the $d$'s, we can remove $d$ from the equation altogether.
+
+$$p(x_{\ell, i} | \theta) = \sum_k p(x_{\ell, i} | z_{\ell, i} = k; \theta) p(z_{\ell, i} = k | \theta).$$
+
It is effectively a modification of SMM by making $n_d$ copies of $\pi$.
More specifically the parameters are
$\theta = (\pi_{1 : n_d, 1 : n_z}, \eta_{1 : n_z, 1 : n_x})$, where we
@@ -358,7 +363,7 @@ of SMM wherever applicable.
The updates at the E-step is
-$$r_{\ell i k} = p(z_{\ell i} = k | x_{\ell i}, d_{\ell, i} = \ell) \propto \pi_{\ell k} \eta_{k, x_{\ell i}}.$$
+$$r_{\ell i k} = p(z_{\ell i} = k | x_{\ell i}; \theta) \propto \pi_{\ell k} \eta_{k, x_{\ell i}}.$$
And at the M-step