aboutsummaryrefslogtreecommitdiff
path: root/posts
diff options
context:
space:
mode:
authorYuchen Pei <me@ypei.me>2019-02-18 10:51:03 +0100
committerYuchen Pei <me@ypei.me>2019-02-18 10:51:03 +0100
commitcf3a03b0e4ecd7c5f0ba92693be732b676061642 (patch)
tree42d1e32b4c66c5664bb638fa4efe42e929ffb97b /posts
parente28f5e7ee788bf74f8e3495e8488a064edd0a82d (diff)
minor fixes
Diffstat (limited to 'posts')
-rw-r--r--posts/2019-02-14-raise-your-elbo.md8
1 files changed, 4 insertions, 4 deletions
diff --git a/posts/2019-02-14-raise-your-elbo.md b/posts/2019-02-14-raise-your-elbo.md
index 3b8c1a5..5700899 100644
--- a/posts/2019-02-14-raise-your-elbo.md
+++ b/posts/2019-02-14-raise-your-elbo.md
@@ -86,7 +86,7 @@ as variational Bayes (VB).
**Definition**. Variational inference is concerned with
maximising the ELBO $L(w, q)$.
-There are mainly two version of VI, the half Bayesian and the fully
+There are mainly two versions of VI, the half Bayesian and the fully
Bayesian cases. Half Bayesian VI, to which expectation-maximisation
algorithms (EM) apply, instantiates (1.3) with
@@ -272,8 +272,8 @@ ocurrance of word $x$ in document $d$.
For each datapoint $(d_{i}, x_{i})$,
$$\begin{aligned}
-p(d_i, x_i; \theta) &= \sum_z p(z; \theta) p(d_i | z; \theta) p(x_i | z; \theta) \qquad (2.91)\\
-&= p(d_i; \theta) \sum_z p(x_i | z; \theta) p (z | d_i; \theta) \qquad (2.92).
+p(d_i, x_i; \theta) &= \sum_{z_i} p(z; \theta) p(d_i | z_i; \theta) p(x_i | z_i; \theta) \qquad (2.91)\\
+&= p(d_i; \theta) \sum_{z_i} p(x_i | z_i; \theta) p (z_i | d_i; \theta) \qquad (2.92).
\end{aligned}$$
Of the two formulations, (2.91) corresponds to pLSA type 1, and (2.92)
@@ -285,7 +285,7 @@ The pLSA1 model (Hoffman 2000) is basically SMM with $x_i$ substituted
with $(d_i, x_i)$, which conditioned on $z$ are independently
categorically distributed:
-$$p(d_i = u, x_i = w | z = k) = p(d_i | \xi_k) p(x_i; \eta_k) = \xi_{ku} \eta_{kw}.$$
+$$p(d_i = u, x_i = w | z_i = k) = p(d_i | \xi_k) p(x_i; \eta_k) = \xi_{ku} \eta_{kw}.$$
The model can be illustrated in the plate notations: