diff options
Diffstat (limited to 'posts/2019-02-14-raise-your-elbo.md')
| -rw-r--r-- | posts/2019-02-14-raise-your-elbo.md | 14 |
1 files changed, 7 insertions, 7 deletions
diff --git a/posts/2019-02-14-raise-your-elbo.md b/posts/2019-02-14-raise-your-elbo.md index 4080d0b..36bd364 100644 --- a/posts/2019-02-14-raise-your-elbo.md +++ b/posts/2019-02-14-raise-your-elbo.md @@ -138,7 +138,7 @@ $p(x_{1 : m}; \theta)$. Represented as a DAG (a.k.a the plate notations), the model looks like this: -{style="width:250px"} +{style="width:250px"} where the boxes with $m$ mean repitition for $m$ times, since there $m$ indepdent pairs of $(x, z)$, and the same goes for $\eta$. @@ -298,7 +298,7 @@ $$p(d_i = u, x_i = w | z_i = k; \theta) = p(d_i ; \xi_k) p(x_i; \eta_k) = \xi_{k The model can be illustrated in the plate notations: -{style="width:350px"} +{style="width:350px"} So the solution of the M-step is @@ -365,7 +365,7 @@ pLSA1, $(x | z = k) \sim \text{Cat}(\eta_{k, \cdot})$. Illustrated in the plate notations, pLSA2 is: -{style="width:350px"} +{style="width:350px"} The computation is basically adding an index $\ell$ to the computation of SMM wherever applicable. @@ -411,7 +411,7 @@ $$p(z_{i1}) = \pi_{z_{i1}}$$ So the parameters are $\theta = (\pi, \xi, \eta)$. And HMM can be shown in plate notations as: -{style="width:350px"} +{style="width:350px"} Now we apply EM to HMM, which is called the [Baum-Welch algorithm](https://en.wikipedia.org/wiki/Baum%E2%80%93Welch_algorithm). @@ -569,7 +569,7 @@ later in this section that the posterior $q(\eta_k)$ belongs to the same family as $p(\eta_k)$. Represented in a plate notations, a fully Bayesian mixture model looks like: -{style="width:450px"} +{style="width:450px"} Given this structure we can write down the mean-field approximation: @@ -672,7 +672,7 @@ As the second example of fully Bayesian mixture models, Latent Dirichlet allocation (LDA) (Blei-Ng-Jordan 2003) is the fully Bayesian version of pLSA2, with the following plate notations: -{style="width:450px"} +{style="width:450px"} It is the smoothed version in the paper. @@ -782,7 +782,7 @@ $$L(p, q) = \sum_{k = 1 : T} \mathbb E_{q(\theta_k)} \log {p(\theta_k) \over q(\ The plate notation of this model is: -{style="width:450px"} +{style="width:450px"} As it turns out, the infinities can be tamed in this case. |