aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorYuchen Pei <me@ypei.me>2018-05-15 13:57:23 +0200
committerYuchen Pei <me@ypei.me>2018-05-15 13:57:23 +0200
commita81cc6772f544ae974fb497c86b67ffff38ba136 (patch)
tree03238f642103d7eec93b475ebc50a1eef8e02f2a
parent6b19a185cd83eb36a6f6b0e741542d2cd03a3617 (diff)
added an mpost
-rw-r--r--assets/resources/sl-vs-ml.pngbin0 -> 150277 bytes
-rw-r--r--microposts/random-forests.md14
2 files changed, 14 insertions, 0 deletions
diff --git a/assets/resources/sl-vs-ml.png b/assets/resources/sl-vs-ml.png
new file mode 100644
index 0000000..11ad6a8
--- /dev/null
+++ b/assets/resources/sl-vs-ml.png
Binary files differ
diff --git a/microposts/random-forests.md b/microposts/random-forests.md
new file mode 100644
index 0000000..de2757c
--- /dev/null
+++ b/microposts/random-forests.md
@@ -0,0 +1,14 @@
+---
+date: 2018-05-15
+---
+
+### Notes on random froests
+
+[Stanford Lagunita's statistical learning course](https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/info) has some excellent lectures on random forests. It starts with explanations of decision trees, followed by bagged trees and random forests, and ends with boosting. From these lectures it seems that:
+
+1. The term "predictors" in statistical learning = "features" in machine learning.
+2. The main idea of random forests of dropping predictors for individual trees and aggregate by majority or average is the same as the idea of dropout in neural networks, where a proportion of neurons in the hidden layers are dropped temporarily during different minibatches of training, effectively averaging over an emsemble of subnetworks. Both tricks are used as regularisations, i.e. to reduce the variance. The only difference is: in random forests, all but a square root number of the total number of features are dropped, whereas the dropout ratio in neural networks is usually a half.
+
+By the way, here a comparison between statistical learning and machine learning from the slides of the Statistcal Learning course:
+
+<a href="../assets/resources/sl-vs-ml.png"><img src="../assets/resources/sl-vs-ml.png" alt="SL vs ML" style="width:38em" /></a>