blob: f92feb61cca54f6b22bb68ab361f4f46669e4d7d (
plain) (
blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
|
#+title: neural-nets-regularization
#+date: <2018-05-08>
#+begin_quote
no-one has yet developed an entirely convincing theoretical
explanation for why regularization helps networks generalize. Indeed,
researchers continue to write papers where they try different
approaches to regularization, compare them to see which works better,
and attempt to understand why different approaches work better or
worse. And so you can view regularization as something of a kludge.
While it often helps, we don't have an entirely satisfactory
systematic understanding of what's going on, merely incomplete
heuristics and rules of thumb.
There's a deeper set of issues here, issues which go to the heart of
science. It's the question of how we generalize. Regularization may
give us a computational magic wand that helps our networks generalize
better, but it doesn't give us a principled understanding of how
generalization works, nor of what the best approach is.
#+end_quote
Michael Nielsen,
[[http://neuralnetworksanddeeplearning.com/chap3.html#why_does_regularization_help_reduce_overfitting][Neural
networks and deep learning]]
|
