blob: 9f2866df9fdf338e669d48897cd7e38e71161130 (
plain) (
blame)
1
2
3
4
5
6
7
8
|
---
date: 2018-05-08
---
> no-one has yet developed an entirely convincing theoretical explanation for why regularization helps networks generalize. Indeed, researchers continue to write papers where they try different approaches to regularization, compare them to see which works better, and attempt to understand why different approaches work better or worse. And so you can view regularization as something of a kludge. While it often helps, we don't have an entirely satisfactory systematic understanding of what's going on, merely incomplete heuristics and rules of thumb.
>
> There's a deeper set of issues here, issues which go to the heart of science. It's the question of how we generalize. Regularization may give us a computational magic wand that helps our networks generalize better, but it doesn't give us a principled understanding of how generalization works, nor of what the best approach is.
Michael Nielsen, [Neural networks and deep learning](http://neuralnetworksanddeeplearning.com/chap3.html#why_does_regularization_help_reduce_overfitting)
|
