diff options
Diffstat (limited to 'microposts/neural-turing-machine.org')
-rw-r--r-- | microposts/neural-turing-machine.org | 37 |
1 files changed, 37 insertions, 0 deletions
diff --git a/microposts/neural-turing-machine.org b/microposts/neural-turing-machine.org new file mode 100644 index 0000000..b4212c2 --- /dev/null +++ b/microposts/neural-turing-machine.org @@ -0,0 +1,37 @@ +#+title: neural-turing-machine + +#+date: <2018-05-09> + +#+begin_quote + One way RNNs are currently being used is to connect neural networks + more closely to traditional ways of thinking about algorithms, ways of + thinking based on concepts such as Turing machines and (conventional) + programming languages. [[https://arxiv.org/abs/1410.4615][A 2014 + paper]] developed an RNN which could take as input a + character-by-character description of a (very, very simple!) Python + program, and use that description to predict the output. Informally, + the network is learning to "understand" certain Python programs. + [[https://arxiv.org/abs/1410.5401][A second paper, also from 2014]], + used RNNs as a starting point to develop what they called a neural + Turing machine (NTM). This is a universal computer whose entire + structure can be trained using gradient descent. They trained their + NTM to infer algorithms for several simple problems, such as sorting + and copying. + + As it stands, these are extremely simple toy models. Learning to + execute the Python program =print(398345+42598)= doesn't make a + network into a full-fledged Python interpreter! It's not clear how + much further it will be possible to push the ideas. Still, the results + are intriguing. Historically, neural networks have done well at + pattern recognition problems where conventional algorithmic approaches + have trouble. Vice versa, conventional algorithmic approaches are good + at solving problems that neural nets aren't so good at. No-one today + implements a web server or a database program using a neural network! + It'd be great to develop unified models that integrate the strengths + of both neural networks and more traditional approaches to algorithms. + RNNs and ideas inspired by RNNs may help us do that. +#+end_quote + +Michael Nielsen, +[[http://neuralnetworksanddeeplearning.com/chap6.html#other_approaches_to_deep_neural_nets][Neural +networks and deep learning]] |