1 files changed, 37 insertions, 0 deletions
diff --git a/microposts/neural-turing-machine.org b/microposts/neural-turing-machine.org
new file mode 100644
index 0000000..b4212c2
--- /dev/null
+++ b/microposts/neural-turing-machine.org
@@ -0,0 +1,37 @@
+#+title: neural-turing-machine
+
+#+date: <2018-05-09>
+
+#+begin_quote
+  One way RNNs are currently being used is to connect neural networks
+  more closely to traditional ways of thinking about algorithms, ways of
+  thinking based on concepts such as Turing machines and (conventional)
+  programming languages. [[https://arxiv.org/abs/1410.4615][A 2014
+  paper]] developed an RNN which could take as input a
+  character-by-character description of a (very, very simple!) Python
+  program, and use that description to predict the output. Informally,
+  the network is learning to "understand" certain Python programs.
+  [[https://arxiv.org/abs/1410.5401][A second paper, also from 2014]],
+  used RNNs as a starting point to develop what they called a neural
+  Turing machine (NTM). This is a universal computer whose entire
+  structure can be trained using gradient descent. They trained their
+  NTM to infer algorithms for several simple problems, such as sorting
+  and copying.
+
+  As it stands, these are extremely simple toy models. Learning to
+  execute the Python program =print(398345+42598)= doesn't make a
+  network into a full-fledged Python interpreter! It's not clear how
+  much further it will be possible to push the ideas. Still, the results
+  are intriguing. Historically, neural networks have done well at
+  pattern recognition problems where conventional algorithmic approaches
+  have trouble. Vice versa, conventional algorithmic approaches are good
+  at solving problems that neural nets aren't so good at. No-one today
+  implements a web server or a database program using a neural network!
+  It'd be great to develop unified models that integrate the strengths
+  of both neural networks and more traditional approaches to algorithms.
+  RNNs and ideas inspired by RNNs may help us do that.
+#+end_quote
+
+Michael Nielsen,
+[[http://neuralnetworksanddeeplearning.com/chap6.html#other_approaches_to_deep_neural_nets][Neural
+networks and deep learning]]