diff options
| author | Yuchen Pei <me@ypei.me> | 2018-05-11 12:39:31 +0200 |
|---|---|---|
| committer | Yuchen Pei <me@ypei.me> | 2018-05-11 12:39:31 +0200 |
| commit | 37701d5c28fa019986b75801a0f9480d49a69c76 (patch) | |
| tree | 16ad6785296c2b9ab4dd2cf1062dfcee74f662c3 | |
| parent | ac9007d3cf2a3eabb1c34a6e7f6155372b2bde09 (diff) | |
minor edits
| -rw-r--r-- | microposts/rnn-fsm.md | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/microposts/rnn-fsm.md b/microposts/rnn-fsm.md index 38ff333..567063d 100644 --- a/microposts/rnn-fsm.md +++ b/microposts/rnn-fsm.md @@ -5,7 +5,7 @@ date: 2018-05-11 Related to a previous micropost. -[The slides from Toronto](http://www.cs.toronto.edu/~rgrosse/csc321/lec9.pdf) is a nice introduction to RNN (recurrent neural network) from a computational point of view. It states the relation between RNN and FSM (finite state machine, a.k.a. finite automata abbr. FA) with a toy example computing the parity of a binary string. +[The slides from Toronto](http://www.cs.toronto.edu/~rgrosse/csc321/lec9.pdf) is a nice introduction to RNN (recurrent neural network) from a computational point of view. It states that RNN can simulate any FSM (finite state machine, a.k.a. finite automata abbr. FA) with a toy example computing the parity of a binary string. [Goodfellow et. al.'s book](http://www.deeplearningbook.org/contents/rnn.html) (see page 372 and 374) goes one step further, stating that RNN with a hidden-to-hidden layer can simulate Turing machines, and not only that, but also the *universal* Turing machine abbr. UTM (the book referenced [Siegelmann-Sontag](https://www.sciencedirect.com/science/article/pii/S0022000085710136)), a property not shared by the weaker network where the hidden-to-hidden layer is replaced by an output-to-hidden layer (page 376). |