From 0e8e006ea3f511a8e2e86e0f38ac9b1ab9847b8c Mon Sep 17 00:00:00 2001 From: Yuchen Pei Date: Thu, 10 May 2018 10:10:57 +0200 Subject: 1 --- microposts/math-writing-decoupling.md | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 microposts/math-writing-decoupling.md (limited to 'microposts/math-writing-decoupling.md') diff --git a/microposts/math-writing-decoupling.md b/microposts/math-writing-decoupling.md new file mode 100644 index 0000000..6a7b438 --- /dev/null +++ b/microposts/math-writing-decoupling.md @@ -0,0 +1,10 @@ +--- +2018-05-10 +--- +### Writing readable mathematics like writing an operating system + +One way to write readable mathematics is to decouple concepts. One idea is the following template. First write a toy example with all the important components present in this example, then analyse each component individually and elaborate how (perhaps more complex) variations of the component can extend the toy example and induce more complex or powerful versions of the toy example. Through such incremental development, one should be able to arrive at any result in cutting edge research after a pleasant journey. + +It's a bit like the UNIX philosophy, where you have a basic system of modules like IO, memory management, graphics etc, and modify / improve each module individually (H/t [NAND2Tetris](http://nand2tetris.org/)). + +The book [Neutral networks and deep learning](http://neuralnetworksanddeeplearning.com/) by Michael Nielsen is an example of such approach. It begins the journey with a very simple neutral net with one hidden layer, no regularisation, and sigmoid activations. It then analyses each component including cost functions, the back propagation algorithm, the activation functions, regularisation and the overall architecture (from fully connected to CNN) individually and improve the toy example incrementally. Over the course the accuracy of the example of mnist grows incrementally from 95.42% to 99.63%. -- cgit v1.2.3