diff options
Diffstat (limited to 'pages')
| -rw-r--r-- | pages/all-microposts.org | 773 | ||||
| -rw-r--r-- | pages/blog.org | 20 | ||||
| -rw-r--r-- | pages/microblog.org | 683 |
3 files changed, 1476 insertions, 0 deletions
diff --git a/pages/all-microposts.org b/pages/all-microposts.org new file mode 100644 index 0000000..92896bc --- /dev/null +++ b/pages/all-microposts.org @@ -0,0 +1,773 @@ +#+title: Yuchen's Microblog + +*** 2020-08-02: ia-lawsuit + :PROPERTIES: + :CUSTOM_ID: ia-lawsuit + :END: +The four big publishers Hachette, HarperCollins, Wiley, and Penguin +Random House are still pursuing Internet Archive. + +#+begin_quote + [Their] lawsuit does not stop at seeking to end the practice of + Controlled Digital Lending. These publishers call for the destruction + of the 1.5 million digital books that Internet Archive makes available + to our patrons. This form of digital book burning is unprecedented and + unfairly disadvantages people with print disabilities. For the blind, + ebooks are a lifeline, yet less than one in ten exists in accessible + formats. Since 2010, Internet Archive has made our lending library + available to the blind and print disabled community, in addition to + sighted users. If the publishers are successful with their lawsuit, + more than a million of those books would be deleted from the + Internet's digital shelves forever. +#+end_quote + +[[https://blog.archive.org/2020/07/29/internet-archive-responds-to-publishers-lawsuit/][Libraries +lend books, and must continue to lend books: Internet Archive responds +to publishers' lawsuit]] +*** 2020-08-02: fsf-membership + :PROPERTIES: + :CUSTOM_ID: fsf-membership + :END: +I am a proud associate member of Free Software Freedom. For me the +philosophy of Free Software is about ensuring the enrichment of a +digital commons, so that knowledge and information are not concentrated +in the hands of selected privileged people and locked up as +"intellectual property". The genius of copyleft licenses like GNU (A)GPL +ensures software released for the public, remains public. Open source +does not care about that. + +If you also care about the public good, the hacker ethics, or the spirit +of the web, please take a moment to consider joining FSF as an associate +member. It comes with [[https://www.fsf.org/associate/benefits][numerous +perks and benefits]]. +*** 2020-06-21: how-can-you-help-ia + :PROPERTIES: + :CUSTOM_ID: how-can-you-help-ia + :END: +[[https://blog.archive.org/2020/06/14/how-can-you-help-the-internet-archive/][How +can you help the Internet Archive?]] Use it. It's more than the Wayback +Machine. And get involved. +*** 2020-06-12: open-library + :PROPERTIES: + :CUSTOM_ID: open-library + :END: +Open Library was cofounded by Aaron Swartz. As part of the Internet +Archive, it has done good work to spread knowledge. However it is +currently +[[https://arstechnica.com/tech-policy/2020/06/internet-archive-ends-emergency-library-early-to-appease-publishers/][being +sued by four major publishers]] for the +[[https://archive.org/details/nationalemergencylibrary][National +Emergency Library]]. IA decided to +[[https://blog.archive.org/2020/06/10/temporary-national-emergency-library-to-close-2-weeks-early-returning-to-traditional-controlled-digital-lending/][close +the NEL two weeks earlier than planned]], but the lawsuit is not over, +which in the worst case scenario has the danger of resulting in +Controlled Digital Lending being considered illegal and (less likely) +bancruptcy of the Internet Archive. If this happens it will be a big +setback of the free-culture movement. +*** 2020-04-15: sanders-suspend-campaign + :PROPERTIES: + :CUSTOM_ID: sanders-suspend-campaign + :END: +Suspending the campaign is different from dropping out of the race. +Bernie Sanders remains on the ballot, and indeed in his campaign +suspension speech he encouraged people to continue voting for him in the +democratic primaries to push for changes in the convention. +*** 2019-09-30: defense-stallman + :PROPERTIES: + :CUSTOM_ID: defense-stallman + :END: +Someone wrote a bold article titled +[[https://geoff.greer.fm/2019/09/30/in-defense-of-richard-stallman/]["In +Defense of Richard Stallman"]]. Kudos to him. + +Also, an interesting read: +[[https://cfenollosa.com/blog/famous-computer-public-figure-suffers-the-consequences-for-asshole-ish-behavior.html][Famous +public figure in tech suffers the consequences for asshole-ish +behavior]]. +*** 2019-09-29: stallman-resign + :PROPERTIES: + :CUSTOM_ID: stallman-resign + :END: +Last week Richard Stallman resigned from FSF. It is a great loss for the +free software movement. + +The apparent cause of his resignation and the events that triggered it +reflect some alarming trends of the zeitgeist. Here is a detailed review +of what happened: [[https://sterling-archermedes.github.io/][Low grade +"journalists" and internet mob attack RMS with lies. In-depth review.]]. +Some interesting articles on this are: +[[https://jackbaruth.com/?p=16779][Weekly Roundup: The Passion Of Saint +iGNUcius Edition]], +[[http://techrights.org/2019/09/17/rms-witch-hunt/][Why I Once Called +for Richard Stallman to Step Down]]. + +Dishonest and misleading media pieces involved in this incident include +[[https://www.thedailybeast.com/famed-mit-computer-scientist-richard-stallman-defends-epstein-victims-were-entirely-willing][The +Daily Beast]], +[[https://www.vice.com/en_us/article/9ke3ke/famed-computer-scientist-richard-stallman-described-epstein-victims-as-entirely-willing][Vice]], +[[https://techcrunch.com/2019/09/16/computer-scientist-richard-stallman-who-defended-jeffrey-epstein-resigns-from-mit-csail-and-the-free-software-foundation/][Tech +Crunch]], +[[https://www.wired.com/story/richard-stallmans-exit-heralds-a-new-era-in-tech/][Wired]]. +*** 2019-03-16: decss-haiku + :PROPERTIES: + :CUSTOM_ID: decss-haiku + :END: + +#+begin_quote + #+begin_example + Muse! When we learned to + count, little did we know all + the things we could do + + some day by shuffling + those numbers: Pythagoras + said "All is number" + + long before he saw + computers and their effects, + or what they could do + + by computation, + naive and mechanical + fast arithmetic. + + It changed the world, it + changed our consciousness and lives + to have such fast math + + available to + us and anyone who cared + to learn programming. + + Now help me, Muse, for + I wish to tell a piece of + controversial math, + + for which the lawyers + of DVD CCA + don't forbear to sue: + + that they alone should + know or have the right to teach + these skills and these rules. + + (Do they understand + the content, or is it just + the effects they see?) + + And all mathematics + is full of stories (just read + Eric Temple Bell); + + and CSS is + no exception to this rule. + Sing, Muse, decryption + + once secret, as all + knowledge, once unknown: how to + decrypt DVDs. + #+end_example +#+end_quote + +Seth Schoen, [[https://en.wikipedia.org/wiki/DeCSS_haiku][DeCSS haiku]] +*** 2019-01-27: learning-undecidable + :PROPERTIES: + :CUSTOM_ID: learning-undecidable + :END: +My take on the +[[https://www.nature.com/articles/s42256-018-0002-3][Nature paper +/Learning can be undecidable/]]: + +Fantastic article, very clearly written. + +So it reduces a kind of learninability called estimating the maximum +(EMX) to the cardinality of real numbers which is undecidable. + +When it comes to the relation between EMX and the rest of machine +learning framework, the article mentions that EMX belongs to "extensions +of PAC learnability include Vapnik's statistical learning setting and +the equivalent general learning setting by Shalev-Shwartz and +colleagues" (I have no idea what these two things are), but it does not +say whether EMX is representative of or reduces to common learning +tasks. So it is not clear whether its undecidability applies to ML at +large. + +Another condition to the main theorem is the union bounded closure +assumption. It seems a reasonable property of a family of sets, but then +again I wonder how that translates to learning. + +The article says "By now, we know of quite a few independence [from +mathematical axioms] results, mostly for set theoretic questions like +the continuum hypothesis, but also for results in algebra, analysis, +infinite combinatorics and more. Machine learning, so far, has escaped +this fate." but the description of the EMX learnability makes it more +like a classical mathematical / theoretical computer science problem +rather than machine learning. + +An insightful conclusion: "How come learnability can neither be proved +nor refuted? A closer look reveals that the source of the problem is in +defining learnability as the existence of a learning function rather +than the existence of a learning algorithm. In contrast with the +existence of algorithms, the existence of functions over infinite +domains is a (logically) subtle issue." + +In relation to practical problems, it uses an example of ad targeting. +However, A lot is lost in translation from the main theorem to this ad +example. + +The EMX problem states: given a domain X, a distribution P over X which +is unknown, some samples from P, and a family of subsets of X called F, +find A in F that approximately maximises P(A). + +The undecidability rests on X being the continuous [0, 1] interval, and +from the insight, we know the problem comes from the cardinality of +subsets of the [0, 1] interval, which is "logically subtle". + +In the ad problem, the domain X is all potential visitors, which is +finite because there are finite number of people in the world. In this +case P is a categorical distribution over the 1..n where n is the +population of the world. One can have a good estimate of the parameters +of a categorical distribution by asking for sufficiently large number of +samples and computing the empirical distribution. Let's call the +estimated distribution Q. One can choose the from F (also finite) the +set that maximises Q(A) which will be a solution to EMX. + +In other words, the theorem states: EMX is undecidable because not all +EMX instances are decidable, because there are some nasty ones due to +infinities. That does not mean no EMX instance is decidable. And I think +the ad instance is decidable. Is there a learning task that actually +corresponds to an undecidable EMX instance? I don't know, but I will not +believe the result of this paper is useful until I see one. + +h/t Reynaldo Boulogne +*** 2018-12-11: gavin-belson + :PROPERTIES: + :CUSTOM_ID: gavin-belson + :END: + +#+begin_quote + I don't know about you people, but I don't want to live in a world + where someone else makes the world a better place better than we do. +#+end_quote + +Gavin Belson, Silicon Valley S2E1. + +I came across this quote in +[[https://slate.com/business/2018/12/facebook-emails-lawsuit-embarrassing-mark-zuckerberg.html][a +Slate post about Facebook]] +*** 2018-10-05: margins + :PROPERTIES: + :CUSTOM_ID: margins + :END: +With Fermat's Library's new tool +[[https://fermatslibrary.com/margins][margins]], you can host your own +journal club. +*** 2018-09-18: rnn-turing + :PROPERTIES: + :CUSTOM_ID: rnn-turing + :END: +Just some non-rigorous guess / thought: Feedforward networks are like +combinatorial logic, and recurrent networks are like sequential logic +(e.g. data flip-flop is like the feedback connection in RNN). Since NAND ++ combinatorial logic + sequential logic = von Neumann machine which is +an approximation of the Turing machine, it is not surprising that RNN +(with feedforward networks) is Turing complete (assuming that neural +networks can learn the NAND gate). +*** 2018-09-07: zitierkartell + :PROPERTIES: + :CUSTOM_ID: zitierkartell + :END: +[[https://academia.stackexchange.com/questions/116489/counter-strategy-against-group-that-repeatedly-does-strategic-self-citations-and][Counter +strategy against group that repeatedly does strategic self-citations and +ignores other relevant research]] +*** 2018-09-05: short-science + :PROPERTIES: + :CUSTOM_ID: short-science + :END: + +#+begin_quote + + - ShortScience.org is a platform for post-publication discussion + aiming to improve accessibility and reproducibility of research + ideas. + - The website has over 800 summaries, mostly in machine learning, + written by the community and organized by paper, conference, and + year. + - Reading summaries of papers is useful to obtain the perspective and + insight of another reader, why they liked or disliked it, and their + attempt to demystify complicated sections. + - Also, writing summaries is a good exercise to understand the content + of a paper because you are forced to challenge your assumptions when + explaining it. + - Finally, you can keep up to date with the flood of research by + reading the latest summaries on our Twitter and Facebook pages. +#+end_quote + +[[https://shortscience.org][ShortScience.org]] +*** 2018-08-13: darknet-diaries + :PROPERTIES: + :CUSTOM_ID: darknet-diaries + :END: +[[https://darknetdiaries.com][Darknet Diaries]] is a cool podcast. +According to its about page it covers "true stories from the dark side +of the Internet. Stories about hackers, defenders, threats, malware, +botnets, breaches, and privacy." +*** 2018-06-20: coursera-basic-income + :PROPERTIES: + :CUSTOM_ID: coursera-basic-income + :END: +Coursera is having +[[https://www.coursera.org/learn/exploring-basic-income-in-a-changing-economy][a +Teach-Out on Basic Income]]. +*** 2018-06-19: pun-generator + :PROPERTIES: + :CUSTOM_ID: pun-generator + :END: +[[https://en.wikipedia.org/wiki/Computational_humor#Pun_generation][Pun +generators exist]]. +*** 2018-06-15: hackers-excerpt + :PROPERTIES: + :CUSTOM_ID: hackers-excerpt + :END: + +#+begin_quote + But as more nontechnical people bought computers, the things that + impressed hackers were not as essential. While the programs themselves + had to maintain a certain standard of quality, it was quite possible + that the most exacting standards---those applied by a hacker who + wanted to add one more feature, or wouldn't let go of a project until + it was demonstrably faster than anything else around---were probably + counterproductive. What seemed more important was marketing. There + were plenty of brilliant programs which no one knew about. Sometimes + hackers would write programs and put them in the public domain, give + them away as easily as John Harris had lent his early copy of + Jawbreaker to the guys at the Fresno computer store. But rarely would + people ask for public domain programs by name: they wanted the ones + they saw advertised and discussed in magazines, demonstrated in + computer stores. It was not so important to have amazingly clever + algorithms. Users would put up with more commonplace ones. + + The Hacker Ethic, of course, held that every program should be as good + as you could make it (or better), infinitely flexible, admired for its + brilliance of concept and execution, and designed to extend the user's + powers. Selling computer programs like toothpaste was heresy. But it + was happening. Consider the prescription for success offered by one of + a panel of high-tech venture capitalists, gathered at a 1982 software + show: "I can summarize what it takes in three words: marketing, + marketing, marketing." When computers are sold like toasters, programs + will be sold like toothpaste. The Hacker Ethic notwithstanding. +#+end_quote + +[[http://www.stevenlevy.com/index.php/books/hackers][Hackers: Heroes of +Computer Revolution]], by Steven Levy. +*** 2018-06-11: catalan-overflow + :PROPERTIES: + :CUSTOM_ID: catalan-overflow + :END: +To compute Catalan numbers without unnecessary overflow, use the +recurrence formula \(C_n = {4 n - 2 \over n + 1} C_{n - 1}\). +*** 2018-06-04: boyer-moore + :PROPERTIES: + :CUSTOM_ID: boyer-moore + :END: +The +[[https://en.wikipedia.org/wiki/Boyer–Moore_majority_vote_algorithm][Boyer-Moore +algorithm for finding the majority of a sequence of elements]] falls in +the category of "very clever algorithms". + +#+begin_example + int majorityElement(vector<int>& xs) { + int count = 0; + int maj = xs[0]; + for (auto x : xs) { + if (x == maj) count++; + else if (count == 0) maj = x; + else count--; + } + return maj; + } +#+end_example +*** 2018-05-30: how-to-learn-on-your-own + :PROPERTIES: + :CUSTOM_ID: how-to-learn-on-your-own + :END: +Roger Grosse's post +[[https://metacademy.org/roadmaps/rgrosse/learn_on_your_own][How to +learn on your own (2015)]] is an excellent modern guide on how to learn +and research technical stuff (especially machine learning and maths) on +one's own. +*** 2018-05-25: 2048-mdp + :PROPERTIES: + :CUSTOM_ID: 2048-mdp + :END: +[[http://jdlm.info/articles/2018/03/18/markov-decision-process-2048.html][This +post]] models 2048 as an MDP and solves it using policy iteration and +backward induction. +*** 2018-05-22: ats + :PROPERTIES: + :CUSTOM_ID: ats + :END: + +#+begin_quote + ATS (Applied Type System) is a programming language designed to unify + programming with formal specification. ATS has support for combining + theorem proving with practical programming through the use of advanced + type systems. A past version of The Computer Language Benchmarks Game + has demonstrated that the performance of ATS is comparable to that of + the C and C++ programming languages. By using theorem proving and + strict type checking, the compiler can detect and prove that its + implemented functions are not susceptible to bugs such as division by + zero, memory leaks, buffer overflow, and other forms of memory + corruption by verifying pointer arithmetic and reference counting + before the program compiles. Additionally, by using the integrated + theorem-proving system of ATS (ATS/LF), the programmer may make use of + static constructs that are intertwined with the operative code to + prove that a function attains its specification. +#+end_quote + +[[https://en.wikipedia.org/wiki/ATS_(programming_language)][Wikipedia +entry on ATS]] +*** 2018-05-20: bostoncalling + :PROPERTIES: + :CUSTOM_ID: bostoncalling + :END: +(5-second fame) I sent a picture of my kitchen sink to BBC and got +mentioned in the [[https://www.bbc.co.uk/programmes/w3cswg8c][latest +Boston Calling episode]] (listen at 25:54). +*** 2018-05-18: colah-blog + :PROPERTIES: + :CUSTOM_ID: colah-blog + :END: +[[https://colah.github.io/][colah's blog]] has a cool feature that +allows you to comment on any paragraph of a blog post. Here's an +[[https://colah.github.io/posts/2015-08-Understanding-LSTMs/][example]]. +If it is doable on a static site hosted on Github pages, I suppose it +shouldn't be too hard to implement. This also seems to work more +seamlessly than [[https://fermatslibrary.com/][Fermat's Library]], +because the latter has to embed pdfs in webpages. Now fantasy time: +imagine that one day arXiv shows html versions of papers (through author +uploading or conversion from TeX) with this feature. +*** 2018-05-15: random-forests + :PROPERTIES: + :CUSTOM_ID: random-forests + :END: +[[https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/info][Stanford +Lagunita's statistical learning course]] has some excellent lectures on +random forests. It starts with explanations of decision trees, followed +by bagged trees and random forests, and ends with boosting. From these +lectures it seems that: + +1. The term "predictors" in statistical learning = "features" in machine + learning. +2. The main idea of random forests of dropping predictors for individual + trees and aggregate by majority or average is the same as the idea of + dropout in neural networks, where a proportion of neurons in the + hidden layers are dropped temporarily during different minibatches of + training, effectively averaging over an emsemble of subnetworks. Both + tricks are used as regularisations, i.e. to reduce the variance. The + only difference is: in random forests, all but a square root number + of the total number of features are dropped, whereas the dropout + ratio in neural networks is usually a half. + +By the way, here's a comparison between statistical learning and machine +learning from the slides of the Statistcal Learning course: +*** 2018-05-14: open-review-net + :PROPERTIES: + :CUSTOM_ID: open-review-net + :END: +Open peer review means peer review process where communications +e.g. comments and responses are public. + +Like [[https://scipost.org/][SciPost]] mentioned in +[[/posts/2018-04-10-update-open-research.html][my post]], +[[https://openreview.net][OpenReview.net]] is an example of open peer +review in research. It looks like their focus is machine learning. Their +[[https://openreview.net/about][about page]] states their mission, and +here's [[https://openreview.net/group?id=ICLR.cc/2018/Conference][an +example]] where you can click on each entry to see what it is like. We +definitely need this in the maths research community. +*** 2018-05-11: rnn-fsm + :PROPERTIES: + :CUSTOM_ID: rnn-fsm + :END: +Related to [[#neural-turing-machine][a previous micropost]]. + +[[http://www.cs.toronto.edu/~rgrosse/csc321/lec9.pdf][These slides from +Toronto]] are a nice introduction to RNN (recurrent neural network) from +a computational point of view. It states that RNN can simulate any FSM +(finite state machine, a.k.a. finite automata abbr. FA) with a toy +example computing the parity of a binary string. + +[[http://www.deeplearningbook.org/contents/rnn.html][Goodfellow et. +al.'s book]] (see page 372 and 374) goes one step further, stating that +RNN with a hidden-to-hidden layer can simulate Turing machines, and not +only that, but also the /universal/ Turing machine abbr. UTM (the book +referenced +[[https://www.sciencedirect.com/science/article/pii/S0022000085710136][Siegelmann-Sontag]]), +a property not shared by the weaker network where the hidden-to-hidden +layer is replaced by an output-to-hidden layer (page 376). + +By the way, the RNN with a hidden-to-hidden layer has the same +architecture as the so-called linear dynamical system mentioned in +[[https://www.coursera.org/learn/neural-networks/lecture/Fpa7y/modeling-sequences-a-brief-overview][Hinton's +video]]. + +From what I have learned, the universality of RNN and feedforward +networks are therefore due to different arguments, the former coming +from Turing machines and the latter from an analytical view of +approximation by step functions. +*** 2018-05-10: math-writing-decoupling + :PROPERTIES: + :CUSTOM_ID: math-writing-decoupling + :END: +One way to write readable mathematics is to decouple concepts. One idea +is the following template. First write a toy example with all the +important components present in this example, then analyse each +component individually and elaborate how (perhaps more complex) +variations of the component can extend the toy example and induce more +complex or powerful versions of the toy example. Through such +incremental development, one should be able to arrive at any result in +cutting edge research after a pleasant journey. + +It's a bit like the UNIX philosophy, where you have a basic system of +modules like IO, memory management, graphics etc, and modify / improve +each module individually (H/t [[http://nand2tetris.org/][NAND2Tetris]]). + +The book [[http://neuralnetworksanddeeplearning.com/][Neutral networks +and deep learning]] by Michael Nielsen is an example of such approach. +It begins the journey with a very simple neutral net with one hidden +layer, no regularisation, and sigmoid activations. It then analyses each +component including cost functions, the back propagation algorithm, the +activation functions, regularisation and the overall architecture (from +fully connected to CNN) individually and improve the toy example +incrementally. Over the course the accuracy of the example of mnist +grows incrementally from 95.42% to 99.67%. +*** 2018-05-09: neural-turing-machine + :PROPERTIES: + :CUSTOM_ID: neural-turing-machine + :END: + +#+begin_quote + One way RNNs are currently being used is to connect neural networks + more closely to traditional ways of thinking about algorithms, ways of + thinking based on concepts such as Turing machines and (conventional) + programming languages. [[https://arxiv.org/abs/1410.4615][A 2014 + paper]] developed an RNN which could take as input a + character-by-character description of a (very, very simple!) Python + program, and use that description to predict the output. Informally, + the network is learning to "understand" certain Python programs. + [[https://arxiv.org/abs/1410.5401][A second paper, also from 2014]], + used RNNs as a starting point to develop what they called a neural + Turing machine (NTM). This is a universal computer whose entire + structure can be trained using gradient descent. They trained their + NTM to infer algorithms for several simple problems, such as sorting + and copying. + + As it stands, these are extremely simple toy models. Learning to + execute the Python program =print(398345+42598)= doesn't make a + network into a full-fledged Python interpreter! It's not clear how + much further it will be possible to push the ideas. Still, the results + are intriguing. Historically, neural networks have done well at + pattern recognition problems where conventional algorithmic approaches + have trouble. Vice versa, conventional algorithmic approaches are good + at solving problems that neural nets aren't so good at. No-one today + implements a web server or a database program using a neural network! + It'd be great to develop unified models that integrate the strengths + of both neural networks and more traditional approaches to algorithms. + RNNs and ideas inspired by RNNs may help us do that. +#+end_quote + +Michael Nielsen, +[[http://neuralnetworksanddeeplearning.com/chap6.html#other_approaches_to_deep_neural_nets][Neural +networks and deep learning]] +*** 2018-05-09: neural-nets-activation + :PROPERTIES: + :CUSTOM_ID: neural-nets-activation + :END: + +#+begin_quote + What makes the rectified linear activation function better than the + sigmoid or tanh functions? At present, we have a poor understanding of + the answer to this question. Indeed, rectified linear units have only + begun to be widely used in the past few years. The reason for that + recent adoption is empirical: a few people tried rectified linear + units, often on the basis of hunches or heuristic arguments. They got + good results classifying benchmark data sets, and the practice has + spread. In an ideal world we'd have a theory telling us which + activation function to pick for which application. But at present + we're a long way from such a world. I should not be at all surprised + if further major improvements can be obtained by an even better choice + of activation function. And I also expect that in coming decades a + powerful theory of activation functions will be developed. Today, we + still have to rely on poorly understood rules of thumb and experience. +#+end_quote + +Michael Nielsen, +[[http://neuralnetworksanddeeplearning.com/chap6.html#convolutional_neural_networks_in_practice][Neutral +networks and deep learning]] +*** 2018-05-08: sql-injection-video + :PROPERTIES: + :CUSTOM_ID: sql-injection-video + :END: +Computerphile has some brilliant educational videos on computer science, +like [[https://www.youtube.com/watch?v=ciNHn38EyRc][a demo of SQL +injection]], [[https://www.youtube.com/watch?v=eis11j_iGMs][a toy +example of the lambda calculus]], and +[[https://www.youtube.com/watch?v=9T8A89jgeTI][explaining the Y +combinator]]. +*** 2018-05-08: nlp-arxiv + :PROPERTIES: + :CUSTOM_ID: nlp-arxiv + :END: +Primer Science is a tool by a startup called Primer that uses NLP to +summarize contents (but not single papers, yet) on arxiv. A developer of +this tool predicts in +[[https://twimlai.com/twiml-talk-136-taming-arxiv-w-natural-language-processing-with-john-bohannon/#][an +interview]] that progress on AI's ability to extract meanings from AI +research papers will be the biggest accelerant on AI research. +*** 2018-05-08: neural-nets-regularization + :PROPERTIES: + :CUSTOM_ID: neural-nets-regularization + :END: + +#+begin_quote + no-one has yet developed an entirely convincing theoretical + explanation for why regularization helps networks generalize. Indeed, + researchers continue to write papers where they try different + approaches to regularization, compare them to see which works better, + and attempt to understand why different approaches work better or + worse. And so you can view regularization as something of a kludge. + While it often helps, we don't have an entirely satisfactory + systematic understanding of what's going on, merely incomplete + heuristics and rules of thumb. + + There's a deeper set of issues here, issues which go to the heart of + science. It's the question of how we generalize. Regularization may + give us a computational magic wand that helps our networks generalize + better, but it doesn't give us a principled understanding of how + generalization works, nor of what the best approach is. +#+end_quote + +Michael Nielsen, +[[http://neuralnetworksanddeeplearning.com/chap3.html#why_does_regularization_help_reduce_overfitting][Neural +networks and deep learning]] +*** 2018-05-07: learning-knowledge-graph-reddit-journal-club + :PROPERTIES: + :CUSTOM_ID: learning-knowledge-graph-reddit-journal-club + :END: +It is a natural idea to look for ways to learn things like going through +a skill tree in a computer RPG. + +For example I made a +[[https://ypei.me/posts/2015-04-02-juggling-skill-tree.html][DAG for +juggling]]. + +Websites like [[https://knowen.org][Knowen]] and +[[https://metacademy.org][Metacademy]] explore this idea with added +flavour of open collaboration. + +The design of Metacademy looks quite promising. It also has a nice +tagline: "your package manager for knowledge". + +There are so so many tools to assist learning / research / knowledge +sharing today, and we should keep experimenting, in the hope that +eventually one of them will scale. + +On another note, I often complain about the lack of a place to discuss +math research online, but today I found on Reddit some journal clubs on +machine learning: +[[https://www.reddit.com/r/MachineLearning/comments/8aluhs/d_machine_learning_wayr_what_are_you_reading_week/][1]], +[[https://www.reddit.com/r/MachineLearning/comments/8elmd8/d_anyone_having_trouble_reading_a_particular/][2]]. +If only we had this for maths. On the other hand r/math does have some +interesting recurring threads as well: +[[https://www.reddit.com/r/math/wiki/everythingaboutx][Everything about +X]] and +[[https://www.reddit.com/r/math/search?q=what+are+you+working+on?+author:automoderator+&sort=new&restrict_sr=on&t=all][What +Are You Working On?]]. Hopefully these threads can last for years to +come. +*** 2018-05-02: simple-solution-lack-of-math-rendering + :PROPERTIES: + :CUSTOM_ID: simple-solution-lack-of-math-rendering + :END: +The lack of maths rendering in major online communication platforms like +instant messaging, email or Github has been a minor obsession of mine +for quite a while, as I saw it as a big factor preventing people from +talking more maths online. But today I realised this is totally a +non-issue. Just do what people on IRC have been doing since the +inception of the universe: use a (latex) pastebin. +*** 2018-05-01: neural-networks-programming-paradigm + :PROPERTIES: + :CUSTOM_ID: neural-networks-programming-paradigm + :END: + +#+begin_quote + Neural networks are one of the most beautiful programming paradigms + ever invented. In the conventional approach to programming, we tell + the computer what to do, breaking big problems up into many small, + precisely defined tasks that the computer can easily perform. By + contrast, in a neural network we don't tell the computer how to solve + our problem. Instead, it learns from observational data, figuring out + its own solution to the problem at hand. +#+end_quote + +Michael Nielsen - +[[http://neuralnetworksanddeeplearning.com/about.html][What this book +(Neural Networks and Deep Learning) is about]] + +Unrelated to the quote, note that Nielsen's book is licensed under +[[https://creativecommons.org/licenses/by-nc/3.0/deed.en_GB][CC BY-NC]], +so one can build on it and redistribute non-commercially. +*** 2018-04-30: google-search-not-ai + :PROPERTIES: + :CUSTOM_ID: google-search-not-ai + :END: + +#+begin_quote + But, users have learned to accommodate to Google not the other way + around. We know what kinds of things we can type into Google and what + we can't and we keep our searches to things that Google is likely to + help with. We know we are looking for texts and not answers to start a + conversation with an entity that knows what we really need to talk + about. People learn from conversation and Google can't have one. It + can pretend to have one using Siri but really those conversations tend + to get tiresome when you are past asking about where to eat. +#+end_quote + +Roger Schank - +[[http://www.rogerschank.com/fraudulent-claims-made-by-IBM-about-Watson-and-AI][Fraudulent +claims made by IBM about Watson and AI]] +*** 2018-04-06: hacker-ethics + :PROPERTIES: + :CUSTOM_ID: hacker-ethics + :END: + +#+begin_quote + + - Access to computers---and anything that might teach you something + about the way the world works---should be unlimited and total. + Always yield to the Hands-On Imperative! + - All information should be free. + - Mistrust Authority---Promote Decentralization. + - Hackers should be judged by their hacking, not bogus criteria such + as degrees, age, race, or position. + - You can create art and beauty on a computer. + - Computers can change your life for the better. +#+end_quote + +[[https://en.wikipedia.org/wiki/Hacker_ethic][The Hacker Ethic]], +[[https://en.wikipedia.org/wiki/Hackers:_Heroes_of_the_Computer_Revolution][Hackers: +Heroes of Computer Revolution]], by Steven Levy +*** 2018-03-23: static-site-generator + :PROPERTIES: + :CUSTOM_ID: static-site-generator + :END: + +#+begin_quote + "Static site generators seem like music databases, in that everyone + eventually writes their own crappy one that just barely scratches the + itch they had (and I'm no exception)." +#+end_quote + +__david__@hackernews + +So did I. diff --git a/pages/blog.org b/pages/blog.org new file mode 100644 index 0000000..d8928f5 --- /dev/null +++ b/pages/blog.org @@ -0,0 +1,20 @@ +#+TITLE: All posts + +- *[[file:posts/2019-03-14-great-but-manageable-expectations.org][Great but Manageable Expectations]]* - 2019-03-14 +- *[[file:posts/2019-03-13-a-tail-of-two-densities.org][A Tail of Two Densities]]* - 2019-03-13 +- *[[file:posts/2019-02-14-raise-your-elbo.org][Raise your ELBO]]* - 2019-02-14 +- *[[file:posts/2019-01-03-discriminant-analysis.org][Discriminant analysis]]* - 2019-01-03 +- *[[file:posts/2018-12-02-lime-shapley.org][Shapley, LIME and SHAP]]* - 2018-12-02 +- *[[file:posts/2018-06-03-automatic_differentiation.org][Automatic differentiation]]* - 2018-06-03 +- *[[file:posts/2018-04-10-update-open-research.org][Updates on open research]]* - 2018-04-29 +- *[[file:posts/2017-08-07-mathematical_bazaar.org][The Mathematical Bazaar]]* - 2017-08-07 +- *[[file:posts/2017-04-25-open_research_toywiki.org][Open mathematical research and launching toywiki]]* - 2017-04-25 +- *[[file:posts/2016-10-13-q-robinson-schensted-knuth-polymer.org][A \(q\)-Robinson-Schensted-Knuth algorithm and a \(q\)-polymer]]* - 2016-10-13 +- *[[file:posts/2015-07-15-double-macdonald-polynomials-macdonald-superpolynomials.org][AMS review of 'Double Macdonald polynomials as the stable limit of Macdonald superpolynomials' by Blondeau-Fournier, Lapointe and Mathieu]]* - 2015-07-15 +- *[[file:posts/2015-07-01-causal-quantum-product-levy-area.org][On a causal quantum double product integral related to Lévy stochastic area.]]* - 2015-07-01 +- *[[file:posts/2015-05-30-infinite-binary-words-containing-repetitions-odd-periods.org][AMS review of 'Infinite binary words containing repetitions of odd period' by Badkobeh and Crochemore]]* - 2015-05-30 +- *[[file:posts/2015-04-02-juggling-skill-tree.org][jst]]* - 2015-04-02 +- *[[file:posts/2015-04-01-unitary-double-products.org][Unitary causal quantum stochastic double products as universal]]* - 2015-04-01 +- *[[file:posts/2015-01-20-weighted-interpretation-super-catalan-numbers.org][AMS review of 'A weighted interpretation for the super Catalan]]* - 2015-01-20 +- *[[file:posts/2014-04-01-q-robinson-schensted-symmetry-paper.org][Symmetry property of \(q\)-weighted Robinson-Schensted algorithms and branching algorithms]]* - 2014-04-01 +- *[[file:posts/2013-06-01-q-robinson-schensted-paper.org][A \(q\)-weighted Robinson-Schensted algorithm]]* - 2013-06-01
\ No newline at end of file diff --git a/pages/microblog.org b/pages/microblog.org new file mode 100644 index 0000000..fb39a67 --- /dev/null +++ b/pages/microblog.org @@ -0,0 +1,683 @@ +#+TITLE: Microblog + +- 2020-08-02 - *[[file:microposts/ia-lawsuit.org][ia-lawsuit]]* + + The four big publishers Hachette, HarperCollins, Wiley, and Penguin + Random House are still pursuing Internet Archive. + + #+begin_quote + [Their] lawsuit does not stop at seeking to end the practice of + Controlled Digital Lending. These publishers call for the destruction + of the 1.5 million digital books that Internet Archive makes available + to our patrons. This form of digital book burning is unprecedented and + unfairly disadvantages people with print disabilities. For the blind, + ebooks are a lifeline, yet less than one in ten exists in accessible + formats. Since 2010, Internet Archive has made our lending library + available to the blind and print disabled community, in addition to + sighted users. If the publishers are successful with their lawsuit, + more than a million of those books would be deleted from the + Internet's digital shelves forever. + #+end_quote + + [[https://blog.archive.org/2020/07/29/internet-archive-responds-to-publishers-lawsuit/][Libraries + lend books, and must continue to lend books: Internet Archive responds + to publishers' lawsuit]] +- 2020-08-02 - *[[file:microposts/fsf-membership.org][fsf-membership]]* + + I am a proud associate member of Free Software Freedom. For me the + philosophy of Free Software is about ensuring the enrichment of a + digital commons, so that knowledge and information are not concentrated + in the hands of selected privileged people and locked up as + "intellectual property". The genius of copyleft licenses like GNU (A)GPL + ensures software released for the public, remains public. Open source + does not care about that. + + If you also care about the public good, the hacker ethics, or the spirit + of the web, please take a moment to consider joining FSF as an associate + member. It comes with [[https://www.fsf.org/associate/benefits][numerous + perks and benefits]]. +- 2020-06-21 - *[[file:microposts/how-can-you-help-ia.org][how-can-you-help-ia]]* + + [[https://blog.archive.org/2020/06/14/how-can-you-help-the-internet-archive/][How + can you help the Internet Archive?]] Use it. It's more than the Wayback + Machine. And get involved. +- 2020-06-12 - *[[file:microposts/open-library.org][open-library]]* + + Open Library was cofounded by Aaron Swartz. As part of the Internet + Archive, it has done good work to spread knowledge. However it is + currently + [[https://arstechnica.com/tech-policy/2020/06/internet-archive-ends-emergency-library-early-to-appease-publishers/][being + sued by four major publishers]] for the + [[https://archive.org/details/nationalemergencylibrary][National + Emergency Library]]. IA decided to + [[https://blog.archive.org/2020/06/10/temporary-national-emergency-library-to-close-2-weeks-early-returning-to-traditional-controlled-digital-lending/][close + the NEL two weeks earlier than planned]], but the lawsuit is not over, + which in the worst case scenario has the danger of resulting in + Controlled Digital Lending being considered illegal and (less likely) + bancruptcy of the Internet Archive. If this happens it will be a big + setback of the free-culture movement. +- 2020-04-15 - *[[file:microposts/sanders-suspend-campaign.org][sanders-suspend-campaign]]* + + Suspending the campaign is different from dropping out of the race. + Bernie Sanders remains on the ballot, and indeed in his campaign + suspension speech he encouraged people to continue voting for him in the + democratic primaries to push for changes in the convention. +- 2019-09-30 - *[[file:microposts/defense-stallman.org][defense-stallman]]* + + Someone wrote a bold article titled + [[https://geoff.greer.fm/2019/09/30/in-defense-of-richard-stallman/]["In + Defense of Richard Stallman"]]. Kudos to him. + + Also, an interesting read: + [[https://cfenollosa.com/blog/famous-computer-public-figure-suffers-the-consequences-for-asshole-ish-behavior.html][Famous + public figure in tech suffers the consequences for asshole-ish + behavior]]. +- 2019-09-29 - *[[file:microposts/stallman-resign.org][stallman-resign]]* + + Last week Richard Stallman resigned from FSF. It is a great loss for the + free software movement. + + The apparent cause of his resignation and the events that triggered it + reflect some alarming trends of the zeitgeist. Here is a detailed review + of what happened: [[https://sterling-archermedes.github.io/][Low grade + "journalists" and internet mob attack RMS with lies. In-depth review.]]. + Some interesting articles on this are: + [[https://jackbaruth.com/?p=16779][Weekly Roundup: The Passion Of Saint + iGNUcius Edition]], + [[http://techrights.org/2019/09/17/rms-witch-hunt/][Why I Once Called + for Richard Stallman to Step Down]]. + + Dishonest and misleading media pieces involved in this incident include + [[https://www.thedailybeast.com/famed-mit-computer-scientist-richard-stallman-defends-epstein-victims-were-entirely-willing][The + Daily Beast]], + [[https://www.vice.com/en_us/article/9ke3ke/famed-computer-scientist-richard-stallman-described-epstein-victims-as-entirely-willing][Vice]], + [[https://techcrunch.com/2019/09/16/computer-scientist-richard-stallman-who-defended-jeffrey-epstein-resigns-from-mit-csail-and-the-free-software-foundation/][Tech + Crunch]], + [[https://www.wired.com/story/richard-stallmans-exit-heralds-a-new-era-in-tech/][Wired]]. +- 2019-03-16 - *[[file:microposts/decss-haiku.org][decss-haiku]]* + + #+begin_quote + #+begin_example + Muse! When we learned to + count, little did we know all + the things we could do + + some day by shuffling + those numbers: Pythagoras + said "All is number" + + long before he saw + computers and their effects, + or what they could do + + by computation, + naive and mechanical + fast arithmetic. + + It changed the world, it + changed our consciousness and lives + to have such fast math + + available to + us and anyone who cared + to learn programming. + + Now help me, Muse, for + I wish to tell a piece of + controversial math, + + for which the lawyers + of DVD CCA + don't forbear to sue: + + that they alone should + know or have the right to teach + these skills and these rules. + + (Do they understand + the content, or is it just + the effects they see?) + + And all mathematics + is full of stories (just read + Eric Temple Bell); + + and CSS is + no exception to this rule. + Sing, Muse, decryption + + once secret, as all + knowledge, once unknown: how to + decrypt DVDs. + #+end_example + #+end_quote + + Seth Schoen, [[https://en.wikipedia.org/wiki/DeCSS_haiku][DeCSS haiku]] +- 2019-01-27 - *[[file:microposts/learning-undecidable.org][learning-undecidable]]* + + My take on the + [[https://www.nature.com/articles/s42256-018-0002-3][Nature paper + /Learning can be undecidable/]]: + + Fantastic article, very clearly written. + + So it reduces a kind of learninability called estimating the maximum + (EMX) to the cardinality of real numbers which is undecidable. + + When it comes to the relation between EMX and the rest of machine + learning framework, the article mentions that EMX belongs to "extensions + of PAC learnability include Vapnik's statistical learning setting and + the equivalent general learning setting by Shalev-Shwartz and + colleagues" (I have no idea what these two things are), but it does not + say whether EMX is representative of or reduces to common learning + tasks. So it is not clear whether its undecidability applies to ML at + large. + + Another condition to the main theorem is the union bounded closure + assumption. It seems a reasonable property of a family of sets, but then + again I wonder how that translates to learning. + + The article says "By now, we know of quite a few independence [from + mathematical axioms] results, mostly for set theoretic questions like + the continuum hypothesis, but also for results in algebra, analysis, + infinite combinatorics and more. Machine learning, so far, has escaped + this fate." but the description of the EMX learnability makes it more + like a classical mathematical / theoretical computer science problem + rather than machine learning. + + An insightful conclusion: "How come learnability can neither be proved + nor refuted? A closer look reveals that the source of the problem is in + defining learnability as the existence of a learning function rather + than the existence of a learning algorithm. In contrast with the + existence of algorithms, the existence of functions over infinite + domains is a (logically) subtle issue." + + In relation to practical problems, it uses an example of ad targeting. + However, A lot is lost in translation from the main theorem to this ad + example. + + The EMX problem states: given a domain X, a distribution P over X which + is unknown, some samples from P, and a family of subsets of X called F, + find A in F that approximately maximises P(A). + + The undecidability rests on X being the continuous [0, 1] interval, and + from the insight, we know the problem comes from the cardinality of + subsets of the [0, 1] interval, which is "logically subtle". + + In the ad problem, the domain X is all potential visitors, which is + finite because there are finite number of people in the world. In this + case P is a categorical distribution over the 1..n where n is the + population of the world. One can have a good estimate of the parameters + of a categorical distribution by asking for sufficiently large number of + samples and computing the empirical distribution. Let's call the + estimated distribution Q. One can choose the from F (also finite) the + set that maximises Q(A) which will be a solution to EMX. + + In other words, the theorem states: EMX is undecidable because not all + EMX instances are decidable, because there are some nasty ones due to + infinities. That does not mean no EMX instance is decidable. And I think + the ad instance is decidable. Is there a learning task that actually + corresponds to an undecidable EMX instance? I don't know, but I will not + believe the result of this paper is useful until I see one. + + h/t Reynaldo Boulogne +- 2018-12-11 - *[[file:microposts/gavin-belson.org][gavin-belson]]* + + #+begin_quote + I don't know about you people, but I don't want to live in a world + where someone else makes the world a better place better than we do. + #+end_quote + + Gavin Belson, Silicon Valley S2E1. + + I came across this quote in + [[https://slate.com/business/2018/12/facebook-emails-lawsuit-embarrassing-mark-zuckerberg.html][a + Slate post about Facebook]] +- 2018-10-05 - *[[file:microposts/margins.org][margins]]* + + With Fermat's Library's new tool + [[https://fermatslibrary.com/margins][margins]], you can host your own + journal club. +- 2018-09-18 - *[[file:microposts/rnn-turing.org][rnn-turing]]* + + Just some non-rigorous guess / thought: Feedforward networks are like + combinatorial logic, and recurrent networks are like sequential logic + (e.g. data flip-flop is like the feedback connection in RNN). Since NAND + - combinatorial logic + sequential logic = von Neumann machine which is + an approximation of the Turing machine, it is not surprising that RNN + (with feedforward networks) is Turing complete (assuming that neural + networks can learn the NAND gate). +- 2018-09-07 - *[[file:microposts/zitierkartell.org][zitierkartell]]* + + [[https://academia.stackexchange.com/questions/116489/counter-strategy-against-group-that-repeatedly-does-strategic-self-citations-and][Counter + strategy against group that repeatedly does strategic self-citations and + ignores other relevant research]] +- 2018-09-05 - *[[file:microposts/short-science.org][short-science]]* + + #+begin_quote + + + - ShortScience.org is a platform for post-publication discussion + aiming to improve accessibility and reproducibility of research + ideas. + - The website has over 800 summaries, mostly in machine learning, + written by the community and organized by paper, conference, and + year. + - Reading summaries of papers is useful to obtain the perspective and + insight of another reader, why they liked or disliked it, and their + attempt to demystify complicated sections. + - Also, writing summaries is a good exercise to understand the content + of a paper because you are forced to challenge your assumptions when + explaining it. + - Finally, you can keep up to date with the flood of research by + reading the latest summaries on our Twitter and Facebook pages. + #+end_quote + + [[https://shortscience.org][ShortScience.org]] +- 2018-08-13 - *[[file:microposts/darknet-diaries.org][darknet-diaries]]* + + [[https://darknetdiaries.com][Darknet Diaries]] is a cool podcast. + According to its about page it covers "true stories from the dark side + of the Internet. Stories about hackers, defenders, threats, malware, + botnets, breaches, and privacy." +- 2018-06-20 - *[[file:microposts/coursera-basic-income.org][coursera-basic-income]]* + + Coursera is having + [[https://www.coursera.org/learn/exploring-basic-income-in-a-changing-economy][a + Teach-Out on Basic Income]]. +- 2018-06-19 - *[[file:microposts/pun-generator.org][pun-generator]]* + + [[https://en.wikipedia.org/wiki/Computational_humor#Pun_generation][Pun + generators exist]]. +- 2018-06-15 - *[[file:microposts/hackers-excerpt.org][hackers-excerpt]]* + + #+begin_quote + But as more nontechnical people bought computers, the things that + impressed hackers were not as essential. While the programs themselves + had to maintain a certain standard of quality, it was quite possible + that the most exacting standards---those applied by a hacker who + wanted to add one more feature, or wouldn't let go of a project until + it was demonstrably faster than anything else around---were probably + counterproductive. What seemed more important was marketing. There + were plenty of brilliant programs which no one knew about. Sometimes + hackers would write programs and put them in the public domain, give + them away as easily as John Harris had lent his early copy of + Jawbreaker to the guys at the Fresno computer store. But rarely would + people ask for public domain programs by name: they wanted the ones + they saw advertised and discussed in magazines, demonstrated in + computer stores. It was not so important to have amazingly clever + algorithms. Users would put up with more commonplace ones. + + The Hacker Ethic, of course, held that every program should be as good + as you could make it (or better), infinitely flexible, admired for its + brilliance of concept and execution, and designed to extend the user's + powers. Selling computer programs like toothpaste was heresy. But it + was happening. Consider the prescription for success offered by one of + a panel of high-tech venture capitalists, gathered at a 1982 software + show: "I can summarize what it takes in three words: marketing, + marketing, marketing." When computers are sold like toasters, programs + will be sold like toothpaste. The Hacker Ethic notwithstanding. + #+end_quote + + [[http://www.stevenlevy.com/index.php/books/hackers][Hackers: Heroes of + Computer Revolution]], by Steven Levy. +- 2018-06-11 - *[[file:microposts/catalan-overflow.org][catalan-overflow]]* + + To compute Catalan numbers without unnecessary overflow, use the + recurrence formula \(C_n = {4 n - 2 \over n + 1} C_{n - 1}\). +- 2018-06-04 - *[[file:microposts/boyer-moore.org][boyer-moore]]* + + The + [[https://en.wikipedia.org/wiki/Boyer–Moore_majority_vote_algorithm][Boyer-Moore + algorithm for finding the majority of a sequence of elements]] falls in + the category of "very clever algorithms". + + #+begin_example + int majorityElement(vector<int>& xs) { + int count = 0; + int maj = xs[0]; + for (auto x : xs) { + if (x == maj) count++; + else if (count == 0) maj = x; + else count--; + } + return maj; + } + #+end_example +- 2018-05-30 - *[[file:microposts/how-to-learn-on-your-own.org][how-to-learn-on-your-own]]* + + Roger Grosse's post + [[https://metacademy.org/roadmaps/rgrosse/learn_on_your_own][How to + learn on your own (2015)]] is an excellent modern guide on how to learn + and research technical stuff (especially machine learning and maths) on + one's own. +- 2018-05-25 - *[[file:microposts/2048-mdp.org][2048-mdp]]* + + [[http://jdlm.info/articles/2018/03/18/markov-decision-process-2048.html][This + post]] models 2048 as an MDP and solves it using policy iteration and + backward induction. +- 2018-05-22 - *[[file:microposts/ats.org][ats]]* + + #+begin_quote + ATS (Applied Type System) is a programming language designed to unify + programming with formal specification. ATS has support for combining + theorem proving with practical programming through the use of advanced + type systems. A past version of The Computer Language Benchmarks Game + has demonstrated that the performance of ATS is comparable to that of + the C and C++ programming languages. By using theorem proving and + strict type checking, the compiler can detect and prove that its + implemented functions are not susceptible to bugs such as division by + zero, memory leaks, buffer overflow, and other forms of memory + corruption by verifying pointer arithmetic and reference counting + before the program compiles. Additionally, by using the integrated + theorem-proving system of ATS (ATS/LF), the programmer may make use of + static constructs that are intertwined with the operative code to + prove that a function attains its specification. + #+end_quote + + [[https://en.wikipedia.org/wiki/ATS_(programming_language)][Wikipedia + entry on ATS]] +- 2018-05-20 - *[[file:microposts/bostoncalling.org][bostoncalling]]* + + (5-second fame) I sent a picture of my kitchen sink to BBC and got + mentioned in the [[https://www.bbc.co.uk/programmes/w3cswg8c][latest + Boston Calling episode]] (listen at 25:54). +- 2018-05-18 - *[[file:microposts/colah-blog.org][colah-blog]]* + + [[https://colah.github.io/][colah's blog]] has a cool feature that + allows you to comment on any paragraph of a blog post. Here's an + [[https://colah.github.io/posts/2015-08-Understanding-LSTMs/][example]]. + If it is doable on a static site hosted on Github pages, I suppose it + shouldn't be too hard to implement. This also seems to work more + seamlessly than [[https://fermatslibrary.com/][Fermat's Library]], + because the latter has to embed pdfs in webpages. Now fantasy time: + imagine that one day arXiv shows html versions of papers (through author + uploading or conversion from TeX) with this feature. +- 2018-05-15 - *[[file:microposts/random-forests.org][random-forests]]* + + [[https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/info][Stanford + Lagunita's statistical learning course]] has some excellent lectures on + random forests. It starts with explanations of decision trees, followed + by bagged trees and random forests, and ends with boosting. From these + lectures it seems that: + + 1. The term "predictors" in statistical learning = "features" in machine + learning. + 1. The main idea of random forests of dropping predictors for individual + trees and aggregate by majority or average is the same as the idea of + dropout in neural networks, where a proportion of neurons in the + hidden layers are dropped temporarily during different minibatches of + training, effectively averaging over an emsemble of subnetworks. Both + tricks are used as regularisations, i.e. to reduce the variance. The + only difference is: in random forests, all but a square root number + of the total number of features are dropped, whereas the dropout + ratio in neural networks is usually a half. + + By the way, here's a comparison between statistical learning and machine + learning from the slides of the Statistcal Learning course: +- 2018-05-14 - *[[file:microposts/open-review-net.org][open-review-net]]* + + Open peer review means peer review process where communications + e.g. comments and responses are public. + + Like [[https://scipost.org/][SciPost]] mentioned in + [[file:/posts/2018-04-10-update-open-research.html][my post]], + [[https://openreview.net][OpenReview.net]] is an example of open peer + review in research. It looks like their focus is machine learning. Their + [[https://openreview.net/about][about page]] states their mission, and + here's [[https://openreview.net/group?id=ICLR.cc/2018/Conference][an + example]] where you can click on each entry to see what it is like. We + definitely need this in the maths research community. +- 2018-05-11 - *[[file:microposts/rnn-fsm.org][rnn-fsm]]* + + Related to [[file:neural-turing-machine][a previous micropost]]. + + [[http://www.cs.toronto.edu/~rgrosse/csc321/lec9.pdf][These slides from + Toronto]] are a nice introduction to RNN (recurrent neural network) from + a computational point of view. It states that RNN can simulate any FSM + (finite state machine, a.k.a. finite automata abbr. FA) with a toy + example computing the parity of a binary string. + + [[http://www.deeplearningbook.org/contents/rnn.html][Goodfellow et. + al.'s book]] (see page 372 and 374) goes one step further, stating that + RNN with a hidden-to-hidden layer can simulate Turing machines, and not + only that, but also the /universal/ Turing machine abbr. UTM (the book + referenced + [[https://www.sciencedirect.com/science/article/pii/S0022000085710136][Siegelmann-Sontag]]), + a property not shared by the weaker network where the hidden-to-hidden + layer is replaced by an output-to-hidden layer (page 376). + + By the way, the RNN with a hidden-to-hidden layer has the same + architecture as the so-called linear dynamical system mentioned in + [[https://www.coursera.org/learn/neural-networks/lecture/Fpa7y/modeling-sequences-a-brief-overview][Hinton's + video]]. + + From what I have learned, the universality of RNN and feedforward + networks are therefore due to different arguments, the former coming + from Turing machines and the latter from an analytical view of + approximation by step functions. +- 2018-05-10 - *[[file:microposts/math-writing-decoupling.org][math-writing-decoupling]]* + + One way to write readable mathematics is to decouple concepts. One idea + is the following template. First write a toy example with all the + important components present in this example, then analyse each + component individually and elaborate how (perhaps more complex) + variations of the component can extend the toy example and induce more + complex or powerful versions of the toy example. Through such + incremental development, one should be able to arrive at any result in + cutting edge research after a pleasant journey. + + It's a bit like the UNIX philosophy, where you have a basic system of + modules like IO, memory management, graphics etc, and modify / improve + each module individually (H/t [[http://nand2tetris.org/][NAND2Tetris]]). + + The book [[http://neuralnetworksanddeeplearning.com/][Neutral networks + and deep learning]] by Michael Nielsen is an example of such approach. + It begins the journey with a very simple neutral net with one hidden + layer, no regularisation, and sigmoid activations. It then analyses each + component including cost functions, the back propagation algorithm, the + activation functions, regularisation and the overall architecture (from + fully connected to CNN) individually and improve the toy example + incrementally. Over the course the accuracy of the example of mnist + grows incrementally from 95.42% to 99.67%. +- 2018-05-09 - *[[file:microposts/neural-nets-activation.org][neural-nets-activation]]* + + #+begin_quote + What makes the rectified linear activation function better than the + sigmoid or tanh functions? At present, we have a poor understanding of + the answer to this question. Indeed, rectified linear units have only + begun to be widely used in the past few years. The reason for that + recent adoption is empirical: a few people tried rectified linear + units, often on the basis of hunches or heuristic arguments. They got + good results classifying benchmark data sets, and the practice has + spread. In an ideal world we'd have a theory telling us which + activation function to pick for which application. But at present + we're a long way from such a world. I should not be at all surprised + if further major improvements can be obtained by an even better choice + of activation function. And I also expect that in coming decades a + powerful theory of activation functions will be developed. Today, we + still have to rely on poorly understood rules of thumb and experience. + #+end_quote + + Michael Nielsen, + [[http://neuralnetworksanddeeplearning.com/chap6.html#convolutional_neural_networks_in_practice][Neutral + networks and deep learning]] +- 2018-05-09 - *[[file:microposts/neural-turing-machine.org][neural-turing-machine]]* + + #+begin_quote + One way RNNs are currently being used is to connect neural networks + more closely to traditional ways of thinking about algorithms, ways of + thinking based on concepts such as Turing machines and (conventional) + programming languages. [[https://arxiv.org/abs/1410.4615][A 2014 + paper]] developed an RNN which could take as input a + character-by-character description of a (very, very simple!) Python + program, and use that description to predict the output. Informally, + the network is learning to "understand" certain Python programs. + [[https://arxiv.org/abs/1410.5401][A second paper, also from 2014]], + used RNNs as a starting point to develop what they called a neural + Turing machine (NTM). This is a universal computer whose entire + structure can be trained using gradient descent. They trained their + NTM to infer algorithms for several simple problems, such as sorting + and copying. + + As it stands, these are extremely simple toy models. Learning to + execute the Python program =print(398345+42598)= doesn't make a + network into a full-fledged Python interpreter! It's not clear how + much further it will be possible to push the ideas. Still, the results + are intriguing. Historically, neural networks have done well at + pattern recognition problems where conventional algorithmic approaches + have trouble. Vice versa, conventional algorithmic approaches are good + at solving problems that neural nets aren't so good at. No-one today + implements a web server or a database program using a neural network! + It'd be great to develop unified models that integrate the strengths + of both neural networks and more traditional approaches to algorithms. + RNNs and ideas inspired by RNNs may help us do that. + #+end_quote + + Michael Nielsen, + [[http://neuralnetworksanddeeplearning.com/chap6.html#other_approaches_to_deep_neural_nets][Neural + networks and deep learning]] +- 2018-05-08 - *[[file:microposts/nlp-arxiv.org][nlp-arxiv]]* + + Primer Science is a tool by a startup called Primer that uses NLP to + summarize contents (but not single papers, yet) on arxiv. A developer of + this tool predicts in + [[https://twimlai.com/twiml-talk-136-taming-arxiv-w-natural-language-processing-with-john-bohannon/#][an + interview]] that progress on AI's ability to extract meanings from AI + research papers will be the biggest accelerant on AI research. +- 2018-05-08 - *[[file:microposts/neural-nets-regularization.org][neural-nets-regularization]]* + + #+begin_quote + no-one has yet developed an entirely convincing theoretical + explanation for why regularization helps networks generalize. Indeed, + researchers continue to write papers where they try different + approaches to regularization, compare them to see which works better, + and attempt to understand why different approaches work better or + worse. And so you can view regularization as something of a kludge. + While it often helps, we don't have an entirely satisfactory + systematic understanding of what's going on, merely incomplete + heuristics and rules of thumb. + + There's a deeper set of issues here, issues which go to the heart of + science. It's the question of how we generalize. Regularization may + give us a computational magic wand that helps our networks generalize + better, but it doesn't give us a principled understanding of how + generalization works, nor of what the best approach is. + #+end_quote + + Michael Nielsen, + [[http://neuralnetworksanddeeplearning.com/chap3.html#why_does_regularization_help_reduce_overfitting][Neural + networks and deep learning]] +- 2018-05-08 - *[[file:microposts/sql-injection-video.org][sql-injection-video]]* + + Computerphile has some brilliant educational videos on computer science, + like [[https://www.youtube.com/watch?v=ciNHn38EyRc][a demo of SQL + injection]], [[https://www.youtube.com/watch?v=eis11j_iGMs][a toy + example of the lambda calculus]], and + [[https://www.youtube.com/watch?v=9T8A89jgeTI][explaining the Y + combinator]]. +- 2018-05-07 - *[[file:microposts/learning-knowledge-graph-reddit-journal-club.org][learning-knowledge-graph-reddit-journal-club]]* + + It is a natural idea to look for ways to learn things like going through + a skill tree in a computer RPG. + + For example I made a + [[https://ypei.me/posts/2015-04-02-juggling-skill-tree.html][DAG for + juggling]]. + + Websites like [[https://knowen.org][Knowen]] and + [[https://metacademy.org][Metacademy]] explore this idea with added + flavour of open collaboration. + + The design of Metacademy looks quite promising. It also has a nice + tagline: "your package manager for knowledge". + + There are so so many tools to assist learning / research / knowledge + sharing today, and we should keep experimenting, in the hope that + eventually one of them will scale. + + On another note, I often complain about the lack of a place to discuss + math research online, but today I found on Reddit some journal clubs on + machine learning: + [[https://www.reddit.com/r/MachineLearning/comments/8aluhs/d_machine_learning_wayr_what_are_you_reading_week/][1]], + [[https://www.reddit.com/r/MachineLearning/comments/8elmd8/d_anyone_having_trouble_reading_a_particular/][2]]. + If only we had this for maths. On the other hand r/math does have some + interesting recurring threads as well: + [[https://www.reddit.com/r/math/wiki/everythingaboutx][Everything about + X]] and + [[https://www.reddit.com/r/math/search?q=what+are+you+working+on?+author:automoderator+&sort=new&restrict_sr=on&t=all][What + Are You Working On?]]. Hopefully these threads can last for years to + come. +- 2018-05-02 - *[[file:microposts/simple-solution-lack-of-math-rendering.org][simple-solution-lack-of-math-rendering]]* + + The lack of maths rendering in major online communication platforms like + instant messaging, email or Github has been a minor obsession of mine + for quite a while, as I saw it as a big factor preventing people from + talking more maths online. But today I realised this is totally a + non-issue. Just do what people on IRC have been doing since the + inception of the universe: use a (latex) pastebin. +- 2018-05-01 - *[[file:microposts/neural-networks-programming-paradigm.org][neural-networks-programming-paradigm]]* + + #+begin_quote + Neural networks are one of the most beautiful programming paradigms + ever invented. In the conventional approach to programming, we tell + the computer what to do, breaking big problems up into many small, + precisely defined tasks that the computer can easily perform. By + contrast, in a neural network we don't tell the computer how to solve + our problem. Instead, it learns from observational data, figuring out + its own solution to the problem at hand. + #+end_quote + + Michael Nielsen - + [[http://neuralnetworksanddeeplearning.com/about.html][What this book + (Neural Networks and Deep Learning) is about]] + + Unrelated to the quote, note that Nielsen's book is licensed under + [[https://creativecommons.org/licenses/by-nc/3.0/deed.en_GB][CC BY-NC]], + so one can build on it and redistribute non-commercially. +- 2018-04-30 - *[[file:microposts/google-search-not-ai.org][google-search-not-ai]]* + + #+begin_quote + But, users have learned to accommodate to Google not the other way + around. We know what kinds of things we can type into Google and what + we can't and we keep our searches to things that Google is likely to + help with. We know we are looking for texts and not answers to start a + conversation with an entity that knows what we really need to talk + about. People learn from conversation and Google can't have one. It + can pretend to have one using Siri but really those conversations tend + to get tiresome when you are past asking about where to eat. + #+end_quote + + Roger Schank - + [[http://www.rogerschank.com/fraudulent-claims-made-by-IBM-about-Watson-and-AI][Fraudulent + claims made by IBM about Watson and AI]] +- 2018-04-06 - *[[file:microposts/hacker-ethics.org][hacker-ethics]]* + + #+begin_quote + + + - Access to computers---and anything that might teach you something + about the way the world works---should be unlimited and total. + Always yield to the Hands-On Imperative! + - All information should be free. + - Mistrust Authority---Promote Decentralization. + - Hackers should be judged by their hacking, not bogus criteria such + as degrees, age, race, or position. + - You can create art and beauty on a computer. + - Computers can change your life for the better. + #+end_quote + + [[https://en.wikipedia.org/wiki/Hacker_ethic][The Hacker Ethic]], + [[https://en.wikipedia.org/wiki/Hackers:_Heroes_of_the_Computer_Revolution][Hackers: + Heroes of Computer Revolution]], by Steven Levy +- 2018-03-23 - *[[file:microposts/static-site-generator.org][static-site-generator]]* + + #+begin_quote + "Static site generators seem like music databases, in that everyone + eventually writes their own crappy one that just barely scratches the + itch they had (and I'm no exception)." + #+end_quote + + __david__@hackernews + + So did I.
\ No newline at end of file |