Reader-friendly introductions to AI topics, for dummies (like me :-) )
A lot of “reader’s digest” versions of tutorial or famous articles :-)
(usually no original research, just an overview of existing works; I’ll try to satisfy all requests for clarifying something, because if it’s unclear, it means that it is not clear enough in my own mind and this matters a lot for me :-) )
Neural networks
LSTM, a great recurrent neural network (and a bit of GRU as well :-) )
BPTT with O(squareRoot(T)) space complexity
Generative models
Variational autoencoders for dummies
Normalizations in deep learning:
Batch normalization: https://docs.google.com/document/d/1zRB_rtnwC45QiCdcHrDe03d29OqksMZTEMF6V1jfZUk/edit#
Weight normalization: https://docs.google.com/document/d/1m_w6t99Xja_MhX0owGIDInZGVYr-EoHokCtBCsWtKnk/edit
Propagation normalization: https://docs.google.com/document/d/1c_4CFMhc3kf6ewXX5xfzPYTcdcNJ5FehgD-0CJtk0mk/edit#heading=h.96m2g4tzv75
Parallel optimization algorithms for (deep) neural networks
A paper by Google concluding that we should distribute stochastic gradient algorithms
Neural networks for text
Optimization algorithms for (deep) neural networks
online natural gradient for deep networks, no original research, just explaining existing papers (in progress).
a brief overview of the "learn the learning rate" paradigm by P.Y. Masse and Y. Ollivier (no original research, just an overview!)
a super-fast introduction to stochastic gradient descent variants
Portfolio for deeplearning: a preliminary work for choosing hyperparameters (includes quasi-random portfolios)
Optimization, others:
Optimizing hyperparameters by murder operators
our paper around comparison-based optimization is accepted at Gecco with excellent reviews. Why do we care about comparison-based optimization ? because most user-feedback-based optimization is based on comparisons :-)
a super-fast introduction to stochastic gradient descent variants
And adagrad for adaptive learning rate
Optimization for machine learning, others:
Applications to power systems:
Others:
Big Oh overview: here
Sometimes, cross-validation sucks (in particular leave-one-out)
Yet another graphical representation of Paxos protocol (with the last of these pictures I could really understand Paxos :-)
AlphaGo vs Fan Hui: notes (English and French) ( ici, vulgarisation en francais and more technical slides on deep neural networks (not convolutional in these slides) and Monte Carlo Tree Search)
