Notions et techniques abordées

Single-layer Neural Networks - Multi-layer perceptron - Radial basis functions - Error functions - Parameter optimization algorithms - Gradient descent optimization - Error Backpropagation - Evaluation of error-function derivatives - Efficiency of backpropagation - Regularization in Neural Networks - Early stopping - Convolutional networks - Pooling - Soft weight sharing - Feedforward Deep Networks - Training Criterion and Regularizer - Universal Approximation Properties and Depth - Feature / Representation Learning - Piecewise Linear Hidden Units - Classical Regularization as Noise Robustness - Semi-Supervised Training - Unsupervised Pretraining - Dropout - Multi-Task Learning - Optimization for training deep models - Unsupervised and Transfer Learning - Domain Adaptation - Auto-Encoders - Regularized Auto-Encoders - Representational Power, Layer Size and Depth - Sequence Modeling: Recurrent and Recursive Nets - Computing the gradient in a recurrent neural network - Auto-Regressive Networks - Deep generative models - Generative Adversarial Networks - Restricted Boltzmann machines - Deep belief networks - Deep Boltzmann machines - Convolutional Boltzmann machines - Large scale deep learning - Fast CPU, GPU and TPU implementation of deep models.