Lecture 3

Theory

Lecture slides:

  • Hidden layers, activation function, ELM: Here

  • Backpropagation: Here

These hands-on exercises focus on

  • Highlighting having a linear hidden layer versus a non-linear one,

  • Evaluating the impact of using one, random value, hidden layer (ie. ELM) along with an activation function (e.g. ReLU, ELU, etc.) for the MNIST and CIFAR-10 datasets.

  • Integrating the Backpropagation (BP) algorithm to the SGD to fully train a network with one hidden layer for the multiclass classification problem (CIFAR-10 dataset).

Simulations