Course Outline
DAY 1 - ARTIFICIAL NEURAL NETWORKS
Introduction to ANNs and Their Structure.
- Comparison between biological neurons and artificial neurons.
- Defining the ANM model.
- Activation functions employed in ANNs.
- Common categories of network architectures.
Mathematical Foundations and Learning Mechanisms.
- Review of vector and matrix algebra.
- Understanding state-space concepts.
- Core principles of optimization.
- Learning through error correction.
- Memory-based learning approaches.
- Hebbian learning principles.
- Competitive learning methods.
Single-Layer Perceptrons.
- Analyzing the structure and learning process of perceptrons.
- Introduction to pattern classifiers and Bayes' classifiers.
- Using perceptrons as pattern classifiers.
- The convergence of perceptrons.
- Inherent limitations of perceptrons.
Feedforward ANNs.
- Structural analysis of multi-layer feedforward networks.
- The backpropagation algorithm.
- Backpropagation: training processes and convergence.
- Applying backpropagation for functional approximation.
- Practical considerations and design challenges in backpropagation learning.
Radial Basis Function Networks.
- Concepts of pattern separability and interpolation.
- Theory of regularization.
- Applying regularization to RBF networks.
- Design and training procedures for RBF networks.
- Approximation characteristics of RBFs.
Competitive Learning and Self-Organizing ANNs.
- Standard clustering procedures.
- Learning Vector Quantization (LVQ).
- Various competitive learning algorithms and structures.
- Development of self-organizing feature maps.
- Key properties of feature maps.
Fuzzy Neural Networks.
- Overview of neuro-fuzzy systems.
- Fundamentals of fuzzy sets and logic.
- Designing fuzzy systems.
- Architecting fuzzy ANNs.
Applications
- Discussion of specific Neural Network applications, highlighting their benefits and associated challenges.
DAY 2 - MACHINE LEARNING
- The PAC Learning Framework
- Guarantees for finite hypothesis sets – consistent case
- Guarantees for finite hypothesis sets – inconsistent case
- General considerations
- Deterministic vs. Stochastic scenarios
- Bayes error noise
- Estimation and approximation errors
- Model selection strategies
- Raeder Complexity and VC Dimension
- The Bias-Variance tradeoff
- Regularization techniques
- Addressing over-fitting
- Validation methods
- Support Vector Machines
- Kriging (Gaussian Process regression)
- PCA and Kernel PCA
- Self-Organisation Maps (SOM)
- Kernel-induced vector spaces
- Mercer Kernels and Kernel-induced similarity metrics
- Reinforcement Learning
DAY 3 - DEEP LEARNING
Content will be integrated with topics covered in Day 1 and Day 2
- Logistic and Softmax Regression
- Sparse Autoencoders
- Vectorization, PCA, and Whitening
- Self-Taught Learning
- Deep Network Architectures
- Linear Decoders
- Convolution and Pooling
- Sparse Coding
- Independent Component Analysis
- Canonical Correlation Analysis
- Demonstrations and Practical Applications
Requirements
A solid grasp of mathematical principles.
Strong competency in fundamental statistical concepts.
While basic programming skills are not strictly mandatory, they are highly advisable for effective participation.
Testimonials (2)
Working from first principles in a focused way, and moving to applying case studies within the same day
Maggie Webb - Department of Jobs, Regions, and Precincts
Course - Artificial Neural Networks, Machine Learning, Deep Thinking
It was very interactive and more relaxed and informal than expected. We covered lots of topics in the time and the trainer was always receptive to talking more in detail or more generally about the topics and how they were related. I feel the training has given me the tools to continue learning as opposed to it being a one off session where learning stops once you've finished which is very important given the scale and complexity of the topic.
