| Module Code | CS7DS2 |
| Module Name | Optimisation Algorithms for Data Analysis |
| ECTS Weighting [1] | 5 ECTS |
| Semester Taught | Semester 2 |
| Module Coordinator/s | Professor Douglas Leith |
Module Learning Outcomes
On successful completion of this module, students will be able to:
- Identify optimisation problems and their applications in data analysis and machine learning;
- Evaluate and design practical algorithms for solving machine learning optimisation problems;
- Compare different algorithms in terms of speed, complexity and scalability.
Module Content
The “learning” part of supervised “machine learning” almost always involves optimisation e.g. selecting model parameters to minimise a measure of prediction error. Commonly, some variant of gradient descent is used for this learning/training step, often in a “black box” fashion with standard/default settings.
In this module you’ll look inside the “black box” and learn how popular optimisation algorithms really work, their key properties (speed of convergence, scalability to big data etc) and the design choices/trade-offs involved. The module will take a hands-on approach and aims to develop practical understanding of the how to select and use optimisation algorithms for learning/training in supervised machine learning.
Topics that we’ll cover include:
- Popular gradient descent algorithms for ML: Adam, Adagrad, RMSprop, Polyak Momentum, Nesterov Acceleration;
- Use of approximation/sampling for better scalability: stochastic gradient descent, mini-batch methods;
- How to include constraints: projected gradient descent, penalty methods, Frank-Wolfe;
- Gradient-free algorithms e.g. for hyperparameter optimisation.
Teaching and Learning Methods
Lectures and coursework.
Assessment Details
| Assessment Component | Brief Description | Learning Outcomes Addressed | % of Total | Week Set | Week Due |
| Individual Project | Assignment applying optimisation methods. | LO1 – LO3 | 60% | Week 8 | TBC |
| Coursework | Weekly assignments | LO1 – LO3 | 40% | Week 1 | TBC |
Reassessment Details
Assignment (100%).
Contact Hours and Indicative Student Workload
| Contact Hours (scheduled hours per student over full module), broken down by: | 22 hours |
| Lecture | 22 hours |
| Laboratory | 0 hours |
| Independent Study (outside scheduled contact hours), broken down by: | 94 hours |
| Preparation for classes and review of material (including preparation for examination, if applicable) | 44 hours |
| Completion of assessments (including examination, if applicable) | 50 hours |
| Total Hours | 116 hours |
Module Pre-requisites
Prerequisite modules:
CS7CS4/CSU44061: Introduction to Machine Learning (or equivalent). Familiarity with programming in python. Basic algebra (but this does not aim to be a maths module, we’ll focus on a hands-on approach).
Module Co-requisites
N/A