CSU44005 – Optimisation Algorithms for Data Analysis

Module CodeCSU44005
Module Name Optimisation Algorithms for Data Analysis
ECTS Weighting[1]5 ECTS
Semester taughtSemester 2
Module Coordinator/s  Douglas Leith

Module Learning Outcomes

On successful completion of this module, students will be able to:

  1. Identify optimisation problems and their applications in data analysis and machine learning;
  2. Evaluate and design practical algorithms for solving machine learning optimisation problems;
  3. Compare different algorithms in terms of speed, complexity and scalability.

Module Content

The “learning” part of supervised “machine learning” almost always involves optimisation e.g. selecting model parameters to minimise a measure of prediction error. Commonly, some variant of gradient descent is used for this learning/training step, often in a “black box” fashion with standard/default settings.

In this module you’ll look inside the “black box” and learn how popular optimisation algorithms really work, their key properties (speed of convergence, scalability to big data etc) and the design choices/trade-offs involved. The module will take a hands-on approach and aims to develop practical understanding of the how to select and use optimisation algorithms for learning/training in supervised machine learning.

Topics that we’ll cover include:

Gradient-free algorithms e.g. for hyperparameter optimisation.

Popular gradient descent algorithms for ML: Adam, Adagrad, RMSprop, Polyak Momentum, Nesterov Acceleration;

Use of approximation/sampling for better scalability: stochastic gradient descent, mini-batch methods;

How to include constraints: projected gradient descent, penalty methods, Frank-Wolfe;

Assessment

Assessment ComponentBrief DescriptionLearning Outcomes Addressed% of TotalWeek SetWeek Due
Individual ProjectAssignment applying optimisation methods.LO1 – LO360%Week 8TBC
CourseworkWeekly assignmentsLO1 – LO340%Week 1TBC

Reassessment Details

Assignment (100%).

Contact Hours and Indicative Student Workload

Contact Hours (scheduled hours per student over full module), broken down by: 22 hours
Lecture17 hours
Laboratory5 hours
Independent study (outside scheduled contact hours), broken down by:72  hours
Preparation for classes and review of material (including preparation for examination, if applicable36 hours
completion of assessments (including examination, if applicable)36 hours
Total Hours94 hours

Recommended Reading List

.

Module Pre-requisitesCS7CS4/CSU44061: Introduction to Machine Learning (or equivalent). Familiarity with programming in python. Basic algebra (but this does not aim to be a maths module, we’ll focus on a hands-on approach).

Module Co-requisites

None

Module Website

Blackboard