Mathematics and Applications of Machine Learning

MATH36160 Mathematics and Applications of Machine Learning 20 Full year Undergraduate Level 3 Jonas Latz Unit coordinator Stefan Guettel Unit coordinator 1 FHEQ level (Framework for Higher Education Qualifications) ' Last part of a Bachelors ' European Credit Transfer & Accumulation System Rating : 10.0 Machine learning and artificial intelligence have become a cornerstone of our everyday life. We have self-driving cars on our streets, use large language models to turn ideas into text, and converse with our digital assistant using voice recognition. In this module, we will step into the world of modern machine learning. We begin with an analytical treatment of supervised learning problems from an approximation theoretic viewpoint. We especially study linear and polynomial regression, as well as k-nearest neighbour and support vector machine classification. Based on the Bayesian formulation of classification and as a generalisation of logistic regression, we then introduce and discuss deep neural networks, also with respect to their properties as function approximators. As a training methodology, we introduce gradient descent and stochastic gradient descent algorithms. The first part of the course finishes with a discussion of unsupervised techniques, especially Gaussian mixture models and k-means.  The second part of the course starts with a broad introduction to the predictive modelling pipeline, beginning with in-memory data structures (such as Pandas DataFrames), basic data exploration and cleansing. The focus will then move to model selection, hyperparameter tuning, and model evaluation using the functionality embedded in the open source scikit-learn training software library. Hands-on coding experience will involve linear, polynomial, and logistic regression, k-nearest neighbours, decision trees, as well as different clustering techniques. Whenever possible, students will learn to code simple Python implementations of these methods from scratch to gain a deep algorithmic understanding of these techniques and critically question model outputs – especially with respect to the theory studied in the first part. The lecture material will be complemented with exercises and coding assessments. Machine learning and artificial intelligence have become a cornerstone of our everyday life. We have self-driving cars on our streets, use large language models to turn ideas into text, and converse with our digital assistant using voice recognition. In this module, we will step into the world of modern machine learning. We begin with an analytical treatment of supervised learning problems from an approximation theoretic viewpoint. We especially study linear and polynomial regression, as well as k-nearest neighbour and support vector machine classification. Based on the Bayesian formulation of classification and as a generalisation of logistic regression, we then introduce and discuss deep neural networks, also with respect to their properties as function approximators. As a training methodology, we introduce gradient descent and stochastic gradient descent algorithms. The first part of the course finishes with a discussion of unsupervised techniques, especially Gaussian mixture models and k-means.  The second part of the course starts with a broad introduction to the predictive modelling pipeline, beginning with in-memory data structures (such as Pandas DataFrames), basic data exploration and cleansing. The focus will then move to model selection, hyperparameter tuning, and model evaluation using the functionality embedded in the open source scikit-learn training software library. Hands-on coding experience will involve linear, polynomial, and logistic regression, k-nearest neighbours, decision trees, as well as different clustering techniques. Whenever possible, students will learn to code simple Python implementations of these methods from scratch to gain a deep algorithmic understanding of these techniques and critically question model outputs – especially with respect to the theory studied in the first part. The lecture material will be complemented with exercises and coding assessments.   The unit aims to give the students a rigorous analytical introduction to machine learning methodology in semester 1 that is complemented by an application-driven, computational perspective in semester 2.  ILO 1Distinguish important supervised machine learning models and analyse sources of errors. Explain model selection methodology and the role of approximation-estimation trade off.ILO 2Develop deep neural networks starting from logistic regression and articulate basics of universal approximation. Explain gradient descent and stochastic gradient descent and prove convergence when applied to appropriate target functions.ILO 3Distinguish supervised and unsupervised learning problems. Explain Gaussian mixture models and their role in clustering. Develop the expectation-maximisation algorithm as an approximate soft clustering method and the k-means algorithm as a hard clustering version of it.ILO 4Apply an appropriate framework in Python to handle data and to use data to select and train diverse machine learning models.  ILO 5Explain and implement basic machine learning models and their associated training procedure. 3 contact hours per week. Term 1: 2 hours of lectures, 1 hour of tutorials; Term 2: 1 hour of lectures, 2 hours of computer labs.  0 Other 10% 1 Written exam 45% 2 Written assignment (inc essay) 45% Online quiz in semester 1 Automatically marked online Exam at the end of semester  Generic feedback supplied after exam period.  Coursework 1 in semester  2 Individual feedback / automarkingCoursework 2 in semester  2 Individual feedback / automarkingGroup project  in semester  2 Group feedback MATH24411 Numerical Analysis 1 Pre-Requisite Compulsory PHYS20161 Introduction to Programming for Physicists Pre-Requisite Optional MATH20621 Programming with Python Pre-Requisite Optional COMP16321 Introduction to Programming 1 Pre-Requisite Optional MATH36160 Pre-Requisites: MATH24411, and (one of MATH20621 or COMP16321 or PHYS20161)Students must take MATH2441 and either MATH20621, COMP16321, or PHYS20161. N Bottom, Curtis, Nocedal (2018): Optimization Methods for Large-Scale Machine Learning, SIAM Review 60(2): 223-311. Higham, Higham (2018): Deep Learning: An Introduction for Applied Mathematicians, SIAM Review 61(4): 860-891. James, Witten, Hastie, Tibshirani, Taylor (2023): An Introduction to Statistical Learning with Applications in Python, Springer. Mohri, Rostamizadeh, Talwalkar (2018): Foundations of Machine Learning, second edition, MIT Press. Grus (2019): Data Science from Scratch, second edition, O’Reilly Estève et al. (2022). INRIA/scikit-learn-mooc: (session-3).  https://doi.org/10.5281/zenodo.7220307     Lectures 32 Practical classes & workshops 22 Tutorials 11 0 135