Representation Learning

COMP41021 Representation Learning 15 Semester 1 Undergraduate Level 4 Ke Chen Unit coordinator 1 FHEQ level (Framework for Higher Education Qualifications) ' Masters/Integrated Masters P4 ' European Credit Transfer & Accumulation System Rating : 7.5 A major component of machine learning and AI is dealing with extracting the useful information underlying data, in particular, for high dimensional data. The recent success of deep learning is also attributed to effective data representation. As an emerging area in machine learning, representation learning can extract features from raw data, discover explanatory factors of variation behind data and tackle tough issues arising from high dimensional data. Representation learning has been successfully applied to many domains such as computer vision, audio/speech information processing, natural language processing/understanding, robotics and a variety of medical applications. In this course, we will consider how to develop core algorithms for representation learning and gain insights into these algorithms from theoretical and empirical perspectives. We will demonstrate how essential algorithms are developed in a systematic way as well as how important algorithms can be applied through the use of examples and real world problems. We will cover related key approaches from machine learning, statistics and deep neural networks in this advanced machine learning course unit. A major component of machine learning and AI is dealing with extracting the useful information underlying data, in particular, for high dimensional data. The recent success of deep learning is also attributed to effective data representation. As an emerging area in machine learning, representation learning can extract features from raw data, discover explanatory factors of variation behind data and tackle tough issues arising from high dimensional data. Representation learning has been successfully applied to many domains such as computer vision, audio/speech information processing, natural language processing/understanding, robotics and a variety of medical applications. In this course, we will consider how to develop core algorithms for representation learning and gain insights into these algorithms from theoretical and empirical perspectives. We will demonstrate how essential algorithms are developed in a systematic way as well as how important algorithms can be applied through the use of examples and real world problems. We will cover related key approaches from machine learning, statistics and deep neural networks in this advanced machine learning course unit. This course unit aims to introduce students to classical and state-of-the-art approaches to representation learning and provides experience of research such as literature review and self-learning from research papers. In particular, transferable knowledge/skills essential to original researches are highlighted in this course unit. On successful completion of this unit, a student will be able to:<ul> <li> understand the general motivation and main ideas underlying representation learning</li> <li> describe the curse of dimensionality and its implication in different learning paradigms</li> <li> understand the advantages and the disadvantages of the learning algorithms studied in the course unit and decide which is appropriate for a particular application</li> <li> derive the linear representation learning algorithms from scratch</li> <li> apply the learning algorithms studied in the course unit to simple data sets for feature extraction related applications and visualisation of high-dimensional data</li> <li> implement the core learning algorithms studied in the course unit as well as apply those to real-world datasets</li> <li> evaluate the performance of the core learning algorithms studied in the course unit and whether such a learning algorithm is appropriate for a particular problem</li> <li> understand and appreciate main ideas underlying state-of-the-art representation learning algorithms</li> <li> make a connection between representation learning and other relevant areas in machine learning</li></ul> Analytical skills Innovation/creativity Project management Oral communication Problem solving Research Written communication <ul> <li>Foundation: representation learning background and mathematics essentials</li> <li>Linear models: principal component analysis and canonical correlation analysis</li> <li>Manifold learning: overview, MDS, ISOMAP and LLE algorithms</li> <li>Autoencoder: motivation, theoretical foundation, encoder/decoder architecture, evidence lower bound (ELBO) and variational  autoencoder (VAE)  </li></ul> Lecturesthree hours per week (5 weeks)Laboratoriesthree hours per week (5 weeks) 1 Written exam 50% 2 Written assignment (inc essay) 50% In general, feedback is available for the assessed work. For coursework, the feedback to individuals will be offered.For exam, the general feedback to the whole class will be given in writing. COMP61011 Foundations of Machine Learning Pre-Requisite Compulsory Y Lectures 15 Practical classes & workshops 15 120