Reinforcement Learning

COMP64202 Reinforcement Learning 15 Semester 2 Postgraduate Taught Level 6 Michele Caprio Unit coordinator Department of Computer Science 1 FHEQ level (Framework for Higher Education Qualifications) ' Masters/Integrated Masters P4 ' European Credit Transfer & Accumulation System Rating : 7.5 Reinforcement Learning (COMP64202) is an advanced artificial intelligence course that explores how machines can learn to make sequential decisions through trial and error interaction with their environment, similar to how humans learn. The course covers both theoretical foundations and cutting-edge applications in areas such as game playing, robotics, and resource management. Reinforcement Learning (RL) is revolutionizing how machines learn to solve complex problems, from mastering games like Go to controlling robots and managing data centers. Unlike traditional machine learning approaches, RL enables systems to learn through trial and error, much like humans do.This course explores the key concepts and methods that make RL powerful, starting with fundamental tabular methods for Markov Decision Processes, before diving into how deep learning helps RL scale to real-world problems. We'll then cover advanced approaches including policy gradients and model-based methods, culminating in the challenges of partial observability and multi-agent systems. Throughout, we'll balance theoretical understanding with practical insights into how these methods are applied in real-world settings. The unit aims to provide an explanation of the key ideas and algorithms of reinforcement learning (RL). This course is meant to be both an introductory and advanced treatment of reinforcement learning, emphasizing foundations and ideas as well as the latest developments and mathematical proofs. This course tries to make the work accessible to the broadest possible audience in artificial intelligence, control engineering, operations research, psychology, and neuroscience.  ILO 1: To list the strengths and limitations of modern deep RL approaches.  ILO 2: To explain the underlying concepts of the discussed methods and how they differ from each other.  ILO 3: To derive the objectives and analyse the constraints of selected algorithms.ILO 4: To analyze a new task to decide which algorithms/architectures to apply.   ILO 5: To implement selected algorithms/architectures.     Analytical skills Group/team working Innovation/creativity Project management Problem solving Research Written communication The course covers the following topics:<ul><li>Fundamentals of Markov Decision Processes (MDPs)</li><li>Dynamic Programming methods</li><li>Monte Carlo methods</li><li>Temporal-Difference learning</li><li>Function approximation and Deep Reinforcement Learning</li><li>Policy gradient methods</li><li>Actor-critic algorithms</li><li>Model-based Reinforcement Learning</li><li>Partial observability and multi-agent systems</li><li>Real-world applications and case studies</li></ul> The learning and teaching in this course combines three complementary components. Lectures provide the theoretical foundations and key concepts of reinforcement learning. Tutorial sessions focus on solving theoretical exercises to deepen understanding and develop problem-solving skills. Computer lab sessions offer hands-on experience implementing and experimenting with reinforcement learning algorithms, allowing students to see the practical challenges and opportunities of these methods. 1 Written exam 80% 8 Practical skills assessment 20% Feedback is provided through:<ul><li>Cohort and individual feedback for the practical assignment when marks are returned</li><li>Discussion during tutorial sessions</li><li>Peer feedback during group activities</li><li>Office hours with teaching staff</li></ul> Y An Introduction to Reinforcement Learning, R.S. Sutton and A.G. Barto. MIT Press, 2018.Algorithms for Reinforcement Learning, C. Szepesvari, Morgan and Claypool, 2010. Assessment written exam 2 Lectures 20 Practical classes & workshops 8 Tutorials 5 0 115