Convex Optimization

MATH36062 Convex Optimization 10 Semester 2 Undergraduate Level 3 Matthew Thorpe Unit coordinator Department of Mathematics 1 FHEQ level (Framework for Higher Education Qualifications) ' Last part of a Bachelors ' European Credit Transfer & Accumulation System Rating : 5.0 Optimization is the art of optimal decision making under constraints. Convex optimization refers to a set of problems and methods that can be formulated using convex functions and sets; countless problems from science, engineering and statistics can be cast as convex optimization problems and solved using efficient algorithms. The course is intended as an introduction to convex optimization, focussing on the theory, the modelling techniques, and the algorithm analysis and design. Recent developments such as convex regularization and compressed sensing will be discussed. The problem sessions will be used to present applications from machine learning, signal processing, and finance. Optimization is the art of optimal decision making under constraints. Convex optimization refers to a set of problems and methods that can be formulated using convex functions and sets; countless problems from science, engineering and statistics can be cast as convex optimization problems and solved using efficient algorithms. The course is intended as an introduction to convex optimization, focussing on the theory, the modelling techniques, and the algorithm analysis and design. Recent developments such as convex regularization and compressed sensing will be discussed. The problem sessions will be used to present applications from machine learning, signal processing, and finance. The course aims to introduce students to modern convex optimization and its applications in fields such as machine learning. The course is designed to cover practical modelling aspects, algorithm analysis and design, and the theoretical foundations of the subject. <table border="0" cellpadding="0" cellspacing="0" cols="6" id="ACE_UM_CRSE_DERIVED_UM_GROUPBOX8$0" role="presentation" width="791"> <tbody> <tr> <td align="left" valign="top"> <table> <tbody> <tr> <td align="left">On completion of the course, students should be able to: <ul> <li>recognise problems that can be formulated as convex optimization problem,</li> <li>describe and apply gradient descent and Newton’s method, and explain their performance and limitations,</li> <li>solve linear, quadratic and semidefinite programming problems using interior point methods, and evaluate their performance,</li> <li>derive the Lagrange dual of various standard optimization problems,</li> <li>characterise the solutions of optimization problems using optimality conditions such as the Karush Kuhn Tucker (KKT) conditions,</li> <li>explain the role of convex optimization in machine learning, signal processing, compressed sensing, and finance</li> </ul> </td> </tr> </tbody> </table> </td> </tr> </tbody></table> The lectures and problem sessions will cover the following topics:(1) Overview and examples of optimization problems; (2) Least-squares, gradient descent, Newton’s method; (3) Introduction to CVX; (4) Fundamentals of convex analysis and geometry: convex sets and functions, subdifferential calculus; (5) Linear and quadratic programming, semidefinite programming, conic optimization; (6) Optimality conditions, duality theory, theorems of alternative; (7) Interior-point methods, augmented Lagrangians, alternating direction method of multipliers; (8) Applications in machine learning: convex regularization, compressed sensing and matrix completion. 0 Other 20% 1 Written exam 80% Coursework: Weighting within unit 20%.End of semester examination: Weighting within unit 80%. Feedback tutorials will provide an opportunity for students' work to be discussed and provide feedback on their understanding.  Coursework or in-class tests (where applicable) also provide an opportunity for students to receive feedback.  Students can also get feedback on their understanding directly from the lecturer, for example during the lecturer's office hour. MATH10202 Linear Algebra A Pre-Requisite Compulsory MATH10212 Linear Algebra B Pre-Requisite Compulsory N The main reference is the book<ul> <li>Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2004. The book is available online at <a href="http://www.stanford.edu/~boyd/cvxbook/">http://www.stanford.edu/~boyd/cvxbook/</a>. The lecture will also make use of the CVX software, which is based on MATLAB.</li></ul> Other useful references include:<ul> <li>J. Nocedal and S.J.Wright. Numerical Optimization. Springer, 2006.</li> <li>A. Ben-Tal and A. Nemirovski. Lectures on Modern Convex Optimization. 2013.  </li></ul>CONVEX OPTIMIZATION 3<ul> <li>Y. Nesterov. Introductory lectures on convex optimization: A basic course. Springer, 2004.</li></ul> Lectures 12 Tutorials 12 0 76 The independent study hours will normally comprise the following. During each week of the taught part of the semester:·         You will normally have approximately 60-75 minutes of video content. Normally you would spend approximately 2-2.5 hrs per week studying this content independently·         You will normally have exercise or problem sheets, on which you might spend approximately 1.5hrs per week·         There may be other tasks assigned to you on Blackboard, for example short quizzes or short-answer formative exercises·         In some weeks you may be preparing coursework or revising for mid-semester testsTogether with the timetabled classes, you should be spending approximately 6 hours per week on this course unit.The remaining independent study time comprises revision for and taking the end-of-semester assessment.The above times are indicative only and may vary depending on the week and the course unit. More information can be found on the course unit’s Blackboard page.