Probability and Statistics 2

MATH27720 Probability and Statistics 2 20 Full year Undergraduate Level 2 Robert Gaunt Unit coordinator Korbinian Strimmer Unit coordinator Xiong Jin Unit coordinator Department of Mathematics 1 FHEQ level (Framework for Higher Education Qualifications) ' Middle part of Bachelors ' European Credit Transfer & Accumulation System Rating : 10.0 The first part of this unit continues the development of Probability theory from Year 1 and provides an important basis for many later courses in Probability and Finance.   The second part of this course unit provides students with the methodological foundations in model-based statistical learning, in particular likelihood estimation and inference and Bayesian learning. The theoretical and methodological discussions are complemented by practical computer application.  The first part of this unit continues the development of Probability theory from Year 1 and provides an important basis for many later courses in Probability and Finance.   The second part of this course unit provides students with the methodological foundations in model-based statistical learning, in particular likelihood estimation and inference and Bayesian learning. The theoretical and methodological discussions are complemented by practical computer application. This course unit forms the core of Theme C, and as such is delivered over 2 semesters.  Teaching is composed of two hours of lectures per week, and one tutorial class per fortnight.  Teaching materials will be uploaded to Blackboard for reference and review.  The first part of the course studies probability calculus for univariate and multivariate random variables. This includes topics such as moment generating functions, as well as sampling and convergence. The second part explores statistical learning methods, focusing on information entropy, likelihood-based inference and testing, and Bayesian methods. The first part of this unit continues the development of Probability theory from Year 1 and provides an important basis for many later courses in Probability and Finance.  The second part of this course unit equips students with the methodological foundations of model-based statistical learning, focusing on information entropy, likelihood estimation and inference, as well as Bayesian learning. During the weekly tutorials, some problems are solved using software such as R.This course unit forms the core of Theme C and is delivered over two semesters. It includes three contact hours each week, consisting of two or three hours of lectures and one tutorial class every two weeks. Teaching materials will be available on Canvas for reference and review.  On the successful completion of the course, students will be able to:<ul> <li>calculate marginal distributions of and conditional distributions associated with multivariate random variables </li> <li>describe a range of parametric families to model their probability distribution </li> <li>calculate expectations and conditional expectations </li> <li>evaluate the distribution of functions of random variables </li> <li>use the properties of generating functions to derive moments of distributions, distributions of sums of random variables and limit distributions for sequences of random variables.</li> <li>apply model-based approaches in statistical data analysis; </li> <li>derive maximum likelihood estimates and compute corresponding confidence intervals; </li> <li>perform statistical testing from a likelihood perspective; </li> <li>solve standard modelling and inference problems from a Bayesian point of view; </li> <li>use R to apply techniques from the course on actual data.  </li></ul> Syllabus:  Semester 1:  Chapter 1: Random Variables (4 lectures) Definition of Events and their probabilities; definition of Random variables and their distributions; features of Discrete and Continuous random variables; Functions of random variables and mixed random variables. Chapter 2: Multivariate random variables (6 lectures) Bivariate Distributions; Independence; Sums of several variables; Conditional distributions; the bivariate transform. Chapter 3: Expectation (6 lectures) Expectation of a univariate random variable; Variance and higher Moments; Expectation of a bivariate random variable and conditional expectation; Probability generating functions; Moment generating functions; Sums of random variables using generating functions. Chapter 4: Sampling and convergence (6 lectures); The sample mean; Central limit theorem; Chebyshev's Inequality; Poisson Limit Theorem and characteristic functions; Introduction to the multivariate normal distribution. Semester 2: Section I - Likelihood - Entropy foundations: Shannon and differential entropy, cross-entropy, Kullback-Leibler (KL) divergence, expected Fisher information, minimum KL divergence and maximum likelihood. - Likelihood-based estimation: Likelihood function, regular models, score function, maximum likelihood estimators (MLE), invariance principle, relationship to ordinary least-squares estimation (OLS), observed Fisher information. - Quadratic approximation and normal asymptotics: Quadratic approximation of log-likelihood function and normal distribution, quantifying the uncertainty of MLEs using Fisher information, (squared) Wald statistic, normal confidence intervals, non-regular models. -  Likelihood-based inference: Likelihood-based confidence interval, Wilks log-likelihood ratio statistic, likelihood ratio test, generalised likelihood ratio test, optimality properties. Section II - Bayes - Conditioning and Bayes rule:  Conditional probability, Bayes’ theorem, conditional mean and variance, conditional entropy and chain rules, complete data log-likelihood, observed data log-likelihood, learning unobservable states using Bayes theorem. - Principles of Bayesian learning:  Prior and posterior probabilities and densities over parameters, marginal likelihood, sequential updates, summaries of posterior distributions and credible intervals, Bayesian and frequentist interpretation of probability. - Standard models: Beta-binomial model (for a proportion), normal-normal model (for the mean), inverse-gamma-normal model (for the variance), properties of Bayesian learning. - Bayesian model comparison: Log-marginal likelihood as penalised likelihood, model complexity, Bayes factor, Schwarz approximation and Bayesian Information Criterion (BIC), Bayesian testing using false discovery rate. - Choosing priors and optimality properties: default priors, uninformative priors, empirical Bayes, shrinkage estimation, James-Stein estimator, Frequentist properties of Bayesian estimators, optimality of Bayes inference (e.g. Cox theorem).  This course unit forms the core of Theme C, and as such is delivered over 2 semesters.  Teaching is composed of two hours of lectures per week, and one tutorial class per fortnight.  Teaching materials will be uploaded to Blackboard for reference and review. 0 Other 10% 1 Written exam 90% Other: one mid-term online timed test and one mid-term written test, 40-50 mins each. Marked scripts within a week of the in-class test and automatically given feedback following the online test. Generic feedback made available after marks are released. MATH11711 Probability I Pre-Requisite Compulsory MATH11712 Statistics I Pre-Requisite Compulsory BSc Mathematics with Finance BSc Mathematics with Finance Second Year Mandatory BSc Mathematics with Finance BSc Mathematics with Finance P Second Year Mandatory BSc Maths with Financial Maths BSc (Hons) Maths w Fin Maths Second Year Mandatory BSc Maths with Financial Maths BSc (Hons) Maths w Fin Maths P Second Year Mandatory BSc(Hons) Mathematics BSc (Hons) Mathematics Second Year Optional BSc(Hons) Mathematics BSc (Hons) Mathematics w PY Second Year Optional MMath Maths with Financl Maths MMath (Hons) Math w Fin Math Second Year Mandatory MMath (Hons) Mathematics MMath (Hons) Mathematics Second Year Optional MMath (Hons) Mathematics MMath (Hons) Mathematics w PY Second Year Optional BSc(Hons)Mathematics and Stat BSc(Hons)Mathematics and Stat Second Year Mandatory BSc(Hons)Mathematics and Stat BSc(Hons)Mathematics and Stat Second Year Mandatory MMath(Hons) Maths & Stats MMath(Hons) Maths & Stats Second Year Mandatory BSc Actuarial Science & Maths BSc Actuarial Science & Maths Second Year Mandatory BSc Actuarial Science & Maths BSc Actuarial Sci & Maths wPY Second Year Mandatory N Robert E. Gaunt. MATH27720 Probability 2 Lecture Notes.  Article URL: https://eu-st01.ext.exlibrisgroup.com/…MATH27720 Statistics 2 Lecture Notes. Korbinian Strimmer. Website URL: https://strimmerlab.github.io/…Probability and Distribution Refresher. Korbinian Strimmer. Website URL: https://strimmerlab.github.io/…Matrix and Calculus Refresher. Korbinian Strimmer. Website URL: https://strimmerlab.github.io/…Sahu, Sujit K. (2024). Introduction to probability, statistics and R: foundations for data-based sciences. Springer. ISBN: 9783031378652Held, Leonhard, author. (2020). Springer,. ISBN: 978Heard, Nicholas. (2021). An introduction to Bayesian inference, methods and computation. Springer. ISBN: 9783030828080Gelman, Andrew. (2014). Bayesian data analysis. CRC Press. ISBN: 9781439840955Jaynes, E. T. (2003). Probability theory: the logic of science. Cambridge University Press. ISBN: 9780511790423Diaconis, Persi. (2018). Ten great ideas about chance. Princeton University Press. ISBN: 9781400888283  Lectures 44 Tutorials 12 0 144