Chip Multiprocessors

COMP35112 Chip Multiprocessors 10 Semester 2 Undergraduate Level 3 Pierre Olivier Unit coordinator Department of Computer Science 1 FHEQ level (Framework for Higher Education Qualifications) ' Last part of a Bachelors ' European Credit Transfer & Accumulation System Rating : 5.0 Due to technological limitations, it is proving increasingly difficult to maintain a continual increase in the performance of individual processors. Therefore, the current trend is to integrate multiple processors on to a single chip and exploit the resulting parallel resources to achieve higher computing power. However, this may require significantly different approaches to both hardware and software particularly for general purpose applications. This course will explore these issues in detail. The unit covers the following topics:<ul><li>Introduction. Trends in technology, limitations and consequences. The move to multi-coreParallelism in programs, ILP, Thread Level, Data Parallelism.</li><li>Parallel Architectures. SIMD, MIMD, Shared Memory, Distributed Memory, strengths and weaknesses.</li><li>Parallel Programming. Multithreaded programming, Data parallel programming, Explicit vs Implicit parallelism, automatic parallelisation. The Case for Shared Memory. When to share?</li><li>Shared Memory Multiprocessors. Basic structures, the cache coherence problem. The MESI protocol. Limitations. Directory based coherence.</li><li>Programming with Locks and Barriers. The need for synchronisation. Problems with explicit synchronisation.</li><li>Other Parallel Programming Approaches. MPI and OpenMP.</li><li>Speculation. The easy route to automatic parallelisation?</li><li>Transactional Memory. Principles. Hardware and Software approaches.</li><li>Memory Issues. Memory system design. Memory consistency.</li><li>Other Architectures and Programming Approaches. GPGPUs, CUDA.</li><li>Data Driven Parallelism. Dataflow principles and Functional Programming.</li></ul> The unit aims to explore the various software and hardware issues brought by the current trends of integrating multiple processors on to a single chip in order to tackle the single-core processors speed stagnation through parallelism. ILO 1: Describe the main issues, along with key proposed solutions, in the design and construction of chip multi-processor hardware (e.g. multi-core CPUs and GPUs) and related programming languages.ILO 2: Evaluate a number of specific examples of extensions to programming languages supporting the writing of correct and efficient code on shared memory multi-processors.ILO 3: Compare a number of extensions to hardware structures supporting, for example, memory coherence, synchronization, speculation and transactional memory, to improve the performance of code executing on chip multiprocessors.ILO 4: Compare a number of extensions to programming languages and supporting user- and operating system-level data structures to improve the performance of code on chip multiprocessors.ILO 5: Evaluate the effectiveness of concurrency support in a popular programming language, including the use of threads, locks and barriers, in the context of experience gained with three simple parallel programming exercises. Analytical skills Problem solving Written communication Other The unit covers the following topics:<ul><li>Introduction. Trends in technology, limitations and consequences. The move to multi-coreParallelism in programs, ILP, Thread Level, Data Parallelism.</li><li>Parallel Architectures. SIMD, MIMD, Shared Memory, Distributed Memory, strengths and weaknesses.</li><li>Parallel Programming. Multithreaded programming, Data parallel programming, Explicit vs Implicit parallelism, automatic parallelisation. The Case for Shared Memory. When to share?</li><li>Shared Memory Multiprocessors. Basic structures, the cache coherence problem. The MESI protocol. Limitations. Directory based coherence.</li><li>Programming with Locks and Barriers. The need for synchronisation. Problems with explicit synchronisation.</li><li>Other Parallel Programming Approaches. MPI and OpenMP.</li><li>Speculation. The easy route to automatic parallelisation?</li><li>Transactional Memory. Principles. Hardware and Software approaches.</li><li>Memory Issues. Memory system design. Memory consistency.</li><li>Other Architectures and Programming Approaches. GPGPUs, CUDA.</li><li>Data Driven Parallelism. Dataflow principles and Functional Programming.</li></ul> 1 Written exam 70% 2 Written assignment (inc essay) 30% Written feedback on reports for laboratory exercises. Students who attempt previous exam questions can get feedback on their answers. COMP25212 System Architecture Pre-Requisite Compulsory Students who are not from the School of Computer Science must have permission from both Computer Science and their home School to enrol. Y Computer architecture : a quantitative approach     Hennessy, John L.    Morgan Kaufmann/Elsevier    2012    ISBN: <a href="https://www.librarysearch.manchester.ac.uk/discovery/search?query=isbn,contains,9780123838735&search_scope=MyInst_and_CI&sortby=rank&vid=44MAN_INST:MU_NUI&lang=en&mode=advanced&offset=0" target="_blank">9780123838735</a> The art of multiprocessor programming [electronic resource]     Herlihy, Maurice.    Morgan Kaufmann    2012    ISBN: <a href="https://www.librarysearch.manchester.ac.uk/discovery/search?query=isbn,contains,0123977959&search_scope=MyInst_and_CI&sortby=rank&vid=44MAN_INST:MU_NUI&lang=en&mode=advanced&offset=0" target="_blank">0123977959</a> Parallel computer architecture [electronic resource] : a hardware/software approach     Culler, David.    Elsevier Science    2014    ISBN: <a href="https://www.librarysearch.manchester.ac.uk/discovery/search?query=isbn,contains,9780080573076&search_scope=MyInst_and_CI&sortby=rank&vid=44MAN_INST:MU_NUI&lang=en&mode=advanced&offset=0" target="_blank">9780080573076</a> The art of multiprocessor programming     Herlihy, Maurice,    Morgan Kaufmann is an imprint of Elsevier    2021    ISBN: <a href="https://www.librarysearch.manchester.ac.uk/discovery/search?query=isbn,contains,9780123914064&search_scope=MyInst_and_CI&sortby=rank&vid=44MAN_INST:MU_NUI&lang=en&mode=advanced&offset=0" target="_blank">9780123914064</a> Programming massively parallel processors : a hands-on approach     Kirk, David,    Elsevier/Morgan Kaufmann    2013    ISBN: <a href="https://www.librarysearch.manchester.ac.uk/discovery/search?query=isbn,contains,9780123914187&search_scope=MyInst_and_CI&sortby=rank&vid=44MAN_INST:MU_NUI&lang=en&mode=advanced&offset=0" target="_blank">9780123914187</a> Patterns for Parallel Programming [electronic resource].    Mattson, Timothy.    Addison Wesley    2004    ISBN: <a href="https://www.librarysearch.manchester.ac.uk/discovery/search?query=isbn,contains,9780321630032&search_scope=MyInst_and_CI&sortby=rank&vid=44MAN_INST:MU_NUI&lang=en&mode=advanced&offset=0" target="_blank">9780321630032</a> Transactional memory [electronic resource]     Harris, Tim,    Morgan & Claypool Publishers    2010    ISBN: <a href="https://www.librarysearch.manchester.ac.uk/discovery/search?query=isbn,contains,9781608452361&search_scope=MyInst_and_CI&sortby=rank&vid=44MAN_INST:MU_NUI&lang=en&mode=advanced&offset=0" target="_blank">9781608452361</a> Introduction to functional programming using Haskell     Bird, Richard,    Prentice Hall Europe    1998    ISBN: <a href="https://www.librarysearch.manchester.ac.uk/discovery/search?query=isbn,contains,0134843460&search_scope=MyInst_and_CI&sortby=rank&vid=44MAN_INST:MU_NUI&lang=en&mode=advanced&offset=0" target="_blank">0134843460</a> Lectures 24 0 76