Data Engineering

IIDS69011 Data Engineering 15 Semester 1 Postgraduate Taught Level 7 Alan Davies Unit coordinator Iliada Eleftheriou Unit coordinator 1 FHEQ level (Framework for Higher Education Qualifications) ' Masters/Integrated Masters P4 ' European Credit Transfer & Accumulation System Rating : 7.5 Clinical Data Scientists need to be able to create data pipelines and merge data sets from different sources before it can be used for onwards analysis such as machine learning. They will also be required to 'wrangle' (pre-process) data into different formats and sub-sets for subsequent analysis. This includes an understanding of structured and unstructured data formats (e.g. tabular form, JSON, XML etc.), how data is modelled in various commonly used databases systems as well as an awareness of data/cyber security. They will be required to access data in a variety of formats and engineer pipelines for data analysis whilst adhering to wider concepts of data protection/privacy regulations and information governance. This module introduces these concepts with applied examples. Clinical Data Scientists need to be able to create data pipelines and merge data sets from different sources before it can be used for onwards analysis such as machine learning. They will also be required to 'wrangle' (pre-process) data into different formats and sub-sets for subsequent analysis. This includes an understanding of structured and unstructured data formats (e.g. tabular form, JSON, XML etc.), how data is modelled in various commonly used databases systems as well as an awareness of data/cyber security. They will be required to access data in a variety of formats and engineer pipelines for data analysis whilst adhering to wider concepts of data protection/privacy regulations and information governance. This module introduces these concepts with applied examples. The unit aims to: <ul> <li style="margin-bottom:11px">Give students hands on experience applying tools and techniques used to access data in different common formats, how to transform and combine this data into a format suitable for subsequent data analysis (e.g. application of statistical methods/machine learning algorithms) by creating data processing pipelines</li> <li style="margin-bottom:11px">Experience using, accessing and querying data in different database storage systems (e.g. relational and NoSQL database systems)</li> <li style="margin-bottom:11px">Understand the importance of data security issues both from a technical and legislative perspective</li> <li style="margin-bottom:11px">Explore the benefits and challenges with accessing health/clinical data </li> <li style="margin-bottom:11px">Understand and practice data cleaning and understand the impact of data provenance and altering data (e.g. variable encoding, missing values, inconstantly entered data and data validation)</li></ul> Learning outcomesOn completion of this unit, succesful students should be able too:<table border="1" cellpadding="1" cellspacing="1" style="width:500px"> <tbody> <tr> <td>Category of outcome</td> <td>Students should be able to:</td> </tr> <tr> <td>A:  Knowledge and understanding</td> <td>LO1: Describe the difference between structured and un-structured data citing relevant examples of eachLO2: Discuss the consequences of cyber-attacks/data breaches and mitigation strategiesLO3: Discuss principles involved in data sharing and information governance with reference to appropriate guidelines and legislationLO4: Critique common data standards depending on intended usageLO5: Explain the challenges and opportunities of big data and approaches for processing such data</td> </tr> <tr> <td>B: Intellectual Skills</td> <td>This unit will cover the following indicative content: <ul> <li>Fundamental data types and structures</li> <li>Structured and unstructured data</li> <li>The fundamentals of using Python for data science and associated libraries/modules </li> <li>How data is modelled in different database systems</li> <li>Querying and filtering data</li> <li>Representing data using dataframes</li> <li>Data cleaning (imputing missing values, encoding variables,</li> <li>Data transformations (wide/long, feature engineering)</li> <li>Combining datasets (data linkage)</li> <li>Data sharing agreements/plans</li> <li>Data and patients</li> <li>Data representation in diagrams (e.g. ERM, Data flow and UML)</li> <li>Common data standards</li> <li>The unit will be delivered online making use of workshops, lectures, labs and self-directed learning material delivered through interactive digital (Jupyter) notebooks to impart core knowledge and skills. A series of synchronous labs using a variety of datasets and formats will be used to foster group work and collaborative working with problem based learning. Case-studies and data will be drawn from The University of Manchester and its affiliates as well as NHS and open-source projects where possible.</li> </ul> </td> </tr> </tbody></table>    This unit will cover the following indicative content:<ul> <li style="margin-bottom:11px">Fundamental data types and structures </li> <li style="margin-bottom:11px">Structured and unstructured data</li> <li style="margin-bottom:11px">The fundamentals of using Python for data science and associated libraries/modules  </li> <li style="margin-bottom:11px">How data is modelled in different database systems</li> <li style="margin-bottom:11px">Querying and filtering data</li> <li style="margin-bottom:11px">Representing data using dataframes </li> <li style="margin-bottom:11px">Data cleaning (imputing missing values, encoding variables,</li> <li style="margin-bottom:11px">Data transformations (wide/long, feature engineering) </li> <li style="margin-bottom:11px">Combining datasets (data linkage) </li> <li style="margin-bottom:11px">Data sharing agreements/plans</li> <li style="margin-bottom:11px">Data and patients </li> <li style="margin-bottom:11px">Data representation in diagrams (e.g. ERM, Data flow and UML)</li> <li style="margin-bottom:11px">Common data standards </li> <li style="margin-bottom:11px">The unit will be delivered online making use of workshops, lectures, labs and self-directed learning material delivered through interactive digital (Jupyter) notebooks to impart core knowledge and skills. A series of synchronous labs using a variety of datasets and formats will be used to foster group work and collaborative working with problem based learning. Case-studies and data will be drawn from The University of Manchester and its affiliates as well as NHS and open-source projects where possible 0 Other 100% <table class="MsoTableGrid" style="border-collapse:collapse; border:none"> <tbody> <tr> <td style="border-bottom:1px solid black; width:160px; padding:0cm 7px 0cm 7px; border-top:1px solid black; border-right:1px solid black; border-left:1px solid black" valign="top">Assessment task</td> <td style="border-bottom:1px solid black; width:159px; padding:0cm 7px 0cm 7px; border-top:1px solid black; border-right:1px solid black; border-left:none" valign="top">Length</td> <td style="border-bottom:1px solid black; width:152px; padding:0cm 7px 0cm 7px; border-top:1px solid black; border-right:1px solid black; border-left:none" valign="top">Weighting within unit</td> </tr> <tr> <td style="border-bottom:1px solid black; width:160px; padding:0cm 7px 0cm 7px; border-top:none; border-right:1px solid black; border-left:1px solid black" valign="top">Data Management Plan</td> <td style="border-bottom:1px solid black; width:159px; padding:0cm 7px 0cm 7px; border-top:none; border-right:1px solid black; border-left:none" valign="top">You will create an authentic data management plan for a fictional scenario or real-world project that you would like to implement in your organisation.</td> <td style="border-bottom:1px solid black; width:152px; padding:0cm 7px 0cm 7px; border-top:none; border-right:1px solid black; border-left:none" valign="top">100%</td> </tr> </tbody></table> Formative assessment and feedback to students is a key feature of the online learning materials for this unit and is provided through self-directed learning activities in the interactive notebooks.    N <ul> <li>McKinney, W (2017) Python for Data Analysis. Beijing: O'Reilly</li> <li>Molin, S (2019) Hands-On Data Analysis with Pandas. Birmingham: Packt</li> <li>Medium (2021) Towards data science: A Medium publication sharing concepts, ideas and codes. <a href="https://towardsdatascience.com/about">https://towardsdatascience.com/abou</a>t</li></ul> 0 0 150