Advanced topics in computational linguistics

LELA60152 Advanced topics in computational linguistics 15 Semester 2 Postgraduate Taught Level 7 Dmitry Nikolaev Unit coordinator Colin James Bannard Unit coordinator Linguistics & English Language 1 FHEQ level (Framework for Higher Education Qualifications) ' Masters/Integrated Masters P4 ' European Credit Transfer & Accumulation System Rating : 7.5 The last several years have seen an explosion in the use of language technologies, ranging from consumer-facing chatbot and voice-assistant applications to large-scale models fuelling social media feeds, news recommendations, and marketing. Building on earlier units and on the simultaneous core unit LELA60332, the unit offers hands-on training in contemporary methods for advanced text generation and human–machine interaction. In particular, the course covers recent developments in tokenisation-free language modelling, reinforcement learning from human and synthetic feedback (RLHF/RLVR), and the use of large language models as components in agent-like systems.   The last several years have seen an explosion in the use of language technologies, ranging from consumer-facing chatbot and voice-assistant applications to large-scale models fuelling social media feeds, news recommendations, and marketing. Building on earlier units and on the simultaneous core unit LELA60332, the unit offers hands-on training in contemporary methods for advanced text generation and human–machine interaction. In particular, the course covers recent developments in tokenisation-free language modelling, reinforcement learning from human and synthetic feedback (RLHF/RLVR), and the use of large language models as components in agent-like systems.   Provide understanding of and opportunities to engage with state-of-the-art approaches to practical tasks in computational linguistics and broader AI.Enable students to build and evaluate machine learning models for analysing linguistic structures on different levels and perform structured prediction.Give students basic knowledge of advanced data analysis techniques applicable to linguistic data.  Students will develop the technical, intellectual, practical and personal skills required to successfully develop and deploy state of the art natural language processing systems in academic and commercial environments. They will learn to demonstrate these new abilities in a way that will aid them in applying for further study or future employment.  Demonstrate understanding of computer comprehension and generation of structures representation of linguistic data. Demonstrate critical understanding of the theoretical and mathematical foundations of state-of-the-art language model training. Critically discuss methods for including text understanding and generation modules into wider test and data-processing modules.Engage in the debates surrounding the ethics and social implications, as well as the effectiveness in different application scenarios, of the text-processing capabilities of modern AI systems. Critically engage with research literatures describing representations and processing modes for linguistic data. Identify open research questions in computational linguistics that text processing capabilities of modern AI systems might contribute to, and assess the potential effectiveness of the learned techniques in tackling them. Write computer programs for text processing. Use machine learning methods for understanding and generation. Design natural language engineering systems for text understanding and generation.Conduct parsing and generation experiments. Write a research report describing a text understanding and generation experiment. Document a text understanding or text generation experiment using a GitHub repository. Reflect upon the social impact of the technologies developed. 1 hour asynchronous lecture each week. These will introduce the theoretical and technical content to the topics covered in the seminars. Asynchronous delivery will allow students to cover the technical content at their own pace.2 hour synchronous seminar in computer lab each week. The focus will be on individual and small group computer-based activities implementing the methods described in the lecture, using Jupyter notebooks.  0 Other 20% 3 Report 80% Other: Group programming project: Code archive. N Jurafsky, D. and J. H. Martin (2023). Speech and language processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. 3rd Edition.Individual research papers from ACL Anthology (https://aclanthology.org/) to accompany weekly topics as appropriate. Lectures 11 Seminars 20 119