CMSC 470

Logistics

Location	CSIC 2117
Q & A	Mon. 15:30 - 16:45pm
Workshop	Wed. 15:30 - 16:45pm
Webpage	http://umiacs.umd.edu/~jbg/teaching/CMSC_470/
Homework Submission	https://www.gradescope.com/courses/670788
Mailing List	http://piazza.com/umd/spring2024/cmsc470
Text	Speech and Natural Language Processing
Syllabus	https://docs.google.com/document/d/1lrCYFd20FYuwSi7Zrhj-Zqk4QotFFnJT92sxdN67z8M/edit?usp=sharing

People

Professor

Jordan Boyd-Graber
IRB 4146
Office Hours (IRB 4146 / Online): By appointment

Teaching Assistants

Yu Hou, Office Hours: Friday 11:00 AM -12:00 PM (Zoom, check Piazza for link)
Shramay Palta, Office Hours: Thursday, 12:00 PM - 1:00 PM (4th Floor Iribe Kitchen Area)

Schedule

Date	In-Class Topic	Assignment Due
24. Jan	Introducing the Course and Linguistic Concepts
	Lecture: Welcome to NLP [Slides] What's a word? [Slides for both this and next video] What's a language? Administrivia: How to read this webpage How to take a class with me [Slides] More on the Flipped Classroom Hybrid classes Grader Expectations Readings: Linguistic Fundamentals for Natural Language Processing: 100 Essentials from Morphology and Syntax (Chapters 1 and 2) [Alternate Link] Optional Readings: Python (and programming) introductory course You may also want to look at first couple of weeks (through February) of INST 414 as review. A probability primer (Optional, if you want more detail) Grinstead and Snell, Chapters 1-2, Chapter 4.1 How the statistical revolution changes (computational) linguistics On the deep learning revolution in NLP
29. Jan	Where AI/NLP is today
	Lecture: What's the difference between NLP, AI, and ChatGPT? What made generative AI possible What to call these models? Shannon Game [Slides] Evaluating Language Models [Slides] The Turing Test Optional Review: What are expectations and entroy? Administrivia: Why you shouldn't use generative AI to cheat in my class Readings: Language Models: A Guide for the Perplexed
Wed 31. Jan	Workshop: Python/Homework Environment
Fri 2. Feb	Homework Due	Warmup: Who was President
5. Feb	Information Retrieval / tf-idf
	Lecture: The Cranfield Paradigm [Slides] What is information retrieval? [Slides] TF-IDF [Slides] Evaluation [Slides] Readings: IR Chapter 1, 6 Evolution of Cranfield (Optional)
Wed 7. Feb	Workshop: tf-idf
Fri 9. Feb	Homework Due	tf-idf
Mon 12. Feb	Logistic Regression
	Lecture: Regression [Slides] Logistic Regression [Slides] Stochastic Gradient Descent [Slides] Feature Engineering in the Age of Deep Learning Feature Engineering for Quiz Bowl Readings: SNLP Chapter 5
Wed 14. Feb	Workshop: SGD
Fri 16. Feb	Homework Due	Logistic Regression
Mon 19. Feb	Project/Homework Framework
	Lectures: Advice / Overview (It's all in one video, but there are smaller segments inside, don't be afraid you have to watch everything in a 30 minute chunk) How to stump a computer Good questions Bad questions Optional Videos: IRT Optional Videos of our Past 470 Finals: 2019 2022
Wed 18. Feb	Workshop: Feature Engineering
Mon 26. Feb	Distributional Semantics
	Lecture: Distributional Semantics [Slides] Word2Vec [Slides] Evaluation [Slides] Readings: SNLP Chapter 6
Wed 28. Feb	Workshop: Wordvectors
Fri 1. Mar	Homework Due	Feature Engineering
Mon 4. Mar	Syntax
	Lecture: What's a part of speech, and why is it a joint problem? [Slides] Hidden Markov Modeling [Slides] Inferring a Hidden Markov Model from Labeled Data [Slides] Dependency parsing [Slides] Readings: SNLP Chapter 8 (more important), 18 (less important) Optional Lectures (won't be on the exam or in homeworks): Viterbi Algorithm [Slides] Shift-Reduce Parser [Slides]
Wed 6. Mar	Workshop: What Makes a Good Question?
Fri 8. Mar	Homework Due	tf-idf Guesser
Mon 11. Mar	Neural Networks for Language
	Lecture: Deep [Slides] Backprop [Slides] Frameworks [Slides] Pytorch [Slides] DAN [Sildes] Readings: SNLP Chapter 7
Wed 13. Mar	Workshop: Pytorch
Fri 15. Mar	Homework Due	Adversarial QA
17-24. March	Spring Break
Mon 25. Mar	Sequence Models
	Lecture: RNN for Sentiment [Slides] RNN for LM [Slides] LSTM [Slides] Sequence models in Pytorch Reading: SNLP 9
Wed 27. Mar	Workshop: BiLSTM Classification
Fri 29. Mar	Project Due	Proposal
Mon 1. April	Midterm Review
Wed 3. April	Midterm
Mon 8. Apr	Midterm Makeup / Eclipse (Only for Documented Absences)
Wed 10. April	Transformers
	Lecture: Contextual Neural Language Models Overview [Slides] Transformer Encoder [Slides] BERTology [Slides, and for below] Decoder Implementation Details Readings: SNLP 7 The Annotated Transformer SNLP 9-9.3.3
Mon 15. April	Alignment and Fine-Tuning
	Lecture: Fine-tuning, prompting, and how they relate to NLP [Slides] RL Review [Slides] Alignment [Slides] Policy Gradient [Slides] RLHF Readings: Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback Optional: UG Research at UMD on Prompting
Wed 17. April	Workshop: Fine-Tuning
Fri 19. Apr	Homework Due	Neural Models for Text
Mon 22. April	QA Datasets
	Lecture: How question answering saved civilization [Slides, including for next lecture] The Turing Test Watson on Jeopardy! [Slides] QA Datasets [Slides] Cranfield vs. Manchester [Slides] Readings: SNLP 23
Wed 24. Apr	Workshop: Project Management (Slurm, managing goals, etc.)
Fri 26. Apr	Project Due	Milestone 1
Mon 29. Apr	QA Methods
	Lecture: Dense Retrieval [Slides] BERT for MRQA Multihop Generation for QA [Slides] Conversational QA Long QA Reading: MRC Literature Review
Wed 1. May	Workshop: QA Systems
Fri 3. May	Project	Milestone 2
Mon 6. May	Machine Translation
	Lecture: Translation is Hard Word-based Translation Phrase-based Translation Neural Machine Translation Simultaneous Translation Evaluation Reading: MRC Literature Review
Wed 8. May	Odds and Ends
	Lecture: Course Evals You got a PhD in Computational Linguistics ... what now? Optimizing ML for Society How far off were my predictions? p(Doom)
Fri 10. May	Optional Extra Credit	Fact Retention Study Fine-tuning
Tue 14. May	1:30 p.m. - 3:30 p.m.: Final Showdown: Systems vs. Questions

Natural Language Processing (CMSC 470)

Logistics

People

Professor

Teaching Assistants

Schedule