Jordan Boyd-Graber

Jordan Boyd-Graber
4146 Iribe
Computer Science
jbg@boydgraber.org

Natural Language Processing (CMSC 470)

Logistics

Location	CSIC 2117
Q & A	Mon. 15:30 - 16:45pm
Workshop	Wed. 15:30 - 16:45pm
Webpage	http://umiacs.umd.edu/~jbg/teaching/CMSC_470/
Homework Submission	https://www.gradescope.com/courses/670788
Mailing List	http://piazza.com/umd/spring2024/cmsc470
Text	Speech and Natural Language Processing
Syllabus	https://docs.google.com/document/d/1lrCYFd20FYuwSi7Zrhj-Zqk4QotFFnJT92sxdN67z8M/edit?usp=sharing

People

Professor

Jordan Boyd-Graber
IRB 4146
Office Hours (IRB 4146 / Online): By appointment

Teaching Assistants

1. Yu Hou, Office Hours: Friday 11:00 AM -12:00 PM (Zoom, check Piazza for link)
2. Shramay Palta, Office Hours: Thursday, 12:00 PM - 1:00 PM (4th Floor Iribe Kitchen Area)

Schedule

Date	In-Class Topic	Assignment Due
24. Jan	Introducing the Course and Linguistic Concepts
	Lecture: Welcome to NLP [Slides] What's a word? [Slides for both this and next video] What's a language? Administrivia: How to read this webpage How to take a class with me [Slides] More on the Flipped Classroom Hybrid classes Grader Expectations Readings: Linguistic Fundamentals for Natural Language Processing: 100 Essentials from Morphology and Syntax (Chapters 1 and 2) [Alternate Link] Optional Readings: Python (and programming) introductory course You may also want to look at first couple of weeks (through February) of INST 414 as review. A probability primer (Optional, if you want more detail) Grinstead and Snell, Chapters 1-2, Chapter 4.1 How the statistical revolution changes (computational) linguistics On the deep learning revolution in NLP
29. Jan	Where AI/NLP is today
	Lecture: What's the difference between NLP, AI, and ChatGPT? What made generative AI possible What to call these models? Shannon Game [Slides] Evaluating Language Models [Slides] The Turing Test Optional Review: What are expectations and entroy? Administrivia: Why you shouldn't use generative AI to cheat in my class Readings: Language Models: A Guide for the Perplexed
Wed 31. Jan	Workshop: Python/Homework Environment
Fri 2. Feb	Homework Due	Warmup: Who was President
5. Feb	Information Retrieval / tf-idf
	Lecture: The Cranfield Paradigm [Slides] What is information retrieval? [Slides] TF-IDF [Slides] Evaluation [Slides] Readings: IR Chapter 1, 6 Evolution of Cranfield (Optional)
Wed 7. Feb	Workshop: tf-idf
Fri 9. Feb	Homework Due	tf-idf
Mon 12. Feb	Logistic Regression
	Lecture: Regression [Slides] Logistic Regression [Slides] Stochastic Gradient Descent [Slides] Feature Engineering in the Age of Deep Learning Feature Engineering for Quiz Bowl Readings: SNLP Chapter 5
Wed 14. Feb	Workshop: SGD
Fri 16. Feb	Homework Due	Logistic Regression
Mon 19. Feb	Project/Homework Framework
	Lectures: Advice / Overview (It's all in one video, but there are smaller segments inside, don't be afraid you have to watch everything in a 30 minute chunk) How to stump a computer Good questions Bad questions Optional Videos: IRT Optional Videos of our Past 470 Finals: 2019 2022
Wed 18. Feb	Workshop: Feature Engineering
Mon 26. Feb	Distributional Semantics
	Lecture: Distributional Semantics [Slides] Word2Vec [Slides] Evaluation [Slides] Readings: SNLP Chapter 6
Wed 28. Feb	Workshop: Wordvectors (Colab
Fri 1. Mar	Homework Due	Feature Engineering
Mon 4. Mar	Syntax
	Lecture: What's a part of speech, and why is it a joint problem? [Slides] Hidden Markov Modeling [Slides] Inferring a Hidden Markov Model from Labeled Data [Slides] Dependency parsing [Slides] Readings: SNLP Chapter 8 (more important), 18 (less important) Optional Lectures (won't be on the exam or in homeworks): Viterbi Algorithm [Slides] Shift-Reduce Parser [Slides]
Wed 6. Mar	Workshop: What Makes a Good Question?
Fri 8. Mar	Homework Due	tf-idf Guesser
Mon 11. Mar	Neural Networks for Language
	Lecture: Deep [Slides] Backprop [Slides] Frameworks [Slides] Pytorch [Slides] DAN [Sildes] Readings: SNLP Chapter 7
Wed 13. Mar	Workshop: Pytorch (A HREF="https://docs.google.com/presentation/d/1G5ra5dmkpi9l2dAv4YXoSlUH-JsyGpg6iYDASr0ly58">Colab)
Fri 15. Mar	Homework Due	Adversarial QA
17-24. March	Spring Break
Mon 25. Mar	Sequence Models
	Lecture: RNN for Sentiment [Slides] RNN for LM [Slides] LSTM [Slides] Sequence models in Pytorch Reading: SNLP 9
Wed 27. Mar	Workshop: BiLSTM Classification
Fri 29. Mar	Project Due	Proposal
Mon 1. April	Midterm Review
Wed 3. April	Midterm
Mon 8. Apr	Midterm Makeup / Eclipse (Only for Documented Absences)
Wed 10. April	Transformers
	Lecture: Contextual Neural Language Models Overview [Slides] Transformer Encoder [Slides] BERTology [Slides, and for below] Decoder Implementation Details Readings: SNLP 7 The Annotated Transformer SNLP 9-9.3.3
Mon 15. April	Alignment and Fine-Tuning
	Lecture: Fine-tuning, prompting, and how they relate to NLP [Slides] RL Review [Slides] Alignment [Slides] Policy Gradient [Slides] RLHF Readings: Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback Optional: UG Research at UMD on Prompting
Wed 17. April	Workshop: Fine-Tuning
Fri 19. Apr	Homework Due	Neural Models for Text
Mon 22. April	QA Datasets
	Lecture: How question answering saved civilization [Slides, including for next lecture] The Turing Test Watson on Jeopardy! [Slides] QA Datasets [Slides] Cranfield vs. Manchester [Slides] Readings: SNLP 23
Wed 24. Apr		Lecture: Dense Retrieval [Slides] BERT for MRQA Multihop Generation for QA [Slides] Conversational QA Long QA Reading: MRC Literature Review
Wed 1. May	Workshop: QA Systems
Mon 6. May	Project	Milestone 2
Mon 6. May	Machine Translation
	Lecture: Translation is Hard Word-based Translation Phrase-based Translation Neural Machine Translation Decoding Evaluation Reading: NMT Book (Chapter 5)
Wed 8. May	Odds and Ends / AMA
	Lecture: Course Evals You got a PhD in Computational Linguistics ... what now? Optimizing ML for Society How far off were my predictions? and p(Doom)
Fri 10. May	Question JSON Upload
Sun 12. May	Leaderboard System Submission
Fri 10. May	Optional Extra Credit	Fact Retention Study Fine-tuning
Tue 14. May	1:30 p.m. - 3:30 p.m.: Final Showdown: Systems vs. Questions

Page by Jordan Boyd-Graber.