Jordan Boyd-Graber

Jordan Boyd-Graber
4146 Iribe
Computer Science
jbg@boydgraber.org

Natural Language Processing (CMSC 470)

Logistics

Location	CSIC 1121
Time	Tue./Thu. 11:00am - 12:15pm
Webpage	http://umiacs.umd.edu/~jbg/teaching/CMSC_470/
Homework Submission	https://www.gradescope.com/courses/303336
Mailing List	http://piazza.com/umd/fall2021/cmsc470
Text	Speech and Natural Language Processing
Syllabus	https://docs.google.com/document/d/1UN3qlJttJFRXP4qtoIriZxHjYQflvsUNsulHLIFVTzQ/edit?usp=sharing

People

Professor

Jordan Boyd-Graber
IRB 4146
Office Hours (IRB 4146): By appointment

Teaching Assistant

Neha Pundlik Srikanth (nehasrik@umd.edu)

Schedule

Date	In-Class Topic	Assignment Due
Tue 31. Aug	Introducing the Course and Linguistic Concepts
	Lecture: Welcome to NLP [Slides] How to take a class with me [Slides] Class in the Time of Corona What's a word? [Slides for both this and next video] What's a language? Optional Readings: Python (and programming) introductory course You may also want to look at first couple of weeks (through February) of INST 414 as review. A probability primer (Optional, if you want more detail) Grinstead and Snell, Chapters 1-2, Chapter 4.1 How the statistical revolution changes (computational) linguistics On the deep learning revolution in NLP
Thu 2. Sep	Information Retrieval / tf-idf
	Lecture: What is information retrieval? [Slides] TF-IDF [Slides] Evaluation [Slides] Exercise Readings: IR Chapter 1, 6
Tue 7. Sep	Logistic Regression Intro [רֹאשׁ הַשָּׁנָה]
	Lecture: Regression [Slides] Logistic Regression [Slides] Exercise Readings: SNLP Chapter 5.1-5.3
Thu 9. Sep	Feature Engineering
	Lecture: Feature Engineering in the Age of Deep Learning Feature Engineering for Quiz Bowl [Slides (for this and below)] Feature Engineering for Spoiler Detection No exercise: we'll talk about homework Readings: Feature Engineering for Machine Learning Chapters 2, 5
Fri 10. Sep	Homework Due	tf-idf
Thu 14. Sep	Stochastic Gradient Descent [יוֹם כִּיפּוּר]
	Lecture: Stochastic Gradient Descent [Slides] Exercise Readings: SNLP Chapter 5.4-5.9 Is there only really one lecture? Yes, because this is a difficult but super important concept. Make sure you understand it!
Tue 14. Sep	Project Framework
	Lectures: Overview of Homeworks Submitting on Codalab Readings: QANTA.org
Fri 17. Sep	Homework Due	tf-idf Guesser
Tue 21. Sep	Distributional Semantics (Word2Vec)
	Lecture: Distributional Semantics [Slides] Word2Vec [Slides] Evaluation [Slides] Exercise Readings: SNLP Chapter 6
Thu 23. Sep	Neural Networks for Language
	Lecture: Deep [Slides] Backprop [Slides] Exercise Readings: SNLP Chapter 7
Fri 24. Sep	Homework Due	Stochastic Gradient Descent
Tue 28. Sep	Deep Learning Frameworks
	Lecture: Frameworks [Slides] Pytorch [Slides] DAN [Sildes]
Thu 30. Sep	Expo Match I (Students)
Fri 1. Oct	Homework Due	Pytorch Logistic Regression
Tue 5. Oct	History of Question Answering
	How question answering saved civilization What question answering researchers can learn from trivia Watson on Jeopardy!
Thu 7. Oct	Neural Sequence Models
	Lecture: Motivation: Part of Speech Tagging Motivation: Language Modeling RNN [Slides<\a>] LSTM [Slides] Exercise Readings: SNLP 9-9.3.3 This material will not be on the midterm.
Fri 8. Oct	Homework Due	Buzzer Feature Engineering
Tue 12. Oct	Midterm Review
Thu 14. Oct	Midterm
Tue 19. Oct	Expo Match II: Course Staff
Thu 21. Oct	Modern QA
	Lecture: QA Datasets [Slides] Cranfield vs. Manchester [Slides] Dr. QA [Slides] Exercise Readings: SNLP 23
Fri 22. Oct	Homework Due	Deep Retrieval
Tue 26. Oct	Transformer Language Models
	Lecture: Contextual Language Models [Slides] Transformer Detail [Slides (this and below)] BERTology Reading: SNLP 9.4-9.6
Thu 28. Oct	Transformer Encoders for QA
	Lecture: Dense Passage Retrieval BERT for Machine Reading Other Ways of Doing QA: Generation / NN
Fri 29. Oct	Homework Due	Project Proposal Due
Tue 2. Nov	Constiuency Syntax
	Lecture: PCFG [Slides] Exercise Readings: SLP Chapter 12 Super old video, I know, but there's nothing really wrong with it. I can't bring myself to redo it.
Thu 4. Nov	Dependency Syntax
	Lecture: Intro [Slides Shift-reduce [Slides] Exercise Readings: SLP Chapter 13
Tue 9. Nov	Entities
	Lecture: Entity Linking [Slides] Coreference [Slides] Exercise
Thu 11. Nov	Expo Match III: MAQT
Fri 12. Nov	HW Due	Run a QA Model
Tue 16. Nov	Machine Translation
	Lecture: Word-Based [Slides] Phrase [Slides] Neural [Neural] Exercise Readings: NLP Notes Chapter 18
Thu 18. Nov	Topic Models
	Lecture: Topic Models in Neural Times Intro [Slides] Gibbs [Slides] Exercise Readings: Chapter 1 of "Applications of Topic Models"
Fri 19. Nov	Homework Due	Project Deliverable
Tue 23. Nov	Slack Day
Thu 25. Nov	Thanksgiving
Tue 30. Nov	Advanced QA: Fact Checking, Long-form QA, Bias
	Readings: The Seven Deadly Sins of AI Predictions How Computers Misunderstand the World
Thu 2. Dec	AMA
	Possible Topics: Writing Papers Careers in NLP Virtual Conferences
Tue 7. Dec	Midterm Review
Thu 9. Dec	Midterm II

Page by Jordan Boyd-Graber.