Jordan Boyd-Graber

Jordan Boyd-Graber
4146 Iribe
Computer Science
jbg@boydgraber.org

Natural Language Processing (CMSC 470)

Logistics

Location	CSIC 1121
Time	Tue./Thu. 11:00am - 12:15pm
Webpage	http://umiacs.umd.edu/~jbg/teaching/CMSC_470/
Homework Submission	https://www.gradescope.com/courses/303336
Mailing List	http://piazza.com/umd/fall2021/cmsc470
Text	Speech and Natural Language Processing
Syllabus	https://docs.google.com/document/d/1UN3qlJttJFRXP4qtoIriZxHjYQflvsUNsulHLIFVTzQ/edit?usp=sharing

People

Professor

Jordan Boyd-Graber
IRB 4146
Office Hours (IRB 4146): By appointment

Teaching Assistant

Neha Pundlik Srikanth (nehasrik@umd.edu)

Schedule

Date	In-Class Topic	Assignment Due
Tue 31. Aug	Introducing the Course and Linguistic Concepts
	Lecture: Welcome to NLP [Slides] How to take a class with me [Slides] Class in the Time of Corona What's a word? [Slides for both this and next video] What's a language? Optional Readings: Python (and programming) introductory course You may also want to look at first couple of weeks (through February) of INST 414 as review. A probability primer (Optional, if you want more detail) Grinstead and Snell, Chapters 1-2, Chapter 4.1 How the statistical revolution changes (computational) linguistics On the deep learning revolution in NLP
Thu 2. Sep	Information Retrieval / tf-idf
	Lecture: What is information retrieval? [Slides] TF-IDF [Slides] Evaluation [Slides] Exercise Readings: IR Chapter 1, 6
Tue 7. Sep	Logistic Regression Intro [רֹאשׁ הַשָּׁנָה]
	Lecture: Regression [Slides] Logistic Regression [Slides] Exercise Readings: SNLP Chapter 5.1-5.3
	Lecture: Feature Engineering in the Age of Deep Learning Feature Engineering for Quiz Bowl [Slides (for this and below)] Feature Engineering for Spoiler Detection No exercise: we'll talk about homework Readings: Feature Engineering for Machine Learning Chapters 2, 5
Fri 10. Sep	Homework Due	tf-idf
Thu 14. Sep	Stochastic Gradient Descent [יוֹם כִּיפּוּר]
	Lecture: Stochastic Gradient Descent [Slides] Exercise Readings: SNLP Chapter 5.4-5.9 Is there only really one lecture? Yes, because this is a difficult but super important concept. Make sure you understand it!
Tue 14. Sep	Project Framework
	Lectures: Overview of Homeworks Submitting on Codalab Readings: QANTA.org
Fri 17. Sep	Homework Due	tf-idf Guesser
Tue 21. Sep	Distributional Semantics (Word2Vec)
	Lecture: Distributional Semantics [Slides] Word2Vec [Slides] Evaluation [Slides] Exercise Readings: SNLP Chapter 6
Thu 23. Sep	Neural Networks for Language
	Lecture: Deep [Slides] Backprop [Slides] Exercise Readings: SNLP Chapter 7
Fri 24. Sep	Homework Due	Stochastic Gradient Descent
Tue 28. Sep	Deep Learning Frameworks
	Lecture: Frameworks [Slides] Pytorch [Slides] DAN [Sildes]
Thu 30. Sep	Expo Match I (Students)
Fri 1. Oct	Homework Due	Pytorch Logistic Regression
Tue 5. Oct	History of Question Answering
	How question answering saved civilization [Slides, including for next lecture] The Turing Test Watson on Jeopardy! [Slides]
Thu 7. Oct	Neural Sequence Models
	Lecture: Motivation: Part of Speech Tagging [Slides] RNN [Slides] LSTM [Slides] Exercise Readings: SNLP 9-9.3.3 This material will not be on the midterm.
Tue 12. Oct	Midterm Review
Thu 14. Oct	Midterm
Tue 19. Oct	Modern QA
	Lecture: QA Datasets [Slides] Cranfield vs. Manchester [Slides] Dr. QA [Slides] Exercise Readings: SNLP 23
The 21. Oct	Transformer Language Models
	Lecture: Contextual Language Models [Slides] Transformer Detail [Slides (this and below)] BERTology Reading: SNLP 9.4-9.6 Exercise: Visualizations of attentions
Fri 22. Oct	Homework Due	Buzzer Feature Engineering
Tue 26. Oct	Expo Match II: Course Staff
Thu 28. Oct	Entity-centric NLP
	Lecture: Entity Linking [Slides] Coreference [Slides] Exercise Lecture: SLP 21
Fri 29. Oct	Homework Due	Project Proposal Due
Tue 2. Nov	Transformers for QA
	Lecture: BERT for Machine Reading Dense Passage Retrieval Reading: A BERT Baseline for Natural Questions [Slides] Dense Passage Retrieval
Thu 4. Nov	Constituency Syntax
	Lecture: PCFG [Slides] Exercise Readings: SLP Chapter 12 Super old video, I know, but there's nothing really wrong with it. I can't bring myself to redo it.
Fri 5. Nov	Homework Due	Deep Averaging Networks
Tue 9. Nov	Dependency Syntax
	Lecture: Intro [Slides Shift-reduce [Slides] Exercise Readings: SLP Chapter 13
Thu 11. Nov	Machine Translation
	Lecture: Word-Based [Slides] Phrase [Slides] Neural [Neural] Exercise Readings: NLP Notes Chapter 18
Fri 12. Nov	HW Due	Run a QA Model
Tue 16. Nov	Topic Models
	Lecture: Topic Models in Neural Times Intro [Slides] Gibbs [Slides] Exercise Readings: Chapter 1 of "Applications of Topic Models"
Thu 18. Nov	Expo Match III: MAQT
Fri 19. Nov	Homework Due	Project Deliverable
Tue 23. Nov	Proposal Workshop (join via Zoom!)
Thu 25. Nov	Thanksgiving
Tue 30. Nov	Advanced QA: Fact Checking, Long-form QA, Bias
	Readings: The Seven Deadly Sins of AI Predictions How Computers Misunderstand the World
Thu 2. Dec	AMA
	Possible Topics: Writing Papers Careers in NLP Virtual Conferences
Tue 7. Dec	Midterm Review
Thu 9. Dec	Midterm II

Page by Jordan Boyd-Graber.