Jordan Boyd-Graber

Jordan Boyd-Graber
4146 Iribe
Computer Science
jbg@boydgraber.org

Graduate Natural Language Processing (INST 735 / CMSC 723 / LING 723)

Logistics

Location	Iribe 0318
Time	Mon./Wed. 15:30pm - 16:45pm
Webpage	http://umiacs.umd.edu/~jbg/teaching/CMSC_723/
Mailing List	http://piazza.com/umd/fall2022/cmscling723inst723/home
Text	Speech and Language Processing
Homework Submission	Gradescope
Syllabus	https://docs.google.com/document/d/1HjiQeO5O5ga3o29T2wkJZFMRU3ofSS_htSgCnVGjARI/edit?usp=sharing

People

Professor

Jordan Boyd-Graber
Office Hours (Iribe 4146)

Teaching Assistants

• Connor Baumler: 1:45pm-3:15pm Wednesdays
• Zongxia Li: 10:00am- 11:30am Mondays

Schedule

Date	In-Class Topic	Assignment Due
Mon 29. Aug	Introduction to the course, Probability, and Python
	Lecture: What is the Course About? [Slides] Linguistic Lingo (Required for computer scientists) How to take a class with me [Slides] Class in the Time of Corona [Slides] Hybrid classes Notebook Rant [Slides] Optional Readings (Background): Python (and programming) introductory course You may also want to look at first couple of weeks (through February) of INST 414 as review for statistics. A probability primer (Optional, if you want more detail) Grinstead and Snell, Chapters 1-2, Chapter 4.1 How the statistical revolution changed (computational) linguistics On the deep learning revolution in NLP Math Review Functions Distributions Vectors Matrix math Gradients and Optimization
Wed 31. Aug	Workshop: Git, Toolkits, Worksheets, First Homework
Mon 5. Sep	No Class: Labor Day!
Wed 7. Sep	Language Modeling
	Lecture: Shannon Game [Slides] Estimating Language Models with Conditional Probability [Slides] Back-off and Smoothing [Slides] Evaluating Language Models [Slides] Readings: SLP 3 Chen and Goodman (Optional) Divination Engines: A Media History of Text Prediction (Optional)
Fri 9. Sept	HW 0	Limericks
Mon 12. Sep	Classification
	Lecture: Regression [Slides] Logistic Regression [Slides] Stochastic Gradient Descent [Slides] Readings: SLP Chapter 5
Wed 14. Sep	Workshop: Logistic Regression
Fri 16. Sept	HW 1	Language Model
Mon 19. Sep	Part of Speech
	Lectures: What's a part of speech, and why is it a joint problem? [Slides] Hidden Markov Modeling [Slides] Inferring a Hidden Markov Model from Labeled Data [Slides] Viterbi Algorithm [Slides] Readings: SLP 8
Wed 21. Sep	Deep Learning Crash Course
	Lecture: What is "Deep" Learning? Backprop Frameworks [Slides] Pytorch [Slides] DAN [Sildes] Readings: Tutorial Tons of NLP Models
Fri 23. Sept	HW 2	Classification
Mon 26. Sep	Workshop: Pytorch
Wed 28. Sep	Semantics
	Lecture: What's semantics? [Slides] Distributional Semantics [Slides] Word2Vec [Slides]
Fri 29. Sept	HW 3	Deep Averaging Networks
Mon 3. Oct	Parsing
	Lectures: What's a CFG? [Slides and for below] Estimating a PCFG from data Constituency Parsing a Sentence Dependency parsing [Slides] Shift-Reduce Parser [Slides] Readings: SLP 13 and 14
Wed 5. Oct	Workshop: Parsing
Fri 7. Oct	HW 4	Semantics
Mon 10. Oct	Neural Language Models I
	Lectures: RNNs [Slides] LSTM [Slides] Contextual Neural Language Models Overview [Slides] Transformer Encoder [Slides]
	Readings: SLP 7
Wed 12. Oct	Workshop: LSTM
Mon 17. Oct	Neural Language Models II
	Lectures: BERTology [Slides, and for below] Decoder Implementation Details Readings: The Annotated Transformer
Wed 19. Oct	Slack Day: Project Brainstorming
Fri 21. Oct	HW 5	Parsing
Mon 24. Oct	Feature Engineering
	Lecture: Feature Engineering in the Age of Deep Learning Feature Engineering for Quiz Bowl [Slides (for this and below)] Feature Engineering for Spoiler Detection Readings: Feature Engineering for Machine Learning Chapters 2, 5
Wed 26. Oct	Midterm Review
Mon 31. Oct	Midterm
Wed 2. Nov	Application: Topic Modeling
	Lecture: Topic modeling in neural times [Slides] What's Topic Modeling? [Slides] Gibbs Sampling [Slides] Topic Model Evaluation Optional Lectures: Variational Inference Big Picture [Slides, and below] Variational Inference for LDA Readings: Applications of Topic Models Chapters 1
Fri 4. Nov	Project Proposal Due	Proposal
Mon 7. Nov	Application: Named Entities / Entity Linking / Coreference
	Lecture: Coreference Resolution Entity Linking / Wikification Relation Extraction Readings: SLP Chapter 21
Wed 9. Nov	Workshop: Fine-Tuning
Fri 11. Nov	HW 7	Fine Tuning
Mon 14. Nov	Application: Machine Translation
	Lecture: Translation is Hard Word-Based [Slides] Phrase [Slides] Neural [Neural] Evaluation Readings: SLP 10
Wed 16. Nov	Project Office Hours (Online w/ JBG), Topic Modeling Workshop in Class w/ TAs
Fri 18. Nov	HW 8	Topic Model
Mon 21. Nov	Application: Question Answering I
	Videos: The Turing Test Statistical QA / Watson Watson on Jeopardy! [Slides] QA Datasets [Slides] [Slides] Cranfield vs. Manchester [Slides] Readings: Turing's Two Tests for Intelligence The Watson Project Cranfield vs. Mancester (Optional) Taxonomy of NLP QA
Mon 28. Nov	Application: Question Answering II
	Videos: Dr. QA [Slides] Dense Retrieval [Slides] BERT for MRQA Multihop Generation for QA Readings: SLP 23
Wed 30. Nov	Careers in NLP
	Videos: Course Evaluations (Not today's topic, but course evals are out!) What are the Primary Venues for Machine Learning and Artificial Intelligence You got a PhD in Computational Linguistics ... what now? How can you have an impact on computational linguistics
Mon 5. Dec	Ethical/Explainable AI
	Videos: What makes for an explanation AI Fairness Optimizing ML for Society Reading: Brooks Etzioni
Wed 7. Dec	AMA
	No videos or reading, we'll talk about whatever you want to talk about (except religion or politics).
Mon 12. Dec	Paper Clinic (Online Only)
	Videos: What's a paper clinic? Why writing well is important How to structure sentences and paragraphs What a good paper looks like How to write an effective abstract Writing with LaTeX Writing for Multiple Audiences Citation mistakes Most Importantly: Have a draft of your report!
Tue 20. Dec 1:30pm - 3:30pm	Final Poster Session

Page by Jordan Boyd-Graber.