CMSC 470

Logistics

Location	Iribe 1116
Q & A	Mon. / Wed. 08:30 AM - 09:45 AM
Webpage	http://umiacs.umd.edu/~jbg/teaching/CMSC_470/
Homework Submission	https://www.gradescope.com/courses/670788
Mailing List	https://piazza.com/umd/fall2025/cmsc470
Text	Speech and Natural Language Processing
Syllabus	https://docs.google.com/document/d/1lrCYFd20FYuwSi7Zrhj-Zqk4QotFFnJT92sxdN67z8M/edit?usp=sharing

People

Professor

Jordan Boyd-Graber
IRB 4146
Office Hours (IRB 4146 / Online): By appointment

Teaching Assistants (Check Piazza for Locations)

Wichayaporn "Joy" Wongkamjan (OH Thursdays 15:00-17:00, Online)
Zongxia Li (OH Fridays 9:00-10:00, 11:00-12:00)
Ashish Seth (OH Fridays 15:00-17:00)

Schedule

Date	In-Class Topic	Assignment Due
Wednesday, 03. September 2025	What the class is about?
	Content: Welcome to NLP What's a word? What's a language? Tokenization BPE Administrivia: How to read this webpage How to take a class with me More on the Flipped Classroom Hybrid classes Grader Expectations Readings: SLP2 Linguistic Fundamentals for Natural Language Processing: 100 Essentials from Morphology and Syntax (Chapters 1 and 2) Alternate Link Optional Readings: Python (and programming) introductory course Jon Hewitt's Math Notes First weeks of INST 414 as review A probability primer Grinstead and Snell, Chapters 1-2, Chapter 4.1 How the statistical revolution changes (computational) linguistics The deep learning revolution in NLP How Much is Enough? The Diminishing Returns of Tokenization Training Data
Friday, 05. September 2025	Homework Due	Warmup
Monday, 08. September 2025	AI vs. NLP
	Videos: What's the difference between NLP, AI, and ChatGPT? What made generative AI possible What to call these models? Shannon Game Evaluating Language Models The Turing Test Optional Review: What are expectations and entropy? administrivia Why you shouldn't use generative AI to cheat in my class Readings: Language Models: A Guide for the Perplexed SLP3
Wednesday, 10. September 2025	Information Retrieval
	Videos: The Cranfield Paradigm What is information retrieval? TF-IDF Evaluation RAG Readings: IR Chapter 1, 6 SLP 11.1 Evolution of Cranfield (Optional) Slides: Cranfield IR Intro IR tf-idf IR eval IR exercise
Friday, 12. September 2025	Homework Due	Tokenization and tf-idf
Monday, 15. September 2025	Logistic Regression
	Videos: Regression Logistic Regression Stochastic Gradient Descent Feature Engineering in the Age of Deep Learning Feature Engineering for Quiz Bowl Slides: Logistic Regression w/ SGD Example Readings: SNLP Chapter 4
Wednesday, 17. September 2025	Continuing Logistic Regression / Homework Workshop

Friday, 19. September 2025	Homework Due	Logistic Regression
Monday, 22. September 2025	Word representations
	Videos: Distributional Semantics Word2Vec Evaluation Readings: SNLP Chapter 5 (formerly 6) Slides: Distributional Semantics Word2Vec Eval Exercise Exercise (Derivation)
Wednesday, 24. September 2025 [Rosh Hashana]	Pytorch
	Videos: Deep Backprop Frameworks Pytorch DAN ADAM Readings: SNLP Chapter 6 (Was 6) Slides: Deep Backprop Frameworks Pytorch DAN ADAM Exercise
Friday, 26. September 2025	Homework Due	Feature Engineering
Monday, 29. September 2025	Homework Framework
	Videos: Advice / Overview How to stump a computer Good questions Bad questions optional_videos IRT optional_past_finals 2019 Finals 2022 Finals
Wednesday, 01. October 2025	Syntax
	Videos: What's a part of speech, and why is it a joint problem? Hidden Markov Modeling Constituency grammars Dependency parsing Readings: SNLP Chapters 17.1 and 17.2 SNLP Chapter 18.1 - 18.3 SNLP Chapter 19.1 - 19.2 Slides Sequence Modeling HMM Constituency grammars Dependency HMM Ex PCFG Ex Dependency Ex Optional Lectures Viterbi Algorithm PCFGs CYK Parsing Shift-Reduce Parser
Friday, 03. October 2025	Homework Due	Sparse Retrieval
Monday, 06. October 2025	Linear neural language models (Last content on Midterm I)
	Videos: Continuous BOW for Sentiment RNN for Sentiment RNN for LM LSTM Backpack Sequence models in Pytorch Readings: SNLP 19 (Was 9) Optional Readings: Backpack LM Slides: RNN sentiment RNN LM LSTM RNN Exercise Optional Videos: LSTM
Wednesday, 08. October 2025 [Succos]	Homework Workshop

Friday, 10. October 2025 [Succos]	Homework Due	Pytorch LR + Adam
Monday, 13. October 2025 [Succos]	Fall Break
Wednesday, 15. October 2025 [Simchas Torah]	Transformers
	Videos: Contextual Neural Language Models Overview Transformer Encoder BERTology Decoder Implementation Details Readings: SNLP 8 SNLP 7-7.3 Optional Readings Annotated Transformer Illustrated Transformer Transformer in an Excel Spreadsheet Topics Muppet Models Attention w/ Backpacks Transformers Induction Head Exercise
Monday, 20. October 2025	Decoding and Efficiency
	Videos: All Videos (Descript) Qantization Mixture of Experts Speculative Decoding KV Caching Slides: Qantization Mixture of Experts Speculative Decoding KV Caching Reading: Efficient Large Language Models: A Survey (LoRA bit is relevant for later class)
Wednesday, 22. October 2025	Alignment
	Slides: Alignment + RLHF RL Review Policy Gradient Gricean Exercise Videos: Alignment RL Review Policy Gradient RLHF Readings: SLP 9 Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Friday, 24. October 2025	Homework Due	DAN
Monday, 27. October 2025	Finetuning
	Videos: Prompting: What is it good for? DSPY Example QLoRA Slides: Promespting DSPy Example QLoRA LoRA Ex Optional Reading: SIMBA optional_videos UG Research at UMD on Prompting
Wednesday, 29. October 2025 [Shavuos]	Machine translation
	Videos: Translation is Hard Word-based Translation Phrase-based Translation Neural Machine Translation Decoding Evaluation Lectures What is MT, why is it hard, how is it evaluated? Word-based Models Phrase-based Models Neural Decoding Nucleus Sampling Exercise Readings: NMT Book (Chapter 5)
Friday, 31. October 2025	Homework Due	Transformer LM (Group)
Monday, 03. November 2025	Midterm Review

Wednesday, 05. November 2025	Midterm I

Monday, 10. November 2025	No Class: Makeup Midterm

Wednesday, 12. November 2025	Zongxia Guest Lecture: Running Jobs on Nexus

Friday, 14. November 2025	Homework Due	Adversarial Authoring (Group)
Monday, 17. November 2025	Question Answering Approaches, Datasets, and Eval
	Videos: Dense Retrieval BERT for MRQA Multihop Watson on Jeopardy! Cranfield vs. Manchester Slides Dense Retrieval Multihop Watson Manchester vs. Cranfield Exercise Readings: MRC Literature Review SLP (old edition) --- Ignore IR review
Wednesday, 19. November 2025	Item Response Theory
	Videos: Standardized Tests IRT IRT for Leaderboards Multiple Choice Slides: Multiple Choice Standardized + IRT Exercise Readings: A visual guide to IRT Optional Readings: IRT Tutorial (Useful for reference list)
Friday, 21. November 2025	Homework Due	LM Optimization (Group)
Monday, 24. November 2025	Adversarial Examples / Red Teaming
	There's no preparation needed for class on Monday, but please make sure to come with a device (and perhaps a pen and paper). We'll be trying to answer each others' adversarial questions. Why it's important to come: This will be a practice for our final project This will count as 10 points on the adversarial HW You'll get a better intuition for how the final project works
Wednesday, 26. November 2025	Thanksgiving
Monday, 01. December 2025	Midterm Review

Wednesday, 03. December 2025	Midterm II

Monday, 08. December 2025	Expo Match

Wednesday, 10. December 2025	Ask Me Anything
	Videos: Course Evals You got a PhD in Computational Linguistics ... what now? Optimizing ML for Society How far off were my predictions? and p(Doom)
Monday, 15. December 2025 [Chanuka]	Final Exam (for computers): 1:30pm - 3:30pm
	How this will work: You'll arrive You'll be given three possible themes for writing a question You'll pick one of them You'll write a multiple choice question on paper over the course of an hour You'll turn in your paper multiple choice question on paper You'll then be able to enter the same question on a device to turn it in We'll then have a break to prepare the questions for everyone to answer You'll then try to answer the questions that people wrote Your score for the question will be the delta between computer difficulty and human difficulty for the question (as measured by IRT).

Natural Language Processing (CMSC 470)

Logistics

People

Professor

Teaching Assistants (Check Piazza for Locations)

Schedule