Jordan Boyd-Graber

Jordan Boyd-Graber
4146 Iribe
Computer Science
jbg@boydgraber.org

How and Why Artificial Intelligence Answers Questions (CMSC 848Q)

Logistics

Location	CSIC 2117
Time	Mon./Wed. 14:00 - 15:15
Webpage	http://umiacs.umd.edu/~jbg/teaching/CMSC_848/
Homework Submission	https://www.gradescope.com/courses/303336
Mailing List	http://piazza.com/umd/fall2023/cmsc848q/home
Syllabus	https://docs.google.com/document/d/1VaIUHiEVU6dZ6y8uega-Z4SV6R4KdFSNkbXAo4V5Dck/edit?usp=sharing

People

Professor

Jordan Boyd-Graber
IRB 4146
Office Hours (IRB 4146): By appointment

Teaching Assistant

Pulkit Kumar Office Hours: Thursday 11AM

Schedule

Date	In-Class Topic	Assignment Due
Mon 28. Aug	Introducing the Course
	Lecture: Structure of the course, why QA is important [Slides] Homework Structure [Note: This is from my undergrad NLP course, we're using the same framework for the first homeworks to get you up to speed.] How to take a class with me [Slides Hybrid classes Flipped Classrooms Grader Expectations Reading: Chapter 1 of Question Answering for the Curated Web Tasks and Methods in QA over Knowledge Bases and Text Collections (Alternate link) Textbook Draft: Questioning Artificial Intelligence, Chapter 1 Source: About Source: Author Source: Structure
Wed 30. Aug	QA History / Background
	[More administrivia] How to read the course webpage How question answering saved civilization [Slides, including Turing Test] What AI Researchers can Learn from Trivia Nerds [Slides] A history of recreational question answering [Slides] The Turing Test Watson on Jeopardy! [Slides] Reading: Introduction to Braniac Turing's Two Tests for Intelligence The Watson Project Textbook Draft: Questioning Artificial Intelligence, Chapters 2--4 Source: Myth Civilization Turing
Wed 6. Sep	Information Retrieval / tf-idf
	Lecture: What is information retrieval? [Slides] TF-IDF [Slides] Evaluation [Slides] The Cranfield Paradigm [Slides] Readings: IR Chapter 1, 6 Evolution of Cranfield (Alternate link)
Fri 8. Sep	Homework Due	Warmup
Mon 11. Sep	History of Computational Question Answering
	Lecture: GOFAI Approaches for QA [Slides] Statistical QA / Watson KB QA Feature Engineering in the Age of Deep Learning Feature Engineering for Quiz Bowl Readings: NLQA: The View from Here Chapter 3 of Question Answering for the Curated Web Tasks and Methods in QA over Knowledge Bases and Text Collections (Alternate link)
Wed 13. Sep	Neural Sequence Models for Language I
	Lecture: Distributional Semantics [Slides] Word2Vec [Slides] RNN for Sentiment [Slides] RNN for LM [Slides] LSTM [Slides] Readings: SNLP Chapter 9
Fri 15. Sep	Homework Due	IDF
Mon 18. Sep	Neural Sequence Models for Language II
	Lecture: Neural Language Models [Slides] What to call these models? [Optional] Follow-up Transformer Encoder [Slides] BERTology [Slides, and for below] Decoder Implementation Details Readings: SNLP Chapter 9 The Annotated Transformer
Wed 20. Sep	Reading and Presenting a Paper
	Lecture: How I Record Videos (Ironically enough, not edited as cleanly as I would like) Lost Opportunity of Conference Videos Why Virtual Conferences will Continue to Suck How to Read a Paper How to Make a Good Presentation (Guest Lecture) Optional: Letitia's recording setup Reading: Role-playing seminar
Fri 22. Sep	Homework Due	Paper Presentation
Mon 25. Sep	QA Datasets
	Lecture: QA Datasets [Slides] [Slides] Cranfield vs. Manchester [Slides] Multihop [Slides] Natural Questions Five Years Later Do iid questions even Exist? Readings: SNLP 23 (Optional) Taxonomy of NLP QA
Wed 27. Sep	Neural Models for QA
	Dr. QA [Slides] Dense Retrieval [Slides] BERT for MRQA RL Review [Slides] Alignment [Slides] Policy Gradient [Slides] RLHF Reading: MRC Literature Review
Fri 29. Sep	Homework Due	QA Exploration
Mon 2. Oct	Computational Frameworks
	Lecture: Frameworks [Slides] Pytorch [Slides] DAN [Sildes] RNN in Pytorch (Guest Lecture) Reading: Minitorch (Up to autodiff)
Wed 4. Oct	Entailment and Fact Checking
	Videos: NLI [Slides] Fact Checking Readings: NLI Automated Fact Checking
Fri 6. Oct	Homework Due	Project Proposal
Mon 9. Oct	Midterm Review
Wed 11. Oct	Midterm
Mon 16. Oct	Beyond Factoid QA: Generation and Long-form QA
	Evaluating Open-Domain Question Answering in the Era of Large Language Models Generation for QA [Slides] Long QA Reading: Generative Long-form Question Answering: Relevance, Faithfulness and Succinctness (Chapter 2) Evaluating Open-Domain Question Answering in the Era of Large Language Models
Wed 18. Oct	Beyond Factoid QA: Conversational
	Ditch the Gold Standard: Re-evaluating Conversational Question Answering Conversational QA Reading: Conversational information seeking (Chapters 2,7) Ditch the Gold Standard: Re-evaluating Conversational Question Answering
Fri 20. Oct	Project	Intro
Mon 23. Oct	How to Write a Paper I
	Videos: Why writing well is important How to structure sentences and paragraphs What a good paper looks like How to write an effective abstract Citation mistakes Readings: On Writing Well (Part I)
Wed 25. Oct	Evaluation: Leaderboards and IRT
	Videos: History of Standardized Tests IRT IRT for QA Eval QA as Gameshow Readings: Recent Developments in Item Response Theory with Implications for Teacher Certification Introduction to IRT for NLP
Fri 27. Oct	Project	Baselines
Mon 30. Oct	Chain of Thought
	Videos: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Answering Questions by Meta-Reasoning over Multiple Chains of Thought Readings: Answering Questions by Meta-Reasoning over Multiple Chains of Thought Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wed 1. Nov	How to Write a Paper II
	Videos: Writing with LaTeX Writing for Multiple Audiences How to Review a Paper Taking advantage of virtual presentations Readings: A Beginner's Guide to LaTeX How to Review Conference Submissions
Mon 6. Nov	Explainable, Cooperative, and Adversarial QA
	Reading: Red Teaming Language Models with Language Models Toward a Rigorous Science of Interpretable Machine Learning Mythos of Interpretability Videos: Red Teaming Language Models with Language Models What makes for an explanation QA Explanation Cooperative QA Adversarial QA
Wed 8. Nov	Multimodal QA: Audio, Video
	Videos: An Empirical Study on Leveraging Scene Graphs for Visual Question Answering Speech [Slides] Images Manchester Multimedia Questions Readings: An Empirical Study on Leveraging Scene Graphs for Visual Question Answering A Survey on VQA: Datasets and Approaches (Very hard to find things that are good and recent, suggestions welcome!)
Mon 13. Nov	QA Memory and Personalization
	Videos: Episodic Memory Question Answering Personalized QA Readings: Episodic Memory Question Answering Situated QA
Wed 15. Nov	Other QA Formats: MC, Crosswords
	Videos: International QA Formats Crosswords Multiple Choice Readings: Dr. Fill
Fri 17. Nov	Project	Draft / Experiment Plan
Mon 20. Nov	Project Workshop I
THANKSGIVING: NO CLASS
Mon 27. Nov	Graphics and Figures
	Reading: Plotnine Videos: The Grammar of Graphics Introducing Plotnine Napoleon in Russia How not to visualize something How to Present a Figure
Wed 29. Nov	Fairness in Question Answering
	Reading: AI Predictions Addressing Fairness, Bias, and Appropriate Use of Artificial Intelligence and Machine Learning in Global Health On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? QA Demographics (Optional) UnQovering (Optional) Videos: AI Fairness QA Fairness AI and Society
Fri 1. Dec	Project	Revision + Plots
Mon 4. Dec	Ask Me Anything: No Reading, No Videos
	Optional Video: Course Evaluations
Wed 6. Dec	Paper Clinic
	Video: What's a Paper Clinic? Most Importantly: Have a draft of your report!
Mon 11. Dec	Project Presentations
Fri 15. Dec	Final Writeup

Page by Jordan Boyd-Graber.