You enter a dark forest. Standing in front of you is:

A professor named Hal Daumé III (he/him). He wields appointments in Computer Science where he is a Perotto Professor, as well as Language Science at UMD (in Fall 2021 he is teaching Just Machine Learning); he is also a Senior Principal Researcher the machine learning and fairness groups at Microsoft Research NYC. He and his wonderful advisees like to study questions related to how to get machines to becomes more adept at human language (and artificial intelligence tasks more broadly), by developing models and algorithms that allow them to learn from data. (Keywords: natural language processing and machine learning.) The two major questions that really drive their research these days are:

    (1) how can we get computers to learn
        through natural interaction with people/users?

and (2) how can we do this in a way that minimize harms
        in the learned models?

He's discussed interactive learning informally in a Talking Machines Podcast and more technically in recent talks; and has discussed fairness/bias in broad terms in a (now somewhat outdated) blog post. He is the author of the online textbook A Course in Machine Learning, which is fully open source.

Hal is super fortunate to be a member of, and have awesome colleagues in the Computional Linguistics and Information Processing Lab (which he formerly directed), the Human-Computer Interaction Lab, and the Center for Machine Learning. If you want to contact him, email is your best bet; you can also find him on @haldaume3 on Twitter. Or, in person, in his office (IRB 4150).

If you're a prospective grad student or grad applicant, please read his FAQ to answer some common questions. If you're thinking of inviting him for a talk or event, please ensure that the event is organized in an inclusive manner (inclusion rider). More generally, if you are organizing a conference, workshop or other event, you may wish to read the NeurIPS D&I survey results (joint with Katherine Heller), Humberto Corona's collection of resources/advice, or two blog posts on this topic.

I acknowledge that I live and work on the ancestral and unceded lands of the Piscataway People, who were among the first in the Western Hemisphere to encounter European colonists, as well as the lands of the Lenape and Nacotchtank people.

Recent Publications:

From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidence
David Alvarez-Melis, Harmanpreet Kaur, Hal Daumé III, Hanna Wallach and Jennifer Wortman Vaughan
HCOMP, 2021
[Abstract] [BibTeX]

We take inspiration from the study of human explanation toinform the design and evaluation of interpretability methodsin machine learning. First, we survey the literature on humanexplanation in philosophy, cognitive science, and the socialsciences, and propose a list of design principles for machine-generated explanations that are meaningful to humans. Usingthe concept of weight of evidence from information theory,we develop a method for generating explanations that adhereto these principles. We show that this method can be adaptedto handle high-dimensional, multi-class settings, yielding aflexible framework for generating explanations. We demon-strate that these explanations can be estimated accuratelyfrom finite samples and are robust to small perturbations ofthe inputs. We also evaluate our method through a qualita-tive user study with machine learning practitioners, wherewe observe that the resulting explanations are usable de-spite some participants struggling with background conceptslike prior class probabilities. Finally, we conclude by surfac-ing design implications for interpretability tools in general.

@inproceedings{daume21woe,
title = {From Human Explanation to Model Interpretability: A Framework Based on
Weight of Evidence},
author = {David Alvarez-Melis and Harmanpreet Kaur and Hal {Daum\'e III} and
Hanna Wallach and Jennifer Wortman Vaughan},
booktitle = {HCOMP},
year = {2021},
url = {http://hal3.name/docs/#daume21woe},
}


Analyzing Stereotypes in Generative Text Inference Tasks
Anna Sotnikova, Yang Trista Cao, Hal Daumé III and Rachel Rudinger
ACL (findings), 2021
[Abstract] [BibTeX]

Stereotypes are inferences drawn about people based on their demographic attributes, which may result in harms to users when a system is deployed. In generative language-inference tasks, given a premise, a model produces plausible hypotheses that follow either logically (natural language inference) or commonsensically (commonsense inference). Such tasks are therefore a fruitful setting in which to explore the degree to which NLP systems encode stereotypes. In our work, we study how stereotypes manifest when the potential targets of stereotypes are situated in real-life, neutral contexts. We collect human judgments on the presence of stereotypes in generated inferences, and compare how perceptions of stereotypes vary due to annotator positionality

@inproceedings{daume21stereotypes,
title = {Analyzing Stereotypes in Generative Text Inference Tasks},
author = {Anna Sotnikova and Yang Trista Cao and Hal {Daum\'e III} and Rachel
Rudinger},
booktitle = {ACL (findings)},
year = {2021},
url = {http://hal3.name/docs/#daume21stereotypes},
}


Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval
Chen Zhao, Chenyan Xiong, Jordan Boyd-Graber and Hal Daumé III
NAACL (short), 2021
[Abstract] [BibTeX]

Complex question answering often requires finding a reasoning chain that consists of multiple evidence pieces. Current approaches incorporate the strengths of structured knowledge and unstructured text, assuming text corpora is semi-structured. Building on dense retrieval methods, we propose a new multi-step retrieval approach (B EAM DR) that iteratively forms an evidence chain through beam search in dense representations. When evaluated on multi-hop question answering, B EAM DR is competitive to state-of-the-art systems, without using any semi-structured information. Through query composition in dense space, B EAM DR captures the implicit relationships between evidence in the reasoning chain. The code is available at https://github.com/ henryzhao5852/BeamDR

@inproceedings{daume21beamdr,
title = {Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval},
author = {Chen Zhao and Chenyan Xiong and Jordan Boyd-Graber and me},
booktitle = {NAACL (short)},
year = {2021},
url = {http://hal3.name/docs/#daume21beamdr},
}


Distantly-Supervised Evidence Retrieval Enables Question Answering without Annotated Evidence Pieces
Chen Zhao, Chenyan Xiong, Jordan Boyd-Graber and Hal Daumé III
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021
[BibTeX]

@inproceedings{daume21distqa,
title = {Distantly-Supervised Evidence Retrieval Enables Question Answering
without Annotated Evidence Pieces},
author = {Chen Zhao and Chenyan Xiong and Jordan Boyd-Graber and me},
booktitle = {Proceedings of the Conference on Empirical Methods in Natural
Language Processing (EMNLP)},
year = {2021},
url = {http://hal3.name/docs/#daume21distqa},
}


Prior and Prejudice: The Novice Reviewers' Bias against Resubmissions in Conference Peer Review
Ivan Stelmakh, Nihar B. Shah, Aarti Singh and Hal Daumé III
CSCW, 2021
[Abstract] [BibTeX]

Modern machine learning and computer science conferences are experiencing a surge in the number of submissions that challenges the quality of peer review as the number of competent reviewers is growing at a much slower rate. To curb this trend and reduce the burden on reviewers, several conferences have started encouraging or even requiring authors to declare the previous submission history of their papers. Such initiatives have been met with skepticism among authors, who raise the concern about a potential bias in reviewers’ recommendations induced by this information. In this work, we investigate whether reviewers exhibit a bias caused by the knowledge that the submission under review was previously rejected at a similar venue, focusing on a population of novice reviewers who constitute a large fraction of the reviewer pool in leading machine learning and computer science conferences. We design and conduct a randomized controlled trial closely replicating the relevant components of the peer-review pipeline with 133 reviewers (master’s, junior PhD students, and recent graduates of top US universities) writing reviews for 19 papers. The analysis reveals that reviewers indeed become negatively biased when they receive a signal about paper being a resubmission, giving almost 1 point lower overall score on a 10-point Likert item (∆ = −0.78, 95\% CI = [−1.30, −0.24]) than reviewers who do not receive such a signal. Looking at specific criteria scores (originality, quality, clarity and significance), we observe that novice reviewers tend to underrate quality the most.

@inproceedings{daume21resubmit,
title = {Prior and Prejudice: The Novice Reviewers' Bias against Resubmissions in
Conference Peer Review},
author = {Ivan Stelmakh and Nihar B. Shah and Aarti Singh and me},
booktitle = {CSCW},
year = {2021},
url = {http://hal3.name/docs/#daume21resubmit},
}


More papers please!

Recent Talks:

(Meta-)Learning from Interaction
NYU Machine Learning Reading Group 2019
[PDF] [ODP]

Out of Order! Flexible neural language generation
NAACL 2019 NeuralGen Workshop
[PPTx]

Beyond demonstrations: Learning behavior from higher-level supervision
ICML 2019 I3 Workshop
[PPTX]

Imitation Learning
Vector Institute Reinforcement Learning Summer School 2018
[PDF] [ODP]

Learning language through interaction
December 2016, Georgetown, Amazon, USC, GATech, UW, ...
[PDF] [ODP] [Video]

Bias in AI
November 2016, UMD MCWIC Diversity Summit
[PDF] [ODP] [PPTx (exported)] [Blog Post]

More talks please!

Contact information:

    email: me AT hal3 DOT name               skype: haldaume3
    phone: 301-405-1073                    twitter: haldaume3
   office: IRB 4150                         github: hal3

I can't reply to all prospective students email; please read this before emailing me.

credits: design and font inspired by Seth Able's LoRD, some images converted to ANSI using ManyTools, original drawing of me by anonymous.
last updated on eleven october, two thousand twenty one.