Hal Daumé III - about me

You enter a dark forest. Standing in front of you is:

A professor named Hal Daumé III (he/him). He wields appointments in Computer Science where he is a Volpi-Cupal Professor, as well as Language Science at UMD where he leads the TRAILS, the Institute for Trustworthy AI in Law & Society (in Fall 2025 he's teaching a grad seminar AI Agents (past: You and I, and Generative AI (S24), Trustworthy ML (F23), AI (S23), Human-AI Interaction (F22), Just ML (F21)); he was formerly also a Senior Principal Researcher at Microsoft Research NYC. He and his wonderful advisees like to study questions related to how to get machines to becomes more adept at human language (and artificial intelligence tasks more broadly), by developing models and algorithms that allow them to learn from data. (Keywords: natural language processing and machine learning.) The two major questions that really drive their research these days are:

    (1) how can we get computers to learn
        through natural interaction with people/users?

and (2) how can we do this in a way that minimize harms
        in the learned models?

He's discussed interactive learning informally in a Talking Machines Podcast and more technically in recent talks; and has discussed fairness/bias in broad terms in a (now somewhat outdated) blog post. He is the author of the online textbook A Course in Machine Learning, which is fully open source.

Hal is super fortunate to be a member of, and have awesome colleagues in the Computional Linguistics and Information Processing Lab (which he formerly directed), the Human-Computer Interaction Lab, and the Center for Machine Learning. If you want to contact him, email is your best bet; you can also find him on @haldaume3 on Twitter. Or, in person, in his office (IRB 4134).

If you're a prospective grad student or grad applicant, please read his FAQ to answer some common questions. If you're thinking of inviting him for a talk or event, please ensure that the event is organized in an inclusive manner (inclusion rider). More generally, if you are organizing a conference, workshop or other event, you may wish to read the NeurIPS D&I survey results (joint with Katherine Heller), Humberto Corona's collection of resources/advice, or two blog posts on this topic.

I acknowledge that I live and work on the ancestral and unceded lands of the Piscataway People, who were among the first in the Western Hemisphere to encounter European colonists, as well as the lands of the Lenape and Nacotchtank people.

Recent Publications:

Beyond Blanket Masking: Examining Granularity for Privacy Protection in Images Captured by Blind and Low Vision Users
Jeffri Murrugarra-Llerena, Haoran Niu, Suzanne Barber, Hal Daumé III, Yang Trista Cao and Paola Cascante-Bonilla
COLM, 2025
[Abstract] [BibTeX]

As visual assistant systems powered by visual language models (VLMs) become more prevalent, concerns over user privacy have grown, particularly for blind and low vision users who may unknowingly capture personal private information in their images. Existing privacy protection methods rely on coarse-grained segmentation, which uniformly masks entire private objects, often at the cost of usability. In this work, we propose FiGPriv, a fine-grained privacy protection framework that selectively masks only high-risk private information while preserving low-risk information. Our approach integrates fine-grained segmentation with a data-driven risk scoring mechanism. We evaluate our framework using the BIV-Priv-Seg dataset and show that FiG-Priv preserves +26% of image content, enhancing the ability of VLMs to provide useful responses by 11% and identify the image content by 45%, while ensuring privacy protection. Project Page: this https URL

@inproceedings{daume25masking, title = {Beyond Blanket Masking: Examining Granularity for Privacy Protection in Images Captured by Blind and Low Vision Users}, author = {Jeffri Murrugarra-Llerena and Haoran Niu and Suzanne Barber and Daum\'e, III, Hal and Yang Trista Cao and Paola Cascante-Bonilla}, booktitle = {COLM}, year = {2025}, url = {http://hal3.name/docs/#daume25masking}, }

Language Models Predict Empathy Gaps Between Social In-groups and Out-groups
Yu Hou, Hal Daumé III and Rachel Rudinger
Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2025
[Abstract] [BibTeX]

Studies of human psychology have demonstrated that people are more motivated to extend empathy to in-group members than out-group members (Cikara et al., 2011). In this study, we investigate how this aspect of intergroup relations in humans is replicated by LLMs in an emotion intensity prediction task. In this task, the LLM is given a short description of an experience a person had that caused them to feel a particular emotion; the LLM is then prompted to predict the intensity of the emotion the person experienced on a numerical scale. By manipulating the group identities assigned to the LLM's persona (the "perceiver") and the person in the narrative (the "experiencer"), we measure how predicted emotion intensities differ between in-group and out-group settings. We observe that LLMs assign higher emotion intensity scores to in-group members than out-group members. This pattern holds across all three types of social groupings we tested: race/ethnicity, nationality, and religion. We perform an in-depth analysis on Llama-3.1-8B, the model which exhibited strongest intergroup bias among those tested.

@inproceedings{daume25empathygap, title = {Language Models Predict Empathy Gaps Between Social In-groups and Out-groups}, author = {Yu Hou and Daum\'e, III, Hal and Rachel Rudinger}, booktitle = {Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)}, year = {2025}, url = {http://hal3.name/docs/#daume25empathygap}, }

Natural Language Inference Improves Compositionality in Vision-Language Models
Paola Cascante-Bonilla, Yu Hou, Yang Trista Cao, Hal Daumé III and Rachel Rudinger
ICLR, 2025
[Abstract] [BibTeX]

Compositional reasoning in Vision-Language Models (VLMs) remains challenging as these models often struggle to relate objects, attributes, and spatial relationships. Recent methods aim to address these limitations by relying on the semantics of the textual description, using Large Language Models (LLMs) to break them down into subsets of questions and answers. However, these methods primarily operate on the surface level, failing to incorporate deeper lexical understanding while introducing incorrect assumptions generated by the LLM. In response to these issues, we present Caption Expansion with Contradictions and Entailments (CECE), a principled approach that leverages Natural Language Inference (NLI) to generate entailments and contradictions from a given premise. CECE produces lexically diverse sentences while maintaining their core meaning. Through extensive experiments, we show that CECE enhances interpretability and reduces overreliance on biased or superficial features. By balancing CECE along the original premise, we achieve significant improvements over previous methods without requiring additional fine-tuning, producing state-of-the-art results on benchmarks that score agreement with human judgments for image-text alignment, and achieving an increase in performance on Winoground of +19.2% (group score) and +12.9% on EqBen (group score) over the best prior work (finetuned with targeted data).

@inproceedings{daume25nli, title = {Natural Language Inference Improves Compositionality in Vision-Language Models}, author = {Paola Cascante-Bonilla and Yu Hou and Yang Trista Cao and Daum\'e, III, Hal and Rachel Rudinger}, booktitle = {ICLR}, year = {2025}, url = {http://hal3.name/docs/#daume25nli}, }

Which Demographic Features Are Relevant for Individual Fairness Evaluation of U.S. Recidivism Risk Assessment Tools?
Tin Nguyen, Jiannan Xu, Phuong-Anh Nguyen-Le, Jonathan Lazar, Donald Braman, Hal Daumé III and Zubin Jelveh
ICAIL, 2025
[Abstract] [BibTeX]

Despite its constitutional relevance, the technical "individual fairness" criterion has not been operationalized in U.S. state or federal statutes/regulations. We conduct a human subjects experiment to address this gap, evaluating which demographic features are relevant for individual fairness evaluation of recidivism risk assessment (RRA) tools. Our analyses conclude that the individual similarity function should consider age and sex, but it should ignore race.

@inproceedings{daume25risk, title = {Which Demographic Features Are Relevant for Individual Fairness Evaluation of U.S. Recidivism Risk Assessment Tools?}, author = {Tin Nguyen and Jiannan Xu and Phuong-Anh Nguyen-Le and Jonathan Lazar and Donald Braman and Daum\'e, III, Hal and Zubin Jelveh}, booktitle = {ICAIL}, year = {2025}, url = {http://hal3.name/docs/#daume25risk}, }

Effort-aware Fairness: Incorporating a Philosophy-informed, Human-centered Notion of Effort into Algorithmic Fairness Metrics
Tin Nguyen, Jiannan Xu, Zora Che, Phuong-Anh Nguyen-Le, Rushil Dandamudi, Donald Braman, Furong Huang, Hal Daumé III and Zubin Jelveh
AIES, 2025
[Abstract] [BibTeX]

Although popularized AI fairness metrics, e.g., demographic parity, have uncovered bias in AI-assisted decision-making outcomes, they do not consider how much effort one has spent to get to where one is today in the input feature space. However, the notion of effort is important in how Philosophy and humans understand fairness. We propose a philosophy-informed approach to conceptualize and evaluate Effort-aware Fairness (EaF), grounded in the concept of Force, which represents the temporal trajectory of predictive features coupled with inertia. Besides theoretical formulation, our empirical contributions include: (1) a pre-registered human subjects experiment, which shows that for both stages of the (individual) fairness evaluation process, people consider the temporal trajectory of a predictive feature more than its aggregate value; (2) pipelines to compute Effort-aware Individual/Group Fairness in the criminal justice and personal finance contexts. Our work may enable AI model auditors to uncover and potentially correct unfair decisions against individuals who have spent significant efforts to improve but are still stuck with systemic disadvantages outside their control.

@inproceedings{daume25eaf, title = { Effort-aware Fairness: Incorporating a Philosophy-informed, Human-centered Notion of Effort into Algorithmic Fairness Metrics}, author = {Tin Nguyen and Jiannan Xu and Zora Che and Phuong-Anh Nguyen-Le and Rushil Dandamudi and Donald Braman and Furong Huang and Daum\'e, III, Hal and Zubin Jelveh}, booktitle = {AIES}, year = {2025}, url = {http://hal3.name/docs/#daume25eaf}, }

More papers please!

Recent Talks:

AI UK: Doing better in data science – from algorithmic fairness to diversity
Anjali Mazumder, Shakir Mohamed, Danielle Belgrave, Maria De-Arteaga, and Hal Daumé III
The Alan Turing Institute AI UK Roadmap, March 2021
[Video]

Coded Bias Panel Discussion at the University of Maryland
Margrét Bjarnadóttir, Nicol Turner Lee, Deborah Raji, Adam Wenchel, and Hal Daumé III (moderator)
March, 2021
[Video]

Responsible AI Systems and Experiences
Abolfazl Asudeh (moderator), Hal Daumé III, Golnoosh Farnadi, Bernease Herman, Bill Howe (moderator), Yuval Moskovitch, Katie Shilton, and Jenn Wortman Vaughan
Panel at VLDB 2021
[Video]

Tech Ethics in a Changing World
Catherine Bannister, Mary Lacity, Cindy Moehring, and Hal Daumé III
Northwest Arkansas Tech Summit, 2021
[Video]

Language (Technology) Is Power: Exploring the Inherent Complexity of NLP Systems
Hal Daumé III and Sam Charrington (host)
TWIML AI Podcast, 2020
[Video]

More talks please!

Contact information:

    email: me AT hal3 DOT name               skype: haldaume3
    phone: 301-405-1073                    twitter: haldaume3
   office: IRB 4150                         github: hal3

I can't reply to all prospective students email; please read this before emailing me.

credits: design and font inspired by Seth Able's LoRD, some images converted to ANSI using ManyTools, original drawing of me by anonymous.
last updated on twenty two october, two thousand twenty five.