You enter a dark forest. Standing in front of you is:

A professor named Hal Daumé III (he/him). He wields appointments in Computer Science where he is a Perotto Professor, as well as Language Science at UMD (in Fall 2021 he is teaching Just Machine Learning); he also spends time in the machine learning and fairness groups at Microsoft Research NYC. He and his wonderful advisees like to study questions related to how to get machines to becomes more adept at human language (and artificial intelligence tasks more broadly), by developing models and algorithms that allow them to learn from data. (Keywords: natural language processing and machine learning.) The two major questions that really drive their research these days are:

    (1) how can we get computers to learn
        through natural interaction with people/users?

and (2) how can we do this in a way that minimize harms
        in the learned models?

He's discussed interactive learning informally in a Talking Machines Podcast and more technically in recent talks; and has discussed fairness/bias in broad terms in a (now somewhat outdated) blog post. He is the author of the online textbook A Course in Machine Learning, which is fully open source.

Hal is super fortunate to have awesome colleagues in the Computional Linguistics and Information Processing Lab (which he formerly directed) and Center for Machine Learning. If you want to contact him, email is your best bet; you can also find him on @haldaume3 on Twitter. Or, in person, in his office (IRB 4150).

If you're a prospective grad student or grad applicant, please read his FAQ to answer some common questions. If you're thinking of inviting him for a talk or event, please ensure that the event is organized in an inclusive manner (inclusion rider). More generally, if you are organizing a conference, workshop or other event, you may wish to read the NeurIPS D&I survey results (joint with Katherine Heller), Humberto Corona's collection of resources/advice, or two blog posts on this topic.

I acknowledge that I live and work on the ancestral and unceded lands of the Piscataway People, who were among the first in the Western Hemisphere to encounter European colonists, as well as the lands of the Lenape and Nacotchtank people.

Recent Publications:

Prior and Prejudice: The Novice Reviewers' Bias against Resubmissions in Conference Peer Review
Ivan Stelmakh, Nihar B. Shah, Aarti Singh and Hal Daumé III
CSCW, 2021
[Abstract] [BibTeX]

Modern machine learning and computer science conferences are experiencing a surge in the number of submissions that challenges the quality of peer review as the number of competent reviewers is growing at a much slower rate. To curb this trend and reduce the burden on reviewers, several conferences have started encouraging or even requiring authors to declare the previous submission history of their papers. Such initiatives have been met with skepticism among authors, who raise the concern about a potential bias in reviewers’ recommendations induced by this information. In this work, we investigate whether reviewers exhibit a bias caused by the knowledge that the submission under review was previously rejected at a similar venue, focusing on a population of novice reviewers who constitute a large fraction of the reviewer pool in leading machine learning and computer science conferences. We design and conduct a randomized controlled trial closely replicating the relevant components of the peer-review pipeline with 133 reviewers (master’s, junior PhD students, and recent graduates of top US universities) writing reviews for 19 papers. The analysis reveals that reviewers indeed become negatively biased when they receive a signal about paper being a resubmission, giving almost 1 point lower overall score on a 10-point Likert item (∆ = −0.78, 95\% CI = [−1.30, −0.24]) than reviewers who do not receive such a signal. Looking at specific criteria scores (originality, quality, clarity and significance), we observe that novice reviewers tend to underrate quality the most.

@inproceedings{daume21resubmit,
title = {Prior and Prejudice: The Novice Reviewers' Bias against Resubmissions in
Conference Peer Review},
author = {Ivan Stelmakh and Nihar B. Shah and Aarti Singh and me},
booktitle = {CSCW},
year = {2021},
url = {http://hal3.name/docs/#daume21resubmit},
}


A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences
Ivan Stelmakh, Nihar B. Shah, Aarti Singh and Hal Daumé III
AAAI, 2021
[Abstract] [BibTeX]

Conference peer review constitutes a human-computation process whose importance cannot be overstated: not only it identifies the best submissions for acceptance, but, ultimately, it impacts the future of the whole research area by promoting some ideas and restraining others. A surge in the number of submissions received by leading AI conferences has challenged the sustainability of the review process by increasing the burden on the pool of qualified reviewers which is growing at a much slower rate. In this work, we consider the problem of reviewer recruiting with a focus on the scarcity of qualified reviewers in large conferences. Specifically, we design a procedure for (i) recruiting reviewers from the population not typically covered by major conferences and (ii) guiding them through the reviewing pipeline. In conjunction with ICML 2020 — a large, top-tier machine learning conference — we recruit a small set of reviewers through our procedure and compare their performance with the general population of ICML reviewers. Our experiment reveals that a combination of the recruiting and guiding mechanisms allows for a principled enhancement of the reviewer pool and results in reviews of superior quality compared to the conventional pool of reviews as evaluated by senior members of the program committee (meta-reviewers).

@inproceedings{daume21novice,
title = {A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers
in Large Conferences},
author = {Ivan Stelmakh and Nihar B. Shah and Aarti Singh and me},
booktitle = {AAAI},
year = {2021},
url = {http://hal3.name/docs/#daume21novice},
}


On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries
Tianze Shi, Chen Zhao, Jordan Boyd-Graber, Hal Daumé III and Lillian Lee
Findings of EMNLP, 2020
[Abstract] [BibTeX]

Large-scale semantic parsing datasets annotated with logical forms have enabled major advances in supervised approaches. But can richer supervision help even more? To explore the utility of fine-grained, lexical-level supervision, we introduce Squall, a dataset that enriches 11,276 WikiTableQuestions English-language questions with manually created SQL equivalents plus alignments between SQL and question fragments. Our annotation enables new training possibilities for encoder-decoder models, including approaches from machine translation previously precluded by the absence of alignments. We propose and test two methods: (1) supervised attention; (2) adopting an auxiliary objective of disambiguating references in the input queries to table columns. In 5-fold cross validation, these strategies improve over strong baselines by 4.4\% execution accuracy. Oracle experiments suggest that annotated alignments can support further accuracy gains of up to 23.9\%.

@inproceedings{daume20alignments,
title = {On the Potential of Lexico-logical Alignments for Semantic Parsing to
SQL Queries},
author = {Tianze Shi and Chen Zhao and Jordan Boyd-Graber and Hal {Daum\'e III}
and Lillian Lee},
booktitle = {Findings of EMNLP},
year = {2020},
url = {http://hal3.name/docs/#daume20alignments},
code = {https://www.github.com/tzshi/squall},
}


Meta-learning for Few-Shot NMT Adaptation
Amr Sharaf, Hany Hassan and Hal Daumé III
WNGT@ACL, 2020
🏆 Best Paper Award
[Abstract] [BibTeX]

We present META-MT, a meta-learning approach to adapt Neural Machine Translation (NMT) systems in a few-shot setting. M ETA MT provides a new approach to make NMT models easily adaptable to many target domains with the minimal amount of in-domain data. We frame the adaptation of NMT systems as a meta-learning problem, where we learn to adapt to new unseen domains based on simulated offline meta-training domain adaptation tasks. We evaluate the proposed metalearning strategy on ten domains with general large scale NMT systems. We show that M ETA -MT significantly outperforms classical domain adaptation when very few indomain examples are available. Our experiments shows that M ETA -MT can outperform classical fine-tuning by up to 2.5 BLEU points after seeing only 4, 000 translated words (300 parallel sentences).

@inproceedings{daume20nmtadapt,
title = {Meta-learning for Few-Shot NMT Adaptation},
author = {Amr Sharaf and Hany Hassan and me},
booktitle = {WNGT@ACL},
year = {2020},
url = {http://hal3.name/docs/#daume20nmtadapt},
}


Operationalizing the Legal Principle of Data Minimization for Personalization
Asia J. Biega, Peter Potash, Hal Daumé III, Fernando Diaz and MichèB;le Finck
Conference on Research and Developments in Information Retrieval (SIGIR), 2020
[Abstract] [BibTeX]

Article 5(1)(c) of the European Union’s General Data Protection Regulation (GDPR) requires that "personal data shall be [...] adequate, relevant, and limited to what is necessary in relation to the purposes for which they are processed (`data minimisation')". To date, the legal and computational definitions of `purpose limitation' and `data minimization' remain largely unclear. In particular, the interpretation of these principles is an open issue for information access systems that optimize for user experience through personalization and do not strictly require personal data collection for the delivery of basic service. In this paper, we identify a lack of a homogeneous interpretation of the data minimization principle and explore two operational definitions applicable in the context of personalization. The focus of our empirical study in the domain of recommender systems is on providing foundational insights about the (i) feasibility of different data minimization definitions, (ii) robustness of different recommendation algorithms to minimization, and (iii) performance of different minimization strategies. We find that the performance decrease incurred by data minimization might not be substantial, but that it might disparately impact different users--a finding which has implications for the viability of different formal minimization definitions. Overall, our analysis uncovers the complexities of the data minimization problem in the context of personalization and maps the remaining computational and regulatory challenges.

@inproceedings{daume20minimization,
title = {Operationalizing the Legal Principle of Data Minimization for
Personalization},
author = {Asia J. Biega and Peter Potash and Daum\'e, III, Hal and Fernando Diaz
and Mich\`ele Finck},
booktitle = {Proceedings of the Conference on Research and Developments in
Information Retrieval (SIGIR)},
year = {2020},
url = {http://hal3.name/docs/#daume20minimization},
}


More papers please!

Recent Talks:

(Meta-)Learning from Interaction
NYU Machine Learning Reading Group 2019
[PDF] [ODP]

Out of Order! Flexible neural language generation
NAACL 2019 NeuralGen Workshop
[PPTx]

Beyond demonstrations: Learning behavior from higher-level supervision
ICML 2019 I3 Workshop
[PPTX]

Imitation Learning
Vector Institute Reinforcement Learning Summer School 2018
[PDF] [ODP]

Learning language through interaction
December 2016, Georgetown, Amazon, USC, GATech, UW, ...
[PDF] [ODP] [Video]

Bias in AI
November 2016, UMD MCWIC Diversity Summit
[PDF] [ODP] [PPTx (exported)] [Blog Post]

More talks please!

Contact information:

    email: me AT hal3 DOT name               skype: haldaume3
    phone: 301-405-1073                    twitter: haldaume3
   office: IRB 4150                         github: hal3

I can't reply to all prospective students email; please read this before emailing me.

credits: design and font inspired by Seth Able's LoRD, some images converted to ANSI using ManyTools, original drawing of me by anonymous.
last updated on twenty april, two thousand twenty one.