Hal Daumé III - about me

You enter a dark forest. Standing in front of you is:

A professor named Hal Daumé III. He wields appointments in Computer Science and Language Science at UMD; he is currently on leave from UMD, spending time in the machine learning and fairness groups at Microsoft Research NYC. He and his wonderful advisees study questions related to how to get machines to becomes more adept at human language, by developing models and algorithms that allow them to learn from data. (Keywords: natural language processing and machine learning.) The two major questions that really drive their research these days are:

    (1) how can we get computers to learn language
        through natural interaction with people/users?

and (2) how can we do this in a way that promotes fairness,
        transparency and explainability in the learned models?

He's discussed interactive learning informally recently in a Talking Machines Podcost and more technically in recent talks; and has discussed fairness/bias in broad terms in a recent blog post. Hal is committed to promoting an inclusive scientific environment; if you are thinking of inviting him for a talk or to participate in an event, please ensure that the event is consistent with this goal (see the first question on the FAQ).

Hal is super fortunate to have awesome colleagues in the Computional Linguistics and Information Processing Lab (which he currently directs). He maintains the structured prediction framework in VW. If you want to contact him, email is your best bet; you can also find him on @haldaume3 on Twitter. Or, in person, in the CLIP lab (AVW 3126) or his office (AVW 3227). If you're a prospective grad student or grad applicant, please read his FAQ!

Recent Publications:

Hierarchical Imitation and Reinforcement Learning
Hoang M. Le, Nan Jiang, Alekh Agarwal, Miroslav Dudík, Yisong Yue and Hal Daumé III
International Conference on Machine Learning (ICML), 2018
[Abstract] [BibTeX]

We study the problem of learning policies over long time horizons. We present a framework that leverages and integrates two key concepts. First, we utilize hierarchical policy classes that enable planning over different time scales, i.e., the high level planner proposes a sequence of subgoals for the low level planner to achieve. Second, we utilize expert demonstrations within the hierarchical action space to dramatically reduce cost of exploration. Our framework is flexible and can incorporate different combinations of imitation learning (IL) and reinforcement learning (RL) at different levels of the hierarchy. Using long-horizon benchmarks, including Montezuma's Revenge, we empirically demonstrate that our approach can learn significantly faster compared to hierarchical RL, and can be significantly more label- and sample-efficient compared to flat IL. We also provide theoretical analysis of the labeling cost for certain instantiations of our framework.

@inproceedings{daume18ilrl, title = { Hierarchical Imitation and Reinforcement Learning}, author = {Hoang M. Le and Nan Jiang and Alekh Agarwal and Miroslav Dudík and Yisong Yue and me}, booktitle = {Proceedings of the International Conference on Machine Learning (ICML)}, year = {2018}, url = {http://hal3.name/docs/#daume18ilrl}, }

Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information
Sudha Rao and Hal Daumé III
Conference of the Association for Computational Linguistics (ACL), 2018
[Abstract] [BibTeX]

Inquiry is fundamental to communication, and machines cannot effectively collaborate with humans unless they can ask questions. In this work, we build a neural network model for the task of ranking clariﬁcation questions. Our model is inspired by the idea of expected value of perfect information: a good question is one whose expected answer will be useful. We study this problem using data from StackExchange, a plentiful online resource in which people routinely ask clarifying questions to posts so that they can better offer assistance to the original poster. We create a dataset of clariﬁcation questions consisting of ∼77K posts paired with a clariﬁcation question (and answer) from three domains of StackExchange: askubuntu, unix and superuser. We evaluate our model on 500 samples of this dataset against expert human judgments and demonstrate signiﬁcant improvements over controlled baselines.

@inproceedings{daume18clarification, title = {Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information}, author = {Sudha Rao and me}, booktitle = {Proceedings of the Conference of the Association for Computational Linguistics (ACL)}, year = {2018}, url = {http://hal3.name/docs/#daume18clarification}, }

Datasheets for Datasets
Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III and Kate Crawford
arxiv, 2018
[Abstract] [BibTeX]

Currently there is no standard way to identify how a dataset was created, and what characteristics, motivations, and potential skews it represents. To begin to address this issue, we propose the concept of a datasheet for datasets, a short document to accompany public datasets, commercial APIs, and pretrained models. The goal of this proposal is to enable better communication between dataset creators and users, and help the AI community move toward greater transparency and accountability. By analogy, in computer hardware, it has become industry standard to accompany everything from the simplest components (e.g., resistors), to the most complex microprocessor chips, with datasheets detailing standard operating characteristics, test results, recommended usage, and other information. We outline some of the questions a datasheet for datasets should answer. These questions focus on when, where, and how the training data was gathered, its recommended use cases, and, in the case of human-centric datasets, information regarding the subjects’ demographics and consent as applicable. We develop prototypes of datasheets for two well-known datasets: Labeled Faces in The Wild [33] and the Pang & Lee Polarity Dataset [45].

@inproceedings{daume18datasheets, title = {Datasheets for Datasets}, author = {Timnit Gebru and Jamie Morgenstern and Briana Vecchione and Jennifer Wortman Vaughan and Hanna Wallach and Hal {Daum\'e III} and Kate Crawford}, booktitle = {arxiv}, year = {2018}, url = {http://hal3.name/docs/#daume18datasheets}, }

Residual Loss Prediction: Reinforcement Learning with no Incremental Feedback
Hal Daumé III, John Langford and Amr Sharaf
ICLR, 2018
[Abstract] [BibTeX]

We consider reinforcement learning and bandit structured prediction problems with very sparse loss feedback: only at the end of an episode. We introduce a novel algorithm, R ESIDUAL L OSS P REDICTION (R ESLOPE), that solves such problems by automatically learning an internal representation of a denser reward function. R ESLOPE operates as a reduction to contextual bandits, using its learned loss representation to solve the credit assignment problem, and a contextual bandit oracle to trade-off exploration and exploitation. R ESLOPE enjoys a no-regret reductionstyle theoretical guarantee and outperforms state of the art reinforcement learning algorithms in both MDP environments and bandit structured prediction settings.

@inproceedings{daume18reslope, title = {Residual Loss Prediction: Reinforcement Learning with no Incremental Feedback}, author = {Hal {Daum\'e III} and John Langford and Amr Sharaf}, booktitle = {ICLR}, year = {2018}, url = {http://hal3.name/docs/#daume18reslope}, }

Hierarchical Imitation and Reinforcement Learning
Hoang M Le, Nan Jiang, Alekh Agarwal, Miroslav Dudík, Yisong Yue and Hal Daumé III
arxiv, 2018
[Abstract] [BibTeX]

We study the problem of learning policies over long time horizons. We present a framework that leverages and integrates two key concepts. First, we utilize hierarchical policy classes that enable planning over different time scales, i.e., the high level planner proposes a sequence of subgoals for the low level planner to achieve. Second, we utilize expert demonstrations within the hierarchical action space to dramatically reduce cost of exploration. Our framework is ﬂexible and can incorporate different combinations of imitation learning (IL) and reinforcement \ learning (RL) at different levels of the hierarchy. Using long-horizon benchmarks, including Montezuma’s Revenge, we empirically demonstrate that our approach can learn signiﬁcantly faster compared to hierarchical RL, and can be signiﬁcantly more labeland sample-efﬁcient compared to ﬂat IL. We also provide theoretical analysis of the labeling cost for certain instantiations of our framework.

@inproceedings{daume18hierilrl, title = {Hierarchical Imitation and Reinforcement Learning}, author = {Hoang M Le and Nan Jiang and Alekh Agarwal and Miroslav Dudík and Yisong Yue and Hal {Daum\'e III}}, booktitle = {arxiv}, year = {2018}, url = {http://hal3.name/docs/#daume18hierilrl}, }

More papers please!

Recent Talks:

Learning language through interaction
December 2016, Georgetown, Amazon, USC, GATech
[PDF] [ODP]

Bias in AI
November 2016, UMD MCWIC Diversity Summit
[PDF] [ODP] [PPTx (exported)] [Blog Post]

Locally optimal learning to search and distant supervision
December 2015, UMD CS Research Seminar
[PDF] [ODP] [Video]

Imitation learning and recurrent neural networks mashup
December 2015, CIFAR NCAP Workshop
[PDF] [ODP]

Algorithms that learn to think of their feet
October 2015, UC Santa Cruz
[PDF] [ODP]

More talks please!

Contact information:

    email: me AT hal3 DOT name               skype: haldaume3
    phone: 301-405-1073                    twitter: haldaume3
   office: AVW 3227                         github: hal3

I can't reply to all prospective students email; please read this before emailing me.

credits: design and font inspired by Seth Able's LoRD, some images converted to ANSI using ManyTools, original drawing of me by anonymous.
last updated on eightteen june, two thousand eightteen.