You enter a dark forest. Standing in front of you is:

A professor named Hal Daumé III. He wields appointments in Computer Science and Language Science at UMD (in Fall 2019 he is teaching Computational Linguistics I); he also spends time in the machine learning and fairness groups at Microsoft Research NYC. He and his wonderful advisees study questions related to how to get machines to becomes more adept at human language, by developing models and algorithms that allow them to learn from data. (Keywords: natural language processing and machine learning.) The two major questions that really drive their research these days are:

    (1) how can we get computers to learn language
        through natural interaction with people/users?

and (2) how can we do this in a way that promotes fairness,
        transparency and explainability in the learned models?

He's discussed interactive learning informally recently in a Talking Machines Podcost and more technically in recent talks; and has discussed fairness/bias in broad terms in a recent blog post. Hal is committed to promoting an inclusive scientific environment; if you are thinking of inviting him for a talk or to participate in an event, please ensure that the event is consistent with this goal (see the first question on the FAQ).

Hal is super fortunate to have awesome colleagues in the Computional Linguistics and Information Processing Lab (which he currently directs). He maintains the structured prediction framework in VW. If you want to contact him, email is your best bet; you can also find him on @haldaume3 on Twitter. Or, in person, in the CLIP lab (AVW 3126) or his office (AVW 3227). If you're a prospective grad student or grad applicant, please read his FAQ!

Recent Publications:

Reinforcement Learning with Convex Constraints
Sobhan Miryoosefi, Kianté Brantley, Hal Daumé III, Miroslav Dudik and Robert Schapire
NeurIPS, 2019
[Abstract] [BibTeX]

In standard reinforcement learning (RL), a learning agent seeks to optimize the overall reward. However, many key aspects of a desired behavior are more naturally expressed as constraints. For instance, the designer may want to limit the use of unsafe actions, increase the diversity of trajectories to enable exploration, or approximate expert trajectories when rewards are sparse. In this paper, we propose an algorithmic scheme that can handle a wide class of constraints in RL tasks, specifically, any constraints that require expected values of some vector measurements (such as the use of an action) to lie in a convex set. This captures previously studied constraints (such as safety and proximity to an expert), but also enables new classes of constraints (such as diversity). Our approach comes with rigorous theoretical guarantees and only relies on the ability to approximately solve standard RL tasks. As a result, it can be easily adapted to work with any model-free or model-based RL algorithm. In our experiments, we show that it matches previous algorithms that enforce safety via constraints, but can also enforce new properties that these algorithms cannot incorporate, such as diversity.

@inproceedings{daume19convexrl,
title = {Reinforcement Learning with Convex Constraints},
author = {Sobhan Miryoosefi and Kiant\'e Brantley and Hal {Daum\'e III} and
Miroslav Dudik and Robert Schapire},
booktitle = {NeurIPS},
year = {2019},
url = {http://hal3.name/docs/#daume19convexrl},
}


Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning
Khanh Nguyen and Hal Daumé III
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2019
[Abstract] [BibTeX]

Mobile agents that can leverage help from humans can potentially accomplish more complex tasks than they could entirely on their own. We develop "Help, Anna!" (HANNA), an interactive photo-realistic simulator in which an agent fulfills object-finding tasks by requesting and interpreting natural languageand-vision assistance. An agent solving tasks in a HANNA environment can leverage simulated human assistants, called ANNA (Automatic Natural Navigation Assistants), which, upon request, provide natural language and visual instructions to direct the agent towards the goals. To address the HANNA problem, we develop a memory-augmented neural agent that hierarchically models multiple levels of decision-making, and an imitation learning algorithm that teaches the agent to avoid repeating past mistakes while simultaneously predicting its own chances of making future progress. Empirically, our approach is able to ask for help more effectively than competitive baselines and, thus, attains higher task success rate on both previously seen and previously unseen environments. We publicly release code and data at this https URL .

@inproceedings{daume19hanna,
title = {Help, Anna! Visual Navigation with Natural Multimodal Assistance via
Retrospective Curiosity-Encouraging Imitation Learning},
author = {Khanh Nguyen and me},
booktitle = {Proceedings of the Conference on Empirical Methods in Natural
Language Processing (EMNLP)},
year = {2019},
url = {http://hal3.name/docs/#daume19hanna},
}


Comparing and Developing Tools to Measure the Readability of Domain-Specific Texts
Elissa Redmiles, Lisa Maszkiewicz, Emily Hwang, Dhruv Kuchhal, Everst Liu, Miraida Morales, Denis Peskov, Sudha Rao, Rock Stevens, Kristina Gligori\'c, Sean Kross, Michelle L. Mazurek and Hal Daumé III
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2019
[Abstract] [BibTeX]

T h e r e a d a bilit y of a di git al t e xt c a n i n fl ue n c e p e o pl e’s a bilit y t o l e ar n n e w t hi n g s a b o ut a r a n g e t o pi c s fr o m di git al r e s o ur c e s ( e. g., Wi ki p e di a, We b M D). R e a d a bilit y al s o i m p a ct s s e ar c h r a n ki n g s, a n d i s u s e d t o e v al u at e t h e p erf or m a n c e of N L P s y st e m s. D e s pit e t hi s, w e l a c k a t h or o u g h u n d er st a n di n g of h o w t o v ali dl y m e a s ur e r e a d a bilit y at s c al e, e s p e ci all y f or d o m ai n- s p e ci fi c t e xt s. I n t hi s w or k, w e pr e s e nt a c o m p ari s o n of t h e v ali dit y of w ell- k n o w n r e a d a bilit y m e as ur e s a n d i ntr o d u c e a n o v el a p pr o a c h, S m a rt Cl o z e , w hi c h i s d e si g n e d t o a d dr e s s s h ortc o mi n g s of e xi sti n g m e a s ur e s. We c o m p ar e t h e s e a p pr o a c h e s a cr o s s f o ur diff er e nt c or p or a: cr o w d w or k er- g e n er at e d st ori e s, Wi ki p e di a arti cl e s, s e c urit y a n d pri v a c y a d vi c e, a n d h e alt h i nf or m ati o n. O n t h e s e c or p or a, w e e v al uat e t h e c o n v er g e nt a n d c o nt e nt v ali dit y of e a c h m e a s ur e, a n d d et ail tr a d e off s i n s c or e pr e ci si o n, d o m ai n- s p e ci fi cit y, a n d p arti ci p a nt b ur d e n. T h e s e r e s ult s pr o vi d e a f o u n d ati o n f or m or e a c c ur at e r e a d a bilit y m e a s ur e m e nt s a n d b ett er e v al u ati o n of n e w n at ur al-l a n g u a g epr o c e s si n g s y st e m s a n d t o ol s.

@inproceedings{daume19readability,
title = {Comparing and Developing Tools to Measure the Readability of
Domain-Specific Texts},
author = {Elissa Redmiles and Lisa Maszkiewicz and Emily Hwang and Dhruv Kuchhal
and Everst Liu and Miraida Morales and Denis Peskov and Sudha Rao
and Rock Stevens and Kristina Gligori\'c and Sean Kross and
Michelle L. Mazurek and me},
booktitle = {Proceedings of the Conference on Empirical Methods in Natural
Language Processing (EMNLP)},
year = {2019},
url = {http://hal3.name/docs/#daume19readability},
}


Interpretable Engagement Models for MOOCs using Hinge-loss Markov Random Fields
Arti Ramesh, Dan Goldwasser, Bert Huang, Hal Daumé III and Lise Getoor
IEEE Transations on Learning Technologies, 2019
[Abstract] [BibTeX]

Maintaining and cultivating student engagement is critical for learning. Understanding factors affecting student engagement can help in designing better courses and improving student retention. The large number of participants in massive open online courses (MOOCs) and data collected from their interactions on the MOOC open up avenues for studying student engagement at scale. In this work, we develop an interpretable statistical relational learning model for understanding student engagement in online courses using a complex combination of behavioral, linguistic, structural, and temporal cues. We show how to abstract student engagement types of active, passive, and disengagement as meaningful latent variables using logical rules in our model connecting student behavioral signals with student success in MOOCs. We demonstrate that the latent formulation for engagement helps in predicting two measures of student success: performance, their final grade in the course, and survival, their continued presence in the course till the end, across seven MOOCs. Further, in order to initiate better instructor interventions, we need to be able to predict student success early in the course. We demonstrate that we can predict student success early in the course reliably using the latent model. We also demonstrate the utility of our models in predicting student success in new courses, by training our models on one course and testing on another course. We show that the latent abstractions are helpful in predicting student success and engagement reliably in new MOOCs that haven’t yet gathered student interaction data. We then perform a closer quantitative analysis of different features derived from student interactions on the MOOC and identify student activities that are good indicators of student success at different points in the course. Through a qualitative analysis of the latent engagement variable values, we demonstrate their utility in understanding students’ engagement levels at various points in the course and movement of students across different types of engagement.

@article{daume19moocs,
title = {Interpretable Engagement Models for {MOOC}s using Hinge-loss Markov
Random Fields},
author = {Arti Ramesh and Dan Goldwasser and Bert Huang and Hal {Daum\'e III} and
Lise Getoor},
journal = {IEEE Transations on Learning Technologies},
year = {2019},
url = {http://hal3.name/docs/#daume19moocs},
}


Meta-Learning for Contextual Bandit Exploration
Amr Sharaf and Hal Daumé III
arxiv, 2019
[Abstract] [BibTeX]

We describe MELEE, a meta-learning algorithm for learning a good exploration policy in the interactive contextual bandit setting. Here, an algorithm must take actions based on contexts, and learn based only on a reward signal from the action taken, thereby generating an exploration/exploitation trade-off. MELEE addresses this trade-off by learning a good exploration strategy for offline tasks based on synthetic data, on which it can simulate the contextual bandit setting. Based on these simulations, MELEE uses an imitation learning strategy to learn a good exploration policy that can then be applied to true contextual bandit tasks at test time. We compare MELEE to seven strong baseline contextual bandit algorithms on a set of three hundred real-world datasets, on which it outperforms alternatives in most settings, especially when differences in rewards are large. Finally, we demonstrate the importance of having a rich feature representation for learning how to explore.

@inproceedings{daume19melee,
title = {Meta-Learning for Contextual Bandit Exploration},
author = {Amr Sharaf and Hal {Daum\'e III}},
booktitle = {arxiv},
year = {2019},
url = {http://hal3.name/docs/#daume19melee},
}


More papers please!

Recent Talks:

Learning language through interaction
December 2016, Georgetown, Amazon, USC, GATech
[PDF] [ODP]

Bias in AI
November 2016, UMD MCWIC Diversity Summit
[PDF] [ODP] [PPTx (exported)] [Blog Post]

Locally optimal learning to search and distant supervision
December 2015, UMD CS Research Seminar
[PDF] [ODP] [Video]

Imitation learning and recurrent neural networks mashup
December 2015, CIFAR NCAP Workshop
[PDF] [ODP]

Algorithms that learn to think of their feet
October 2015, UC Santa Cruz
[PDF] [ODP]

More talks please!

Contact information:

    email: me AT hal3 DOT name               skype: haldaume3
    phone: 301-405-1073                    twitter: haldaume3
   office: AVW 3227                         github: hal3

I can't reply to all prospective students email; please read this before emailing me.

credits: design and font inspired by Seth Able's LoRD, some images converted to ANSI using ManyTools, original drawing of me by anonymous.
last updated on fifteen september, two thousand nineteen.