Bayesian Thinking on Your Feet: Embedding Generative Models in Reinforcement Learning for Sequentially Revealed Data

Project funded by the National Science Foundation (IIS-1320538)
PI: Jordan Boyd-Graber, Co-PI: Hal Daumé III, University of Maryland

The goal of this project is to create algorithms that can “think on their feet”, i.e. to incrementally process input and to decide when enough information has been received to act on those data. This research requires innovation in two areas: content models (to make accurate predictions, even when not all available information is available) and policies (to know when to trust the outputs of the content models---or know they won't get better---versus waiting for more information).

We are applying these models to two problems: synchronous machine translation (or "machine simultaneous interpretation") and question answering (when questions are revealed one piece at a time).

Synchronous machine translation is when a sentence is being produced one word at a time in a foreign language and we want to produce a translation in English simultaneously (i.e., with as little delay between a foreign language word and its English translation). In the machine translation setting, the content model predicts what words are going to appear in the input stream, even though they might not have been seen yet. This is particularly important in verb-final languages like German or Japanese, where an English translation can barely begin until the verb is seen. For the simultaneous translation problem, our content model must predict unseen elements of the sentence (e.g., the main verb in German and Japanese, or relative clauses in Japanese, or post-positions in Japanese). The job of the policy is to decide when to trust the content prediction. It must learn to balance incorrect translation versus timely translations, and must use those predictions to translate the sentence.

For question answering, we use a specially designed dataset that challenges humans: a trivia game called quiz bowl. These questions are written so that they can be interrupted by someone who knows more about the answer; that is, harder clues are at the start of the question and easier clues are at the end of the question. The content model produces guesses of what the answer could be and the policy must decide when to accept the guess.

Quiz bowl is a fun game with excellent opportunities for outreach, but it is also related to core challenges in natural language processing: classification (sorting inputs and making predictions), discourse (using pragmatic clues to guess what will come next), and coreference resolution (knowing which entities are discussed from oblique mentions).

<< back to top

	Jordan Boyd-Graber Assistant Professor, Computer Science (Maryland)
	Hal Daumé III Associate Professor, Computer Science (Maryland)
	Danny Bouman Undergraduate student, Computer Science (BS 2014, Maryland)
	Leonardo Claudino Ph.D. student, Computer Science (Maryland)
	Shi Feng Ph.D. student, Computer Science (Maryland)
	Anupam Guha Ph.D. student, Computer Science (Maryland)
	Alvin Grissom II Ph.D. student, Computer Science (Colorado)
	He He Ph.D. student, Computer Science (Maryland)
	Stephanie Hwa Undergraduate student, Computer Science (BS 2014, Maryland)
	Mohit Iyyer Ph.D. student, Computer Science (Maryland)
	John Morgan MS student, Computer Science (Maryland)
	Khanh Nguyen Ph.D. student, Computer Science (Maryland)
	Pedro Rodriguez Ph.D. student, Computer Science (Colorado)
	Davis Yoshida MS student, Computer Science (BS 2016, MS 2017, Colorado)

<< back to top

• Mohit Iyyer, Varun Manjunatha, Anupam Guha, Yogarshi Vyas, Jordan Boyd-Graber, Hal Daumé III, and Larry Davis. The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives. Computer Vision and Pattern Recognition, 2017. [Code/Data] [Bibtex]
  
  

  @inproceedings{Iyyer:Manjunatha:Guha:Vyas:Boyd-Graber:Daume-III:Davis-2017,
  	Title = {The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives},
  	Author = {Mohit Iyyer and Varun Manjunatha and Anupam Guha and Yogarshi Vyas and Jordan Boyd-Graber and Hal {Daum\'{e} III} and Larry Davis},
  	Booktitle = {Computer Vision and Pattern Recognition},
  	Year = {2017},
  	Url = {http://cs.umd.edu/~jbg//docs/2017_cvpr_comics.pdf},
  }
  

• Mohit Iyyer, Varun Manjunatha, Anupam Guha, Yogarshi Vyas, Jordan Boyd-Graber, Hal Daumé III, and Larry Davis. The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives. Computer Vision and Pattern Recognition, 2017. [Code/Data] [Bibtex]
  
  

  @inproceedings{Iyyer:Manjunatha:Guha:Vyas:Boyd-Graber:Daume-III:Davis-2017,
  	Title = {The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives},
  	Author = {Mohit Iyyer and Varun Manjunatha and Anupam Guha and Yogarshi Vyas and Jordan Boyd-Graber and Hal {Daum\'{e} III} and Larry Davis},
  	Booktitle = {Computer Vision and Pattern Recognition},
  	Year = {2017},
  	Url = {http://cs.umd.edu/~jbg//docs/2017_cvpr_comics.pdf},
  }
  

• Khanh Nguyen, Jordan Boyd-Graber, and Hal Daumé III. Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback. Empirical Methods in Natural Language Processing, 2017. [Code] [Bibtex]
  
  

  @inproceedings{Nguyen:Boyd-Graber:Daume-III-2017,
  	Title = {Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback},
  	Author = {Khanh Nguyen and Jordan Boyd-Graber and Hal {Daum\'{e} III}},
  	Booktitle = {Empirical Methods in Natural Language Processing},
  	Year = {2017},
  	Location = {Copenhagen, Denmark},
  	Url = {http://cs.umd.edu/~jbg//docs/2017_emnlp_bandit_mt.pdf},
  }
  

• Alvin Grissom II, Naho Orita, and Jordan Boyd-Graber. Incremental Prediction of Sentence-final Verbs. Conference on Computational Natural Language Learning, 2016. [Bibtex]
  
  

  @inproceedings{Grissom-II:Orita:Boyd-Graber-2016,
  	Title = {Incremental Prediction of Sentence-final Verbs},
  	Author = {Alvin {Grissom II} and Naho Orita and Jordan Boyd-Graber},
  	Booktitle = {Conference on Computational Natural Language Learning},
  	Year = {2016},
  	Location = {Berlin, Germany},
  	Url = {http://cs.umd.edu/~jbg//docs/2016_conll_verbpred.pdf},
  }
  

  
  
• He He, Jordan Boyd-Graber, Kevin Kwok, and Hal Daumé III. Opponent Modeling in Deep Reinforcement Learning. International Conference on Machine Learning, 2016. [Video] [Bibtex]
  
  

  @inproceedings{He:Boyd-Graber:Kwok:Daume-III-2016,
  	Title = {Opponent Modeling in Deep Reinforcement Learning},
  	Author = {He He and Jordan Boyd-Graber and Kevin Kwok and Hal {Daum\'{e} III}},
  	Booktitle = {International Conference on Machine Learning},
  	Year = {2016},
  	Location = {New York, NY},
  	Url = {http://cs.umd.edu/~jbg//docs/2016_icml_opponent.pdf},
  }
  

• Anupam Guha, Mohit Iyyer, and Jordan Boyd-Graber. A Distorted Skull Lies in the Bottom Center: Identifying Paintings from Text Descriptions. NAACL Human-Computer Question Answering Workshop, 2016. [Data] [Bibtex]
  
  

  @inproceedings{Guha:Iyyer:Boyd-Graber-2016,
  	Title = {A Distorted Skull Lies in the Bottom Center: Identifying Paintings from Text Descriptions},
  	Author = {Anupam Guha and Mohit Iyyer and Jordan Boyd-Graber},
  	Booktitle = {NAACL Human-Computer Question Answering Workshop},
  	Year = {2016},
  	Location = {San Diego, CA},
  	Url = {http://cs.umd.edu/~jbg//docs/2016_naacl_paintings.pdf},
  }
  

• Mohit Iyyer, Anupam Guha, Snigdha Chaturvedi, Jordan Boyd-Graber, and Hal Daumé III. Feuding Families and Former Friends: Unsupervised Learning for Dynamic Fictional Relationships. North American Association for Computational Linguistics, 2016. [Code/Data] [Bibtex]
  
  

  @inproceedings{Iyyer:Guha:Chaturvedi:Boyd-Graber:Daume-III-2016,
  	Title = {Feuding Families and Former Friends: Unsupervised Learning for Dynamic Fictional Relationships},
  	Author = {Mohit Iyyer and Anupam Guha and Snigdha Chaturvedi and Jordan Boyd-Graber and Hal {Daum\'{e} III}},
  	Booktitle = {North American Association for Computational Linguistics},
  	Year = {2016},
  	Location = {San Diego, CA},
  	Url = {http://cs.umd.edu/~jbg//docs/2016_naacl_relationships.pdf},
  }
  

  Best paper award (2 out of 1592)
  
• He He, Jordan Boyd-Graber, and Hal Daumé III. Interpretese vs. Translationese: The Uniqueness of Human Strategies in Simultaneous Interpretation. North American Association for Computational Linguistics, 2016. [Talk] [Bibtex]
  
  

  @inproceedings{He:Boyd-Graber:Daume-III-2016,
  	Title = {Interpretese vs. Translationese: The Uniqueness of Human Strategies in Simultaneous Interpretation},
  	Author = {He He and Jordan Boyd-Graber and Hal {Daum\'{e} III}},
  	Booktitle = {North American Association for Computational Linguistics},
  	Year = {2016},
  	Location = {San Diego, CA},
  	Url = {http://cs.umd.edu/~jbg//docs/2016_naacl_interpretese.pdf},
  }
  

  
  
• Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. Deep Unordered Composition Rivals Syntactic Methods for Text Classification. Association for Computational Linguistics, 2015. [Slides] [Code] [Talk] [Bibtex]
  
  

  @inproceedings{Iyyer:Manjunatha:Boyd-Graber:Daume-III-2015,
  	Title = {Deep Unordered Composition Rivals Syntactic Methods for Text Classification},
  	Author = {Mohit Iyyer and Varun Manjunatha and Jordan Boyd-Graber and Hal {Daum\'{e} III}},
  	Booktitle = {Association for Computational Linguistics},
  	Year = {2015},
  	Location = {Beijing, China},
  	Url = {http://cs.umd.edu/~jbg//docs/2015_acl_dan.pdf},
  }
  

  
  
• Vlad Niculae, Srijan Kumar, Jordan Boyd-Graber, and Cristian Danescu-Niculescu-Mizil. Linguistic Harbingers of Betrayal: A Case Study on an Online Strategy Game. Association for Computational Linguistics, 2015. [Code/Data] [Bibtex]
  
  

  @inproceedings{Niculae:Kumar:Boyd-Graber:Danescu-Niculescu-Mizil-2015,
  	Title = {Linguistic Harbingers of Betrayal: A Case Study on an Online Strategy Game},
  	Author = {Vlad Niculae and Srijan Kumar and Jordan Boyd-Graber and Cristian Danescu-Niculescu-Mizil},
  	Booktitle = {Association for Computational Linguistics},
  	Year = {2015},
  	Location = {Beijing, China},
  	Url = {http://cs.umd.edu/~jbg//docs/2015_acl_diplomacy.pdf},
  }
  

  
  

  Accessible Abstract: This paper introduces the application of natural language processing techniques to understand the relationships (and their dissolution) in the game of Diplomacy. This popular board game simulates Europe at the eve of World War I and forces players to work with each other to forge alliances and make plans together. However, the game's setup also encourages players to turn against each other. This paper analyzes whether we can predict these betrayals (we can!) and the linguistic and social phenomena (demands, politeness, and planning) that can predict when a betrayal will happen.

• He He, Alvin Grissom II, Jordan Boyd-Graber, and Hal Daumé III. Syntax-based Rewriting for Simultaneous Machine Translation. Empirical Methods in Natural Language Processing, 2015. [Bibtex]
  
  

  @inproceedings{He:Grissom-II:Boyd-Graber:Daume-III-2015,
  	Title = {Syntax-based Rewriting for Simultaneous Machine Translation},
  	Author = {He He and Alvin {Grissom II} and Jordan Boyd-Graber and Hal {Daum\'{e} III}},
  	Booktitle = {Empirical Methods in Natural Language Processing},
  	Year = {2015},
  	Location = {Lisbon, Portugal},
  	Url = {http://cs.umd.edu/~jbg//docs/2015_emnlp_rewrite.pdf},
  }
  

  
  
• Jordan Boyd-Graber, Mohit Iyyer, He He, and Hal Daumé III. Interactive Incremental Question Answering. Neural Information Processing Systems, 2015. [Bibtex]
  
  

  @inproceedings{Boyd-Graber:Iyyer:He:Daume-III-2015,
  	Title = {Interactive Incremental Question Answering},
  	Author = {Jordan Boyd-Graber and Mohit Iyyer and He He and Hal {Daum\'{e} III}},
  	Booktitle = {Neural Information Processing Systems},
  	Year = {2015},
  	Location = {Montreal, Canada},
  }
  

  This won the best demonstration award at NIPS 2015
• Anupam Guha, Mohit Iyyer, Danny Bouman, and Jordan Boyd-Graber. Removing the Training Wheels: A Coreference Dataset that Entertains Humans and Challenges Computers. North American Association for Computational Linguistics, 2015. [Code/Data] [Slides] [Video] [LaTeX] [Bibtex]
  
  

  @inproceedings{Guha:Iyyer:Bouman:Boyd-Graber-2015,
  	Title = {Removing the Training Wheels: A Coreference Dataset that Entertains Humans and Challenges Computers},
  	Author = {Anupam Guha and Mohit Iyyer and Danny Bouman and Jordan Boyd-Graber},
  	Booktitle = {North American Association for Computational Linguistics},
  	Year = {2015},
  	Location = {Denver, Colorado},
  	Url = {http://cs.umd.edu/~jbg//docs/2015_naacl_qb_coref.pdf},
  }
  

  
  
• Mohit Iyyer, Jordan Boyd-Graber, Leonardo Claudino, Richard Socher, and Hal Daumé III. A Neural Network for Factoid Question Answering over Paragraphs. Empirical Methods in Natural Language Processing, 2014. [Code/Data] [Bibtex]
  
  

  @inproceedings{Iyyer:Boyd-Graber:Claudino:Socher:Daume-III-2014,
  	Title = {A Neural Network for Factoid Question Answering over Paragraphs},
  	Author = {Mohit Iyyer and Jordan Boyd-Graber and Leonardo Claudino and Richard Socher and Hal {Daum\'{e} III}},
  	Booktitle = {Empirical Methods in Natural Language Processing},
  	Year = {2014},
  	Location = {Doha, Qatar},
  	Url = {http://cs.umd.edu/~jbg//docs/2014_emnlp_qb_rnn.pdf},
  }
  

  The partial derivatives of "C" and "J" with respect to the parameters should be switched in Equation 7.
• Alvin Grissom II, He He, Jordan Boyd-Graber, John Morgan, and Hal Daumé III. Don't Until the Final Verb Wait: Reinforcement Learning for Simultaneous Machine Translation. Empirical Methods in Natural Language Processing, 2014. [Talk] [Bibtex]
  
  

  @inproceedings{Grissom-II:He:Boyd-Graber:Morgan:Daume-III-2014,
  	Title = {Don't Until the Final Verb Wait: Reinforcement Learning for Simultaneous Machine Translation},
  	Author = {Alvin {Grissom II} and He He and Jordan Boyd-Graber and John Morgan and Hal {Daum\'{e} III}},
  	Booktitle = {Empirical Methods in Natural Language Processing},
  	Year = {2014},
  	Location = {Doha, Qatar},
  	Url = {http://cs.umd.edu/~jbg//docs/2014_emnlp_simtrans.pdf},
  }
  

  
  
• Mohit Iyyer, Jordan Boyd-Graber, and Hal Daumé III. Generating Sentences from Semantic Vector Space Representations. NIPS Workshop on Learning Semantics, 2014. [Bibtex]
  
  

  @inproceedings{Iyyer:Boyd-Graber:Daume-III-2014,
  	Title = {Generating Sentences from Semantic Vector Space Representations},
  	Author = {Mohit Iyyer and Jordan Boyd-Graber and Hal {Daum\'{e} III}},
  	Booktitle = {NIPS Workshop on Learning Semantics},
  	Year = {2014},
  	Location = {Montreal, Canada},
  }
  

• Jordan Boyd-Graber, Brianna Satinoff, He He, and Hal Daumé III. Besting the Quiz Master: Crowdsourcing Incremental Classification Games. Empirical Methods in Natural Language Processing, 2012. [Presentation] [Data] [Bibtex]
  
  

  @inproceedings{Boyd-Graber:Satinoff:He:Daume-III-2012,
  	Title = {Besting the Quiz Master: Crowdsourcing Incremental Classification Games},
  	Author = {Jordan Boyd-Graber and Brianna Satinoff and He He and Hal {Daum\'{e} III}},
  	Booktitle = {Empirical Methods in Natural Language Processing},
  	Year = {2012},
  	Url = {http://cs.umd.edu/~jbg//docs/qb_emnlp_2012.pdf},
  	Location = {Jeju, South Korea},
  }
  

  
  

• Interface for crowdsourced coreference annotation
• Deep averaging networks
• Question Answering System
• Relationship Models/Data

• Coreference Annotations
• Quiz bowl questions and (human) answers

• Jordan Boyd-Graber, Pedro Rodriguez, Nathan Murphy, and R. Hentzel. QANTA vs. Quiz Bowl Veterans. 2017. [Bibtex]
  
  

  @online{Boyd-Graber:Rodriguez:Murphy:Hentzel-2017,
  	Author = {Jordan Boyd-Graber and Pedro Rodriguez and Nathan Murphy and R. Hentzel},
  	Year = {2017},
  	Title = {QANTA vs. Quiz Bowl Veterans},
  	Location = {Atlanta, GA},
  	Url = {https://www.youtube.com/watch?v=bYFqMINXayc},
  }
  

  
  
• Jordan Boyd-Graber, Pedro Rodriguez, Nathan Murphy, and R. Hentzel. QANTA vs. Quiz Bowl Veterans. 2017. [Bibtex]
  
  

  @online{Boyd-Graber:Rodriguez:Murphy:Hentzel-2017,
  	Author = {Jordan Boyd-Graber and Pedro Rodriguez and Nathan Murphy and R. Hentzel},
  	Year = {2017},
  	Title = {QANTA vs. Quiz Bowl Veterans},
  	Location = {Atlanta, GA},
  	Url = {https://www.youtube.com/watch?v=bYFqMINXayc},
  }
  

  
  
• Melissa Brachfeld. UMD Computerized System Beats Human Quiz Bowl Team at Atlanta Exhibition. UMIACS, 2017. [Bibtex]
  
  

  @online{Brachfeld-2017,
  	Author = {Melissa Brachfeld},
  	Journal = {UMIACS},
  	Year = {2017},
  	Title = {UMD Computerized System Beats Human Quiz Bowl Team at Atlanta Exhibition},
  	Url = {https://www.umiacs.umd.edu/about-us/news/umd-computerized-system-beats-human-quiz-bowl-team-atlanta-exhibition},
  }
  

• Ikuya Yamada, Jordan Boyd-Graber, and Mohit Iyyer. OUSIA vs. California NASAT Team. 2016. [Bibtex]
  
  

  @online{Yamada:Boyd-Graber:Iyyer-2016,
  	Author = {Ikuya Yamada and Jordan Boyd-Graber and Mohit Iyyer},
  	Year = {2016},
  	Title = {OUSIA vs. California NASAT Team},
  	Location = {San Diego, CA},
  	Url = {https://www.youtube.com/watch?v=bQHo7BApgAU&index=2&list=PLegWUnz91WfscfMYZvuSWEweBbIpfFDlF},
  }
  

  
  
• Ikuya Yamada, Jordan Boyd-Graber, and Mohit Iyyer. OUSIA vs. California NASAT Team. 2016. [Bibtex]
  
  

  @online{Yamada:Boyd-Graber:Iyyer-2016,
  	Author = {Ikuya Yamada and Jordan Boyd-Graber and Mohit Iyyer},
  	Year = {2016},
  	Title = {OUSIA vs. California NASAT Team},
  	Location = {San Diego, CA},
  	Url = {https://www.youtube.com/watch?v=bQHo7BApgAU&index=2&list=PLegWUnz91WfscfMYZvuSWEweBbIpfFDlF},
  }
  

  
  
• Jordan Boyd-Graber, Nathan Murphy, and R. Hentzel. QANTA vs. Quiz Bowl Veterans. 2016. [Bibtex]
  
  

  @online{Boyd-Graber:Murphy:Hentzel-2016,
  	Author = {Jordan Boyd-Graber and Nathan Murphy and R. Hentzel},
  	Year = {2016},
  	Title = {QANTA vs. Quiz Bowl Veterans},
  	Location = {Dallas, TX},
  	Url = {https://www.youtube.com/c2kGD1EdfFw},
  }
  

  
  
• Inderjeet Mani. When robots read books. Aeon, 2016. [Bibtex]
  
  

  @article{Mani-2016,
  	Author = {Inderjeet Mani},
  	Year = {2016},
  	Title = {When robots read books},
  	Journal = {Aeon},
  	Url = {https://aeon.co/essays/how-ai-is-revolutionising-the-role-of-the-literary-critic},
  }
  

• Inderjeet Mani. When robots read books. Aeon, 2016. [Bibtex]
  
  

  @article{Mani-2016,
  	Author = {Inderjeet Mani},
  	Year = {2016},
  	Title = {When robots read books},
  	Journal = {Aeon},
  	Url = {https://aeon.co/essays/how-ai-is-revolutionising-the-role-of-the-literary-critic},
  }
  

• Luke Dormehl. Like parents from the 1950s, AI still can't understand comics. Here's why. Digital Trends, 2016. [Bibtex]
  
  

  @article{Dormehl-2016,
  	Author = {Luke Dormehl},
  	Year = {2016},
  	Title = {Like parents from the 1950s, AI still can't understand comics. Here's why},
  	Journal = {Digital Trends},
  	Url = {http://www.digitaltrends.com/cool-tech/comic-book-ai/},
  }
  

• Katherine Gorman. Automatic Translation and t-SNE. Talking Machines, 2016. [Bibtex]
  
  

  @article{Gorman-2016,
  	Author = {Katherine Gorman},
  	Year = {2016},
  	Title = {Automatic Translation and t-SNE},
  	Journal = {Talking Machines},
  	Url = {http://www.thetalkingmachines.com/episodes/automatic-translation-and-t-sne},
  }
  

• Christopher Chabris. Game On: The Psychology of Betrayal. Wall Street Journal, 2016. [Bibtex]
  
  

  @article{Chabris-2016,
  	Author = {Christopher Chabris},
  	Year = {2016},
  	Title = {Game On: The Psychology of Betrayal},
  	Journal = {Wall Street Journal},
  	Url = {http://www.wsj.com/articles/when-diplomacy-leads-to-betrayal-1455897078},
  }
  

• Jordan Boyd-Graber, Mohit Iyyer, and R. Hentzel. QANTA vs. Jeopardy Champions. 2015. [Bibtex]
  
  

  @online{Boyd-Graber:Iyyer:Hentzel-2015,
  	Author = {Jordan Boyd-Graber and Mohit Iyyer and R. Hentzel},
  	Year = {2015},
  	Title = {QANTA vs. Jeopardy Champions},
  	Location = {Chicago, IL},
  	Url = {https://www.youtube.com/watch?v=LqsUaprYMOw},
  }
  

  
  
• Jordan Boyd-Graber, Ken Jennings, and Noah Smith. QANTA vs. Ken Jennings. 2015. [Bibtex]
  
  

  @online{Boyd-Graber:Jennings:Smith-2015,
  	Author = {Jordan Boyd-Graber and Ken Jennings and Noah Smith},
  	Year = {2015},
  	Title = {QANTA vs. Ken Jennings},
  	Location = {Seattle, WA},
  	Url = {https://www.youtube.com/kTXJCEvCDYk},
  }
  

  
  
• Jordan Boyd-Graber. Thinking on your Feet: Reinforcement Learning for Incremental Language Tasks. 2015. [Bibtex]
  
  

  @online{Boyd-Graber-2015,
  	Author = {Jordan Boyd-Graber},
  	Year = {2015},
  	Title = {Thinking on your Feet: Reinforcement Learning for Incremental Language Tasks},
  	Location = {Yandex Berlin},
  	Url = {https://www.youtube.com/watch?v=YArUk9QcMe0},
  }
  

  
  
• Melissa Dahl. Should you worry about people who are too polite?. CNN, 2015. [Bibtex]
  
  

  @article{Dahl-2015,
  	Author = {Melissa Dahl},
  	Year = {2015},
  	Title = {Should you worry about people who are too polite?},
  	Journal = {CNN},
  	Url = {http://www.cnn.com/2015/08/31/health/beware-polite-people/},
  }
  

• Emily Adams. Professor's Quiz Bowl Robot Goes Head-To-Head With Humans. Colorado Computer Science Newsletter, 2015. [Bibtex]
  
  

  @article{Adams-2015,
  	Author = {Emily Adams},
  	Year = {2015},
  	Title = {Professor's Quiz Bowl Robot Goes Head-To-Head With Humans},
  	Journal = {Colorado Computer Science Newsletter},
  	Url = {http://www.diamondbackonline.com/news/umd-researchers-computer-beats-jeopardy-star-ken-jennings-at-trivia/article_133f05f4-75d9-11e5-9c00-8718fca7f4ec.html},
  }
  

• Morgan Eichensehr. STUDY: Changes in language, tone could reveal impending betrayal. Diamondback, 2015. [Bibtex]
  
  

  @article{Eichensehr-2015,
  	Author = {Morgan Eichensehr},
  	Year = {2015},
  	Title = {STUDY: Changes in language, tone could reveal impending betrayal},
  	Journal = {Diamondback},
  	Url = {http://www.diamondbackonline.com/news/study-changes-in-language-tone-could-reveal-impending-betrayal/article_a4da0a7c-50f4-11e5-80d6-030e2fec745c.html},
  }
  

• Lexie Schapitl. UMD researchers' computer beats 'Jeopardy!' star Ken Jennings at trivia. Diamondback, 2015. [Bibtex]
  
  

  @article{Schapitl-2015,
  	Author = {Lexie Schapitl},
  	Year = {2015},
  	Title = {UMD researchers' computer beats 'Jeopardy!' star Ken Jennings at trivia},
  	Journal = {Diamondback},
  	Url = {http://www.diamondbackonline.com/news/umd-researchers-computer-beats-jeopardy-star-ken-jennings-at-trivia/article_133f05f4-75d9-11e5-9c00-8718fca7f4ec.html},
  }
  

• Lexie Schapitl. UMD researchers' computer beats 'Jeopardy!' star Ken Jennings at trivia. Diamondback, 2015. [Bibtex]
  
  

  @article{Schapitl-2015,
  	Author = {Lexie Schapitl},
  	Year = {2015},
  	Title = {UMD researchers' computer beats 'Jeopardy!' star Ken Jennings at trivia},
  	Journal = {Diamondback},
  	Url = {http://www.diamondbackonline.com/news/umd-researchers-computer-beats-jeopardy-star-ken-jennings-at-trivia/article_133f05f4-75d9-11e5-9c00-8718fca7f4ec.html},
  }
  

• Leigh Weingus. What You Should Know About Really Polite People. Huffington Post, 2015. [Bibtex]
  
  

  @article{Weingus-2015,
  	Author = {Leigh Weingus},
  	Year = {2015},
  	Title = {What You Should Know About Really Polite People},
  	Journal = {Huffington Post},
  	Url = {http://www.huffingtonpost.com/entry/polite-people-more-likely-to-betray_55e06f5ae4b0b7a96338a186},
  }
  

• Melissa Dahl. Here's a Good Reason to Be Wary of Overly Polite People. New York Magazine, 2015. [Bibtex]
  
  

  @article{Dahl-2015,
  	Author = {Melissa Dahl},
  	Year = {2015},
  	Title = {Here's a Good Reason to Be Wary of Overly Polite People},
  	Journal = {New York Magazine},
  	Url = {http://nymag.com/scienceofus/2015/08/good-reason-to-be-wary-of-overly-polite-people.html?mid=twitter_nymag},
  }
  

• Nathan Collins. Detecting a Coming Betrayal. Pacific Standard, 2015. [Bibtex]
  
  

  @article{Collins-2015,
  	Author = {Nathan Collins},
  	Year = {2015},
  	Title = {Detecting a Coming Betrayal},
  	Journal = {Pacific Standard},
  	Url = {http://www.psmag.com/health-and-behavior/diplomacy-detecting-a-coming-betrayal},
  }
  

• Bill Steele. Language analysis predicts a coming betrayal. Phys.org, 2015. [Bibtex]
  
  

  @article{Steele-2015,
  	Author = {Bill Steele},
  	Year = {2015},
  	Title = {Language analysis predicts a coming betrayal},
  	Journal = {Phys.org},
  	Url = {http://phys.org/news/2015-08-language-analysis-betrayal.html},
  }
  

• Rachel Ehrenberg. A few key signs betray betrayal. Science News, 2015. [Bibtex]
  
  

  @article{Ehrenberg-2015,
  	Author = {Rachel Ehrenberg},
  	Year = {2015},
  	Title = {A few key signs betray betrayal},
  	Journal = {Science News},
  	Url = {https://www.sciencenews.org/blog/culture-beaker/few-key-signs-betray-betrayal?mode=blog&context=159},
  }
  

• Rachel Ehrenberg. A few key signs betray betrayal. Science News, 2015. [Bibtex]
  
  

  @article{Ehrenberg-2015,
  	Author = {Rachel Ehrenberg},
  	Year = {2015},
  	Title = {A few key signs betray betrayal},
  	Journal = {Science News},
  	Url = {https://www.sciencenews.org/blog/culture-beaker/few-key-signs-betray-betrayal?mode=blog&context=159},
  }
  

• Liam Farrell. Question-Answering System Built by UMD, UC Boulder Bests Ken Jennings. Terp, 2015. [Bibtex]
  
  

  @article{Farrell-2015,
  	Author = {Liam Farrell},
  	Year = {2015},
  	Title = {Question-Answering System Built by UMD, UC Boulder Bests Ken Jennings},
  	Journal = {Terp},
  	Url = {http://terp.umd.edu/what-is-a-jeopardy-worthy-computer#.Vh2ts7R_NBc},
  }
  

• Question: Which language scientists designed first competitive quiz bowl playing NLP system?. UMD Language Science Newsletter, 2015. [Bibtex]
  
  

  @article{-2015,
  	Year = {2015},
  	Title = {Question: Which language scientists designed first competitive quiz bowl playing NLP system?},
  	Journal = {UMD Language Science Newsletter},
  	Url = {https://languagescience.umd.edu/news/question-which-language-scientists-designed-first-competitive-quiz-bowl-playing-nlp-system},
  }
  

• Melissa Brachfeld. Computerized Question-Answering System Built by UMD, UC Boulder Bests "Jeopardy!" Champion. UMIACS, 2015. [Bibtex]
  
  

  @article{Brachfeld-2015,
  	Author = {Melissa Brachfeld},
  	Year = {2015},
  	Title = {Computerized Question-Answering System Built by UMD, UC Boulder Bests "Jeopardy!" Champion},
  	Journal = {UMIACS},
  	Url = {https://www.umiacs.umd.edu/about-us/news/computerized-question-answering-system-built-umd-uc-boulder-bests-%E2%80%9Cjeopardy%E2%80%9D-champion},
  }
  

• Melissa Brachfeld. Computerized Question-Answering System Built by UMD, UC Boulder Bests "Jeopardy!" Champion. UMIACS, 2015. [Bibtex]
  
  

  @article{Brachfeld-2015,
  	Author = {Melissa Brachfeld},
  	Year = {2015},
  	Title = {Computerized Question-Answering System Built by UMD, UC Boulder Bests "Jeopardy!" Champion},
  	Journal = {UMIACS},
  	Url = {https://www.umiacs.umd.edu/about-us/news/computerized-question-answering-system-built-umd-uc-boulder-bests-%E2%80%9Cjeopardy%E2%80%9D-champion},
  }
  

Workshops Organized by Project Members

• Synchronous Machine Translation
• Human-Computer Question Answering

Acknowledgments

This work is supported by the National Science Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the researchers and do not necessarily reflect the views of the National Science Foundation.