Schedule of Topics
This is the schedule for
Seminar in Computational Linguistics:
Topics in Computational Linguistics and the Cognitive Neuroscience of Language, Fall 2019
THIS SCHEDULE IS A WORK IN PROGRESS!
In addition, some topic areas may take longer than expected, so keep
an eye on the class mailing list or e-mail me for "official"
Readings will either be linked below, or will be available in the Linguistics Locker.
Overview, goals, administrivia; computation, cognition, and the brain
We'll cover administrivial and general preliminaries, talk about goals, and begin looking at the ways that the relevant disciplines -- computation, linguistics, cognitive science, and neuroscience -- will connect with each other over the course of the semester. If you care about computational explanations, is now the right time to be looking at the brain? If you want to understand what the brain is doing, can current computational approaches help? What, if anything, does deep learning have to do with all of this?
Background readings (optional)
- Marr, D. (1982), Vision: A Computational Approach, San Francisco, Freeman & Co., Introduction and Chapter 1
(ask me if you need the username and password to access this file)
Connecting levels of explanation
Leaders: Craig and Sweta
Debate involving levels of explanation and the relationships among them.
- Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A., & Poeppel, D. (2017). Neuroscience Needs Behavior: Correcting a Reductionist Bias. Neuron, 93(3), 480–490. https://doi.org/10.1016/j.neuron.2016.12.041
- Pater, J. (2019). Generative linguistics and neural networks at 60: Foundation, friction, and fusion. Language. https://doi.org/10.1353/lan.2019.0009
- Berent, I., & Marcus, G. (2019). No integration without structured representations: Response to Pater. Language. https://doi.org/10.1353/lan.2019.0011
- Dunbar, E. (2019). Generative grammar, neural networks, and the implementational mapping problem: Response to Pater. Language. https://doi.org/10.1353/lan.2019.0013
Also of interest
- Brain Inspired podcast. BI 025 John Krakauer: Understanding Cognition, Jan 25, 2019.
- Other responses to Pater in that issue of Language by Pearl, Linzen, and others.
- Pereira, F. (2000). Formal grammar and information theory: Together again?, Philosophical Transactions of the Royal Society of London, Series A, Mathematical, Physical and Engineering Sciences, 358(1769), 1239–1253. doi:10.1098/ rsta.2000.0583
Neuroanatomy and neuroimaging
Shohini will lead in a lecture mode for a significant part of this session, making sure we are all up to speed on the fundamentals of language-relevant neuroanatomy and theories on language function and its localization. The Poldrack paper makes an essential methodological point about inferences connecting cognitive theories and brain activity.
Also of interest
- M.E. Raichle, A brief history of human brain mapping, Trends in Neurosciences, 2009.
- Ch. 2: Brain Mapping Methods. Kemmerer, D. (2015). Cognitive Neuroscience of Language. Psychology Press.
- Brain Science podcast. BS 156 with Russ Poldrack, Can fMRI Read Your Mind?
- Ioannidis, J. P. A. (2005). "Why Most Published Research Findings Are False". PLoS Medicine 2 (8): e124.
- Suzanne Dikker et al., MEG and Language, preprint, 2019. Submitted to an issue of Neuroimaging Clinics on Magnetoencephalography, Roland Lee and Mingxiong Huang (eds).
- Ch. 25: Neural Basis of Speech Perception. Hickok, G. and Poeppel, D. (2015). Neurobiology of Language. Academic Press.
Machine learning in neuroscience
Leader: Sweta, Aura
- Glaser, Joshua I., Ari S. Benjamin, Roozbeh Farhoodi, and Konrad P. Kording. The roles of supervised machine learning in systems neuroscience. Progress in Neurobiology (2019).
- Optional, but recommended especially for computational folks who want to play with the software: Glaser, Joshua I., Raeed H. Chowdhury, Matthew G. Perich, Lee E. Miller, and Konrad P. Kording. Machine learning for neural decoding., arXiv preprint arXiv:1708.00909 (2017); see accompanying package for neural decoding with many different machine learning techniques, https://github.com/KordingLab/Neural_Decoding.
- Carlson, T., Goddard, E., Kaplan, D. M., Klein, C., & Ritchie, J. B. (2018). Ghosts in machine learning for cognitive neuroscience: Moving from data to theory. NeuroImage, 180, 88-100.
- Toneva, M., & Wehbe, L. (to appear). Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain). In Advances in Neural Information Processing Systems.
- Also potentially of interest, once it's available: D. Schwartz, M. Toneva , and L. Wehbe, Inducing brain-relevant bias in natural language processing models, NeurIPS (to appear) [see abstract here]
Decoding (and Encoding)
Leader: Carolin, (Aura|Hanna)
Decoding (and encoding) as ways of connecting work on neural representations with work on distributionally derived representations.
- Murphy, Brian, Leila Wehbe, and Alona Fyshe. "Decoding language from the brain." Language, cognition, and computational models (2018): 53-80. (in PDF locker)
- Shailee Jain, Alexander Huth, Incorporating Context into Language Encoding Models for fMRI (2018)
- Francisco Pereira, Bin Lou, Brianna Pritchett, Samuel Ritter, Samuel J. Gershman, Nancy Kanwisher, Matthew Botvinick & Evelina Fedorenko, Toward a universal decoder of linguistic meaning from brain activation (2018)
Also of interest
- Wehbe, L., Murphy, B., Talukdar, P., Fyshe, A., Ramdas, A., & Mitchell, T. (2014). Simultaneously uncovering the patterns of brain regions involved in different story reading subprocesses. PloS one, 9(11), e112575.
- Wehbe, L., Nunez-Elizalde, A. O., Huth, A. G., Deniz, F., Bilenko, N. Y., & Gallant, J. L. Deep multi-view representation learning of brain responses to natural stimuli. [Find longer version?]
- Huth, A. G., Lee, T., Nishimoto, S., Bilenko, N. Y., Vu, A. T., & Gallant, J. L. (2016). Decoding the semantic content of natural movies from human brain activity. Frontiers in systems neuroscience, 10, 81.
- Francisco Pereira, Tom Mitchell, Analysis of Brain Imaging Data With Machine Learning Methods (slides)
Leader: Carolin, Sweta
Further discussion on decoding (and encoding). We have gotten most of the way through discussing Jain and Huth; we'll finish that discussion and also talk about Pereira et al. In addition, the recent episodes of Brain Inspired with David Poeppel seem very relevant to a lot of what we've been discussing and are a lot of fun, so we'll include those also.
Also start thinking about project ideas, but don't feel as if you need to have something full baked yet. And feel free to formulate ideas connected with things we haven't read yet.
No class: Yom Kippur
Instead of the usual class, the class session time will be available for discussion of project ideas. (Shohini will be there but Philip will not.)
Looking at the word level
Progressing from lower level (recognition) up to individual-word level up to composition at phrase level
- Linzen, T., Marantz, A., & Pylkkänen, L. (2013). Syntactic context effects in visual word recognition: An MEG study. The Mental Lexicon, 8(2), 117-139. [MEG]
- Haoyan Xu, Brian Murphy, Alona Fyshe. BrainBench: A Brain-Image Test Suite for Distributional Semantic Models. Empirical Methods for Natural Language Processing, Austin, TX. 2016
- Alona Fyshe, Gustavo Sudr,e Leila Wehbe, Nicole Rafidi, and Tom M. Mitchell (2019). The Lexical Semantics of Adjective-Noun Phrases in the Human Brain. [preprint]
Language modeling and the time course of sentence processing
- Carried over from last time: Xu et al. (2019)
- Yan, Shaorong, and Aaron Steven White. "A Framework for Decoding Event-Related Potentials from Text", NAACL HLT 2019 (2019): 86.
- Ettinger, A., Feldman, N., Resnik, P. and Phillips, C., 2016. Modeling N400 amplitude using vector space models of word representation. In CogSci.
Hierarchy from a cognitive/neuro angle
Optional (will discuss if we have time):
- Ettinger et al. (2016) (from last time)
- Ding, Nai, Lucia Melloni, Xing Tian and David Poeppel. Rule-based and Word-level Statistics-based Processing of Language: Insights from Neuroscience. Language, cognition and neuroscience 32 5 (2017): 570-575 .
- Frank, S. L., & Christiansen, M. H. (2018). Hierarchical and sequential processing of language a response to: Ding, melloni, tian, and poeppel (2017). rule-based and word-level statistics-based processing of language: Insights from neuroscience. language, cognition and neuroscience. Language, Cognition and Neuroscience. https://doi.org/10.1080/23273798.2018.1424347
- Nelson, M. J., El Karoui, I., Giber, K., Yang, X., Cohen, L., Koopman, H., ... & Dehaene, S. (2017). Neurophysiological dynamics of phrase-structure building during sentence processing. Proceedings of the National Academy of Sciences, 114(18), E3669-E3678.
- For reference, the early work: Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic model. Paper presented at the Proceedings of the second meeting of the North American chapter of the Association for Computational Linguistics, Pittsburgh, PA. doi:10.3115/ 1073336.1073357
Also of interest as background:
- Ding, N., Melloni, L., Zhang, H., Tian, X., & Poeppel, D. (2016). Cortical tracking of hierarchical linguistic structures in connected speech. Nature Neuroscience. doi:10.1038/nn. 4186
Hierarchy in distributed representations
Leader: Cassidy, Alexander
The topic of discussion here is the extent to which neural networks might capture syntactic/hiherarchical structure without explicit building-in of hierarchical mechanisms. There is a cottage industry of papers on this topic; see the Yoav Goldberg arXiv report for a very concrete comparison of a number of approaches.
To be discussed if there is time:
- Gulordava, K., Bojanowski, P., Grave, E., Linzen, T., & Baroni, M. (2018). Colorless Green Recurrent Networks Dream Hierarchically. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (pp. 1195–1205). Stroudsburg, PA, USA: Association for Computational Linguistics. https://doi.org/10.18653/v1/N18-1108
- John Hale, Chris Dyer, Adhiguna Kuncoro, and Jonathan Brennan. 2018. Finding syntax in human encephalography with beam search. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2727–2736. Association for Computational Linguistics.
Also of interest:
- Richard Futrell, Ethan Wilcox, Takashi Morita, Miguel Ballesteros, Roger Levy. Neural Language Models as Psycholinguistic Subjects: Representation of Syntactic State. NAACL-HLT 2019
Neuroimaging and natural stimuli: methodological discussion
Leader: Cassidy, Aura
Also of interest:
- Hamilton, L. S., & Huth, A. G. (2018). The revolution will not be controlled: natural stimuli in speech neuroscience. Language, Cognition and Neuroscience. https://doi.org/10.1080/23273798.2018.1499946
- Rodd, J. M., & Davis, M. H. (2017). How to study spoken language understanding: a survey of neuroscientific methods. nbLanguage,
- Kandylaki, Katerina D., and Ina Bornkessel-Schlesewsky. "From story comprehension to the neurobiology of language." (2019): Journal Language, Cognition and Neuroscience, 34, 405-410.
- Include as optional background: something for Christian?
Studies using natural stimuli
Leader: Christian, Shohini
- Bhattasali, S., Fabre, M., Luh, W.-M., Al Saied, H., Constant, M., Pallier, C., … Hale, J. (2018). Localising memory retrieval and syntactic composition: an fMRI study of naturalistic language comprehension. Language, Cognition and Neuroscience, 1–20. doi: 10.1080/23273798.2018.1518533
- Brodbeck, C., Hong, L. E., & Simon, J. Z. (2018). Rapid Transformation from Auditory to Linguistic Representations of Continuous Speech. Current Biology, 28(24), 3976-3983.e5. https://doi.org/10.1016/J.CUB.2018.10.042
Carried over from last week, this paper will be treated as background, with relevant aspects of the paper incorporated into this week's discussion if/as appropriate:
Also of interest:
- Broderick, Michael P., Andrew J. Anderson, Giovanni M. Di Liberto, Michael J. Crosse, and Edmund C. Lalor. "Electrophysiological correlates of semantic dissimilarity reflect the comprehension of natural, narrative speech". Current Biology 28, no. 5 (2018): 803-809.
- Abstract Linguistic Structure Correlates with Temporal Activity during Naturalistic Comprehension, Jonathan R. Brennan, Edward P. Stabler, Sarah E. Van Wagenen, Wen-Ming Luh, John Hale. Brain and Language, volume 157—158 June–July 2016 pages 81-94. http://dx.doi.org/10.1016/j.bandl.2016.04.008. Already peer-reviewed, closely related to Li & Hale chapter.
The predictive brain
Also of interest:
- Clark, Andy. "Whatever next? Predictive brains, situated agents, and the future of cognitive science". Behavioral and brain sciences 36, no. 3 (2013): 181-204.
- Also note: "A exceptionally large number of excellent commentary proposals inspired a special research topic for further discussion of this target article’s subject matter, edited by Axel Cleeremans and Shimon Edelman in Frontiers in Theoretical and Philosophical Psychology. This discussion has a preface by Cleeremans and Edelman and 25 commentaries and includes a separate rejoinder from Andy Clark. See:
- Shain, C., Blank, I. A., van Schijndel, M., Schuler, W., & Fedorenko, E. (2019). fMRI reveals language-specific predictive coding during naturalistic sentence comprehension. bioRxiv, 717512.
- Heeger - Theory of Cortical Function. [Prediction is constantly taking place, but he models things as a dynamical system that is constantly optimizing a loss function that balances top down prediction, bottom-up evidence, and what he calls "prior drive". The loss function also includes parameters intended to capture the tradeoff between top-down and bottom-up so that, for example, external factors like attention can adjust for how you trade off between the two.]
Assortment of other related topics we didn't get to
OTHER NEUROBIOLOGICAL ARCHITECTURE WORK -- OPPORTUNITY FOR A COMPUTATIONAL LOOK?
- Friston - free energy principle, Bayesian brain hypothesis; Bayesian pragmatics applied to brain?
MISC - OTHER THINGS TO CONSIDER
- Bornkessel-Schlesewsky, I., & Schlesewsky, M. (2013). Reconciling time, space and function: a new dorsal–ventral stream model of sentence comprehension. Brain and language, 125(1), 60-76.
- Friederici, A. D. (2012). The cortical language circuit: from auditory perception to sentence comprehension. Trends in cognitive sciences, 16(5), 262-268.
- Hagoort, P., & Indefrey, P. (2014). The neurobiology of language beyond single words. Annual review of neuroscience, 37, 347-362
- Complementary Learning Systems (Kumaran et al. 2016) -- relating non-statistical and statistical learning (hippocampus and cortex)
- 2016. Temporal Lobes as Combinatory Engines for both Form and Meaning. Jixing Li, Jonathan Brennan, Adam Mahar, John Hale.. Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity (CL4LC), Osaka, Japan. Conference-paper sized writeup of Jixing Li’s work aimed at computational linguists. Also look at more recent modeling - Chinese as well as English?
- Hickok, G. (2012). Computational neuroanatomy of speech production. Nature Reviews Neuroscience. https://doi.org/10.1038/nrn3158 - add a day looking at computational modeling for language production and comprehension, feedback?
- Gwilliams, L., Linzen, T., Poeppel, D., & Marantz, A. (2018). In spoken word recognition, the future predicts the past. The Journal of Neuroscience, 38(35), 7585–7599. DOI: https://doi.org/10.1101/150151
- 2015. Modeling fMRI time courses with linguistic structure at various grain sizes David E. Lutz, Wen-Ming Luh, Jonathan R. Brennan, John Hale. Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, North American Association for Computational Lingustics. Denver, CO. [This is a shorter writeup of the same project reported in the Brain and Language article.]
- Looking at the discourse level -- ?
Philip Resnik, Associate Professor
Department of Linguistics and Institute for Advanced Computer Studies
Department of Linguistics
1401 Marie Mount Hall UMIACS phone: (301) 405-6760
University of Maryland Linguistics phone: (301) 405-8903
College Park, MD 20742 USA Fax: (301) 314-2644 / (301) 405-7104
http://umiacs.umd.edu/~resnik E-mail: resnik AT umd _DOT.GOES.HERE_ edu