Dr Mariya Toneva | In search of model organisms for language in the human brain: Text vs speech language models
MPSCog Cognition Colloquium
- Date: Feb 19, 2024
- Time: 04:00 PM - 05:00 PM (Local Time Germany)
- Speaker: Dr Mariya Toneva
- Max Planck Institute for Software Systems, Saarbrücken, Germany
- Room: Zoom Meeting
- Host: Max Planck School of Cognition
Please join online: https://zoom.us/j/93526030034?pwd=ZkJnYlFVOEthU2lDeE5nVmV6TlZLZz09 (Meeting ID: 935 2603 0034 / Passcode: 250171)
When processing the sentence The trophy doesnt fit into the brown suitcase because its too big, we understand the meaning despite the ambiguous pronoun it, which may refer to either the trophy or the suitcase. How does the brain process this sentence and attribute real-world meaning to it? To address this, there are some fundamental preliminary questions to answer about what information is processed where and when in the brain, in order to understand how this information is aggregated across different locations and time points. Neuroscientists have made progress towards answering the what, where, and when questions. However, how the meaning of words is aggregated together by the brain still remains elusive. Meanwhile, recent advances in large language models (LLMs) have created computational systems that aggregate the meaning of words in specific ways in order to perform a specific linguistic task, such as predicting the upcoming word in a sentence. However, it is not clear whether these computational systems truly understand the meaning of a sentence, and whether the how of an LLM is the same as the how of the brain. In this talk, we will discuss recent work that specifically examines text vs speech LLMs as model organisms of language processing elicited by listening or reading to the same set of naturalistic stories. Overall, we present evidence that neurolinguistics can benefit from using LLMs as model organisms for how information is aggregated during language processing in the human brain, despite LLMs differences from the human brain. Poster