Teaching
I teach at UC San Diego in Linguistics (LIGN) and Data Science (DSC).
Language Models as Cognitive Models
Spring 2025.
DSC 291 E00. Special Topics in Data Science – Language Models as Cognitive Models
Course Description We can all see how Large Language Models have transformed technology, the workplace, and society, but what about their impact on how we understand our own minds? In this class, we learn how LLMs are revolutionizing the linguistic and behavioral sciences by serving as test subjects and model organisms. This is a project-based, hands-on course in which we train “Baby LMs” from scratch to simulate human language acquisition, gather psycholinguistic data on LMs as they learn and process language, and explore tools from mechanistic interpretability to unveil the inner workings of connectionist models. In addition to lectures, students in teams will lead weekly laboratory sessions in which the entire class develops these skills. Throughout, we will discuss the philosophical questions underlying this endeavor: What can we learn about ourselves from an intelligence that is so different from our own?
Prerequisites: This is a graduate-level course. There are no formal prerequisites. You must know how to program in Python. Other concepts/frameworks that will be useful but not taught in detail include:
- Some familiarity with libraries like Pandas, Matplotlib, Transformers, Pytorch, Numpy.
- Introductory-level knowledge of cognitive science and linguistics.
- Basic technical understanding of neural networks, language models, machine learning.
- Basic use of supercomputer clusters, shell scripting, kubernetes, jupyter, and docker.
These are NOT required prior knowledge. Many of these things will be covered briefly, either directly or indirectly, and many can be learned on one’s own. However, learning these is not the focus of this class, so you must be willing and able to take the initiative to fill in any gaps as necessary.
Information Theory in Linguistics
Winter 2025.
LIGN 169 (formerly LIGN 187). Information Theory in Linguistics.
Course Description: In 1948 Claude Shannon discovered the concept of entropy, kicking off the field of Information Theory and revolutionizing the study of physics, computer science, and genetics. Information theory is a mathematical theory for quantifying communication, but its impact on linguistic theory are only gradually becoming recognized. This course provides an accessible and intuitive introduction to fundamental concepts such as bits, entropy, and mutual information using linguistic case studies. We then study how information theory is beginning to impact linguistics, cognitive science, and natural language processing.
We will explore questions such as:
- How does the brain process language during reading?
- What principles determine the optimal lengths of words?
- How much information do large language models encode about syntax and semantics?
- What does it mean for a speech act to be relevant to a conversation?
- How can we test large language models for harmful stereotypes?
Assignments: Students will be able to choose between reading-focused or coding-focused homework assignments in line with their interests and skills. The course will culminate in a final project, which can also take a few different forms depending on the student’s interests. Students may write a paper as individuals or as a group of up to three. Individual students may also submit a data analysis.
Prerequisites: This is an upper-division undergraduate course. You must have taken one of the following courses:
- LIGN 101
- LIGN 165
- LIGN 167
- prior coursework in linguistics, cognitive science, or natural language processing (upon instructor approval)
Enrollment information: All students wanting to enroll in LIGN 187 must submit an Enrollment Authorization System (EASy) request.