Teaching
I teach at UC San Diego in Linguistics (LIGN) and Data Science (DSC).
Information Theory in Linguistics
Winter 2025.
LIGN 169 (formerly LIGN 187). Information Theory in Linguistics.
Course Description In 1948 Claude Shannon discovered the concept of entropy, kicking off the field of Information Theory and revolutionizing the study of physics, computer science, and genetics. Information theory is a mathematical theory for quantifying communication, but its impact on linguistic theory are only gradually becoming recognized. This course provides an accessible and intuitive introduction to fundamental concepts such as bits, entropy, and mutual information using linguistic case studies. We then study how information theory is beginning to impact linguistics, cognitive science, and natural language processing.
We will explore questions such as:
- How does the brain process language during reading?
- What principles determine the optimal lengths of words?
- How much information do large language models encode about syntax and semantics?
- What does it mean for a speech act to be relevant to a conversation?
- How can we test large language models for harmful stereotypes?
Assignments Students will be able to choose between reading-focused or coding-focused homework assignments in line with their interests and skills. The course will culminate in a final project, which can also take a few different forms depending on the student’s interests. Students may write a paper as individuals or as a group of up to three. Individual students may also submit a data analysis.
Prerequisites: One of the following:
- LIGN 101
- LIGN 165
- LIGN 167
- prior coursework in linguistics, cognitive science, or natural language processing (upon instructor approval)
Enrollment information: All students wanting to enroll in LIGN 187 must submit an Enrollment Authorization System (EASy) request.