ECE Seminar: Ilan Goodman

ECE Seminar: Ilan Goodman

Zoom Link: https://bostonu.zoom.us/j/96315563219?pwd=IOmlpeLs9NLRYJLTP5aEOoqcuurbRm.1&from=addon

Meeting ID: 963 1556 3219

Passcode: 612775

Title: Frequency Estimation

Abstract: Google Trends tracks the day’s rising searches, exchanges flag the most volatile stocks, retailers rank their best sellers, and security teams hunt for the accounts sending the most spam. Different products, one question underneath: in a stream of events too large to store, which handful of items dominates? In this sample lecture, we build the tools to answer it.

We start with the “heavy hitters” problem—finding every item that appears more than some fraction of the time—and see why the exact solution, easy when the data fits in memory, becomes impossible once we are held to a single pass and far less space than the stream itself. That impossibility is less a dead end than an invitation: if we cannot answer exactly, can we answer approximately, and can we bound how wrong we might be? The question leads to the Count-Min Sketch, a compact probabilistic data structure that estimates how often any item has appeared using a fraction of the memory the data would demand, in exchange for a small, controllable error. We will see where that accuracy comes from, how the same structure recovers the heavy hitters we set out to find, and how it holds up in production systems running at the scale of billions of events. The throughline is the reasoning scalable AI systems demand: knowing not only how to compute an answer, but how much accuracy you can afford to trade for memory and speed.

Bio: Ilan Goodman is a Lecturer in the Department of Computer Science and Engineering at Washington University in St. Louis, where he designed two data engineering courses from scratch and co-teaches the department’s data structures and algorithms sequence. He earned his M.S. in Computer Science (concentrations in Theory and Artificial Intelligence) from Stanford University in 2016, following a B.S. in Physics, and received Stanford’s Centennial Teaching Assistant Award. Before returning to the classroom, he spent eight years as a senior data and machine learning engineer at the Chan Zuckerberg Initiative, Robinhood, and Meta. His interests span computer science education in the age of AI—particularly assessment that measures genuine understanding—together with large-scale and streaming data engineering, applied machine learning, and theoretical computer science.