I study learning for control in large-scale systems. By facilitating decision making in large-scale systems, I aim to enable more sustainable cities and infrastructure. My recent work is motivated by the emerging paradigm for managing traffic by directly controlling vehicles. This has the potential to enable roadways that are not only more efficient, but also safer, healthier, and more environmentally friendly. To understand this emerging paradigm, we design flexible methods for rigorously optimizing a vast range of scenarios and sustainability metrics. Such methods must cope with complexity, diversity, and scale. My work draws from reinforcement learning, deep learning, control, optimization, and stochastic modeling. Broadly, I am interested in applications in large-scale multi-agent systems as well as combinatorial optimization.

I am the Gilbert W. Winslow (1937) Career Development Assistant Professor at MIT, where I am part of the Laboratory for Information & Decision Systems (LIDS), the Department of Civil and Environmental Engineering (CEE), and the Institute for Data, Systems, & Society (IDSS).

Prior to joining MIT, I completed a postdoc with the Microsoft Research Reinforcement Learning group. I completed my PhD in EECS at UC Berkeley. I received a BS and MEng in EECS from MIT. I have also spent time at OpenAI, Waymo, Dropbox, Facebook, and several startups.

Recent news | Teaching | Group | Publications

Recent news

For an introduction to what fascinates me, see The Hidden Cause of Traffic Jams—and How to Solve Them (NOVA, 2022) and How Cities Can Reshape Cars (TEDxMIT, 2021).

For a recent research talk, see Cities as Robots: Scalability, Operations, and Robustness from the RSS Workshop on Learning from Diverse, Offline Data in NYC (June 2022).


Teaching

Courses
6.7950: Reinforcement Learning. Fall 2022, Spring 2021, Spring 2020. Previously 6.246.
6.231 Dynamic Programming and Reinforcement Learning. Spring 2022.
1.041/1.200 Transportation. Spring 2023, Fall 2021, Fall 2020, Fall 2019.
EE290O Multi-agent reinforcement learning for autonomous traffic. Fall 2018.
CS189 Machine Learning. Content Development TA. Fall 2017.
CS170 Algorithms. Head TA. Spring 2016.

Outreach and Professional Education
These courses are intended for industry professionals and not MIT students.
Reinforcement Learning: Summer 2022, Winter 2021
Advanced Reinforcement Learning: Summer 2022, Summer 2021

Group

PhD students
Zhongxia “Zee” Yan
Vindula Jayawardana
Sirui Li
Jiaqi “Vicky” Zhang (co-advised with Prof. Caroline Uhler)
Cameron Hickert
Wenbin Ouyang
Jung Hoon Cho
Mehul Damani (co-advised with Prof. Dylan Hadfield-Menell)

Masters students
Edgar Ramirez Sanchez
Jason Teno (LGO)
Ao Qu (co-advised with Prof. Jinhua Zhao)

UROPs & Interns
Catherine Tang
Kevin Liu
Sunera Avinash (University of Moratuwa)
Edouard Labarthe (École polytechnique)
Anirudh Valiveru
Jiaxin He (Vanderbilt)
Baptiste Freydt (ETH Zurich)

Articles in review / preparation

Learning Surrogates for Diverse Emission Models
Edgar Sanchez*, Catherine Tang*, Vindula Jayawardana, Cathy Wu

What is a Typical Signalized Intersection in a City? A Pipeline for Intersection Data Imputation from OpenStreetMap
Ao Qu*, Anirudh Valiveru*, Catherine Tang, Vindula Jayawardana, Baptiste Freydt, Cathy Wu

Mitigating The Braess’s Paradox in A Closed System Using Reinforcement Learning
Dingyi Zhuang*, Yuzhu Huang*, Vindula Jayawardana, Jinhua Zhao, Dajiang Suo, Cathy Wu
( pre-print )

Cooperation for Scalable Supervision of Autonomy in Mixed Traffic
Cameron Hickert, Sirui Li, Cathy Wu
IEEE Transactions on Robotics (T-RO), conditionally accepted.
( pre-print )

Generalization for Irregular, Under-actuated, and Large-scale Multi-robot Control
Zhongxia Yan, Cathy Wu

Path Clearance for Emergency Vehicles Under Mixed Autonomy: An Opportunity for Low Market Penetration CAVs
Dajiang Suo*, Vindula Jayawardana*, Cathy Wu

User-friendly Interpretations in Reinforcement Learning
Vindula Jayawardana, Cameron Hickert, Tsui-Wei Weng, Sijia Liu, Lam M. Nguyen, Cathy Wu

Publications

An Invisible Issue of Task Underspecification in Deep Reinforcement Learning
Vindula Jayawardana, Catherine Tang, Sirui Li, Dajiang Suo, Cathy Wu
Conference on Neural Information Processing Systems (NeurIPS), 2022.

The Braess Paradox in Dynamic Traffic
Dingyi Zhuang*, Yuzhu Huang*, Vindula Jayawardana, Jinhua Zhao, Dajiang Suo, Cathy Wu
IEEE Intelligent Transportation Systems Conference (ITSC), 2022.
( pre-print )

Unified Automatic Control of Vehicular Systems with Reinforcement Learning
Zhongxia Yan, Abdul R. Kreidieh, Eugene Vinitsky, Alexandre M. Bayen, Cathy Wu
IEEE Transactions on Automation Science and Engineering (T-ASE), 2022.
Additionally selected for presentation at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022.
( journal )
Interview: On the Future of Our RoadsThe Robot Brains Podcast

Learning Eco-Driving Strategies at Signalized Intersections
Vindula Jayawardana, Cathy Wu
European Control Conference (ECC), 2022.
Also, IEEE International Conference on Robotics and Automation (ICRA), 2022. Robotics for Climate Change Workshop. Invited flash talk.
( paper / website / poster )
Press: On the road to cleaner, greener, and faster drivingMIT News. Home page feature.

Reinforcement Learning for Empirical Supervision Scaling of Autonomous Vehicles
Cameron Hickert, Cathy Wu.
IEEE International Conference on Robotics and Automation (ICRA), 2022. Workshop Safe and Reliable Autonomy Under Uncertainty.

Sociotechnical Specification for the Broader Impacts of Autonomous Vehicles
Thomas Krendl Gilbert, Aaron Snoswell, Michael Dennis, Rowan McAllister, Cathy Wu
IEEE International Conference on Robotics and Automation (ICRA), 2022. Fresh Perspectives on Autonomous Driving Workshop.
( paper )

Learning to Delegate for Large-scale Vehicle Routing
Sirui Li*, Zhongxia Yan*, Cathy Wu
Conference on Neural Information Processing Systems (NeurIPS), 2021. Spotlight (<3%).
Also, International Conference on Machine Learning (ICML), 2021. Workshop on Subset Selection in Machine Learning.
( paper / website / OpenReview / poster / github )
Press: Machine learning speeds up vehicle routingMIT News

Learning to Dissipate Traffic Jams with Piecewise Constant Control
Mayuri Sridhar, Cathy Wu
Conference on Neural Information Processing Systems (NeurIPS), 2021. Workshop on Tackling Climate Change with Machine Learning, 2021.
( paper / slides / talk / proceedings )

Piecewise Constant Policies for Human-Compatible Congestion Mitigation
Mayuri Sridhar, Cathy Wu
IEEE Intelligent Transportation Systems Conference (ITSC), 2021.
( proceedings )

Flow: A Modular Learning Framework for Mixed Autonomy Traffic
Cathy Wu, Aboudy Kreidieh, Kanaad Parvate, Eugene Vinitsky, Alexandre M. Bayen
IEEE Transactions on Robotics (T-RO), 2021.
( paper / journal / videos / website / github )
Interview: Simulating the Future of Traffic with RL w/ Cathy WuTWIML AI Podcast
Interview: The Future of Mixed-Autonomy Traffic with Alexandre BayenTWIML AI Podcast
Press: Science, Wired, O’Reilly (Chinese version), Berkeley College of Engineering, abc News, Berkeley Lab, India Times, and Russian Forbes

Reinforcement Learning for Mixed Autonomy Intersections
Zhongxia Yan, Cathy Wu
IEEE Intelligent Transportation Systems Conference (ITSC), 2021.
( paper / proceedings / github )

Mixed Autonomous Supervision in Traffic Signal Control
Vindula Jayawardana, Anna Landler, Cathy Wu
IEEE Intelligent Transportation Systems Conference (ITSC), 2021.
( proceedings )

Block simplex signal recovery: a method comparison and an application to routing
Cathy Wu, Alexei Pozdnoukhov, Alexandre M. Bayen
IEEE Transactions on Intelligent Transportation Systems (T-ITS), 2019.
( journal / github )

Variance Reduction for Policy Gradient Using Action-Dependent Factorized Baselines
Cathy Wu, Aravind Rajeswaran, Yan Duan, Vikash Kumar, Alexandre M Bayen, Sham Kakade, Igor Mordatch, Pieter Abbeel
International Conference on Learning Representations (ICLR), 2018. Oral (2%).
Also, Conference on Neural Information Processing Systems (NeurIPS), 2017. Deep Reinforcement Learning Symposium. Contributed talk.
( paper / OpenReview )

Stabilizing traffic with autonomous vehicles
Cathy Wu, Alexandre M. Bayen, Ankur Mehta
International Conference on Robotics and Automation (ICRA), 2018.
( proceedings / talk video )

Learning and Optimization for Mixed Autonomy Systems – A Mobility Context
Cathy Wu
Thesis. PhD, Electrical Engineering and Computer Sciences, UC Berkeley, 2018.
( thesis / summary )
Award: First Place 2019 IEEE ITSS Best Ph.D. Dissertation Award (news)
Award: 2018 Milton Pikarsky Memorial Award from the Council of University Transportation Centers (CUTC) for the best Doctoral dissertation in the field of science and technology in transportation studies (news).

Lagrangian Control through Deep-RL: Applications to Bottleneck Decongestion
Eugene Vinitsky, Kanaad Parvate, Abdul R. Kreidieh, Cathy Wu, Alexandre M. Bayen
IEEE Intelligent Transportation Systems Conference (ITSC), 2018.
( paper / proceedings )

Dissipating stop-and-go waves in closed and open networks via deep reinforcement learning
Aboudy Kreidieh, Cathy Wu, Alexandre M. Bayen
IEEE Intelligent Transportation Systems Conference (ITSC), 2018.
( paper / proceedings )

Benchmarks for reinforcement learning in mixed-autonomy traffic
Eugene Vinitsky*, Aboudy Kreidieh*, Luc Le Flem, Nishant Kheterpal, Kathy Jang, Cathy Wu, Fangyu Wu, Richard Liaw, Eric Liang, Alexandre M. Bayen.
Conference on Robot Learning (CoRL), 2018.
( paper / proceedings )
Press: Watch just a few self-driving cars stop traffic jamsScience

Emergent behaviors in mixed-autonomy traffic
Cathy Wu, Aboudy Kreidieh, Eugene Vinitsky, Alexandre M. Bayen
Conference on Robot Learning (CoRL), 2017.
( paper / proceedings )
Press: Autonomous Vehicles: The Answer to Our Growing Traffic WoesWired

Multi-lane Reduction: A Stochastic Single-lane Model for Lane Changing
Cathy Wu, Eugene Vinitsky, Abdul R. Kreidieh, Alexandre M. Bayen
IEEE Intelligent Transportation Systems Conference (ITSC), 2017.
( proceedings )

Framework for Control and Deep Reinforcement Learning in Traffic
Cathy Wu, Kanaad Parvate, Nishant Kheterpal, Leah Dickstein, Ankur Mehta, Eugene Vinitsky, Alexandre M. Bayen
IEEE Intelligent Transportation Systems Conference (ITSC), 2017.
( proceedings )

Clustering for Set Partitioning with a Case Study in Ridesharing
Cathy Wu, Ece Kamar, Eric Horvitz
IEEE Intelligent Transportation Systems Conference (ITSC), 2016. Best paper award.
( paper / proceedings )

Optimizing the diamond lane: A more tractable carpool problem and algorithms
Cathy Wu, K. Shankari, Ece Kamar, Randy Katz, David Culler, Christos Papadimitriou, Eric Horvitz, Alexandre M. Bayen
IEEE Intelligent Transportation Systems Conference (ITSC), 2016.
( paper / proceedings )

Cellpath: fusion of cellular and traffic sensor data for route flow estimation via convex optimization
Cathy Wu, Jerome Thai, Steve Yadlowsky, Alexei Pozdnoukhov, Alexandre M. Bayen
Transportation Research: Part C, 2015.
International Symposium on Transportation and Traffic Theory (ISTTT), 2015. Oral (14%).
( paper / journal / proceedings / github (system) / github (algorithm) )

Convex programming on the l1-ball and on the simplex via isotonic regression
Jerome Thai, Cathy Wu, Alexei Pozdnoukhov, Alexandre M. Bayen
Conference on Decision and Control (CDC), 2015.
( paper / proceedings )

Link Density Inference from Cellular Infrastructure
Steve Yadlowsky, Jerome Thai, Cathy Wu, Alexei Pozdnoukhov, Alexandre M. Bayen
Transportation Research Record (TRR), 2015.
Transportation Research Board (TRB) Annual Meeting, 2015.
( paper / journal / proceedings )


Selected other writing

It’s Time to Do Something: Mitigating the Negative Impacts of Computing Through a Change to the Peer Review Process.
Brent Hecht, Lauren Wilcox, Jeffrey P. Bigham, Johannes Schöning, Ehsan Hoque, Jason Ernst, Yonatan Bisk, Luigi De Russis, Lana Yarosh, Bushra Anjum, Danish Contractor, Cathy Wu
ACM Future of Computing Academy Blog, 2018.
( article )

Traffic Jammin’: Making automated transportation a reality
Cathy Wu
Berkeley Science Review, 2016.
( article )

Automating us: The entanglement of people and machines
Daniel Aranki*, Roel Dobbe*, Jaime F. Fisac*, Cathy Wu*
Berkeley Science Review, 2015.
( article )

How to Lead a Technical Reading Group
Cathy Wu, Oct 2012.
( paper )