Many healthcare problems require thinking not only about the immediate effect of a treatment, but possible long-term ramifications. For example, a certain drug cocktail may cause an immediate drop in viral load in HIV, but also cause the presence of resistance mutations that will reduce the number of viable treatment options in the future. Within machine learning, the reinforcement learning framework is designed to think about decision-making under uncertainty when decisions may have long-lasting effects.

In this talk, I will talk about a number of directions we are developing in my lab to identify personalized treatment policies from electronic health and registry records. Our approaches achieve state-of-the-art results on HIV management and initial promising results for sepsis management. Next, I'll dive into how we evaluated these algorithms when we could not test on new patients and had to rely only on the observational data -- highlighting both current work in our lab on off-policy evaluation as well as more general gotchas that remind us to all be careful scientists.

This is joint work with: Sonali Parbhoo, Maurizio Zazzi, Volker Roth, Xuefeng Peng, David Wihl, Yi Ding, Omer Gottesman, Liwei Lehman, Matthieu Komorowski, Aldo Faisal, David Sontag, Fredrik Johansson, Leo Celi, Aniruddh Raghu, Yao Liu, Emma Brunskill, and the CS282 2017 Topics in Machine Learning Course.

Finale Doshi-Velez is an Assistant Professor in Computer Science at the Harvard Paulson School of Engineering and Applied Sciences. She completed MSc at the University of Cambridge as a Marshall Scholar, her PhD at MIT, and her postdoc at Harvard Medical School. She has won an NSF CAREER Award, is a Sloan fellow, and co-organizes the Machine Learning for Healthcare Conference.