Full Transcript for Ideas: Community building, machine learning, and the future of AI

00:04:48.960 --> 00:04:55.120 And, like you, I'd always been interested in people. And during the course of my PhD, 00:04:55.120 --> 00:05:00.800 I realized that I wasn't interested in analyzing text for the sake of text, right. 00:05:00.800 --> 00:05:06.400 I was interested because text is one of these ways that people communicate with each other. You know, 00:05:06.400 --> 00:05:09.520 people don't write text for the sake of writing text. They write it because they're trying to 00:05:09.520 --> 00:05:14.720 convey something. And it was really that that I was interested in. It was these, kind of, social 00:05:14.720 --> 00:05:22.480 aspects of text that I found super interesting. So coming out of the PhD, I then got a postdoc 00:05:22.480 --> 00:05:30.320 job focused on analyzing texts as part of these, sort of, broader social processes. From there, 00:05:30.320 --> 00:05:36.400 I ended up getting a faculty job, also at UMass, as one of four founding members of 00:05:36.400 --> 00:05:43.040 UMass's Computational Social Science Institute. So there was me in computer science, then there was 00:05:43.040 --> 00:05:48.720 another assistant professor in statistics, another in political science, and another in sociology. 00:05:48.720 --> 00:05:56.400 And in many ways, this was my dream job. I was being paid to develop and use machine learning 00:05:56.400 --> 00:06:02.560 methods to study social processes and answer questions that social scientists wanted to study. 00:06:03.600 --> 00:06:09.200 It was pretty awesome. You, I think, started a faculty position at the same time, right? 00:06:09.200 --> 00:06:13.440 WORTMAN VAUGHAN: Yeah. So I also did a postdoc. First, 00:06:13.440 --> 00:06:17.040 I spent a year as a postdoc at Harvard, which was super fun. And 00:06:17.040 --> 00:06:22.720 then I started a tenure track position in computer science at UCLA in 2010. 00:06:22.720 --> 00:06:28.400 Again, you know, it was a very male-dominated environment. My department was mostly men. 00:06:29.040 --> 00:06:35.440 But maybe even more importantly than this, I just didn't really have a network there. You know, it 00:06:35.440 --> 00:06:42.320 was lonely. One exception to this was Mihaela van der Schaar. She was at UCLA at the time, though 00:06:42.320 --> 00:06:47.040 not in my department, and she, kind of, took me under her wing. So I'm very grateful that I had 00:06:47.040 --> 00:06:53.600 that support. But overall, this position just wasn't a great fit for me, and I was under more 00:06:53.600 --> 00:06:58.880 stress then than I think I have been at any other point in my life that I could really remember. 00:06:58.880 --> 00:07:04.800 WALLACH: Yeah. So at that point, then, you ended up transitioning to Microsoft Research, right? 00:07:04.800 --> 00:07:05.027 WORTMAN VAUGHAN: Yep. 00:07:05.027 --> 00:07:09.583 WALLACH: Why did you end up choosing MSR [Microsoft Research]? 00:07:09.583 --> 00:07:15.440 WORTMAN VAUGHAN: Yeah, so this was back in 2012. MSR had

just opened up this new New York 00:07:15.440 --> 00:07:22.400 City lab at the time, and working in this lab was basically my dream job. I think I actually tried 00:07:22.400 --> 00:07:27.600 to apply before they had even officially opened the lab, like when I just heard it was happening. 00:07:28.320 --> 00:07:33.360 So this lab focused in three areas at the time. It focused in machine learning, 00:07:33.360 --> 00:07:38.080 algorithmic economics, and computational social science. And my research at the 00:07:38.080 --> 00:07:43.680 time cut across all three of these areas. So it felt just like this perfect opportunity 00:07:43.680 --> 00:07:49.360 to work in the space where my work would fit in so well and be really appreciated. 00:07:50.400 --> 00:07:54.400 The algorithmic economics group at the time actually was working on building 00:07:54.400 --> 00:08:00.560 prediction markets to aggregate information about future events, and they were already, 00:08:00.560 --> 00:08:04.400 in doing this, building on top of some of my theoretical research, which was just super 00:08:04.400 --> 00:08:10.800 cool to see. So that was exciting. And I already knew a couple of people here. 00:08:10.800 --> 00:08:16.240 I knew John Langford and Dave Pennock, who was in the economics group at the time, 00:08:16.240 --> 00:08:19.920 because I'd done an internship actually with the two of them at Yahoo Research before they 00:08:19.920 --> 00:08:25.440 came to Microsoft. And I was really excited to come back and work with them again, as well. 00:08:25.440 --> 00:08:31.760 You know, even here at the time that I joined the lab, it was 13 men and me. So once again, 00:08:31.760 --> 00:08:37.600 not great numbers. And I think that in some ways this was especially hard on me because I was just 00:08:37.600 --> 00:08:42.240 naturally, like, a very shy person and I hadn't really built up the confidence that I should have 00:08:42.240 --> 00:08:48.080 at that point in my career. But on the other hand, I found the research fit just so spot-on 00:08:48.080 --> 00:08:56.160 that I couldn't say no. And I suspect that this is something that you understand yourself because 00:08:56.160 --> 00:09:00.640 you actually came and joined me here in the New York lab a year or two later. 00:09:00.640 --> 00:09:07.360 So why did you make this switch? WALLACH: Yeah, so I anticipated that I 00:09:07.360 --> 00:09:14.400 was going to love my faculty job. It was focusing on all this stuff that I was so excited about. And 00:09:14.400 --> 00:09:21.520 much to my surprise, though, I kind of didn't. And it wasn't like there was any one particular thing 00:09:21.520 --> 00:09:27.840 that I didn't like. It was more of a mixture of things. I did love my research, though. That was 00:09:27.840 --> 00:09:36.400 pretty clear to me. But I wasn't happy. So I spent a summer talking to as many people as possible in 00:09:36.400 --> 00:09:41.840 all different kinds of jobs, really just with the goal of figuring out what their day-to-day lives 00:09:41.840 --> 00:09:46.240 looked like. You were one of the people I spoke to, but I spoke to a ton of other people, as well. 00:09:46.240 --> 00:09:52.400 And from doing that, at the end of that summer, I ended up deciding to apply to industry jobs, 00:09:52.400 --> 00:09:57.600 and I applied to a bunch of places and got a bunch of offers. But I ended up deciding to 00:09:57.600 --> 00:10:03.760 join Microsoft Research New York City because of all the places I was considering going, 00:10:03.760 --> 00:10:07.360 they were the only place that said, “We love your research. 00:10:07.360 --> 00:10:12.400 We love what you do. Do you want to come here and do that same research?” 00:10:12.400 --> 00:10:16.240 And that was really appealing to me because I loved my research. Of course, I wanted to come 00:10:16.240 --> 00:10:21.040 there and do my same research and especially with all of these amazing people like you, 00:10:21.040 --> 00:10:26.960 Duncan Watts, who’d for many years been somebody I'd really looked up to. He was there, as well, 00:10:26.960 --> 00:10:32.400 at that point in time. There was this real focus on computational social science but with a little 00:10:32.400 --> 00:10:37.760 bit more of an industry perspective. There were also these amazing machine learning researchers. 00:10:38.400 --> 00:10:44.640 Just for many of the same reasons as you, I was just really excited to join that lab 00:10:44.640 --> 00:10:49.343 and particularly excited to be working in the same organization as you again. 00:10:49.343 --> 00:10:53.017 WORTMAN VAUGHAN: Yeah, I'm happy to take at least a little bit of the credit for … 00:10:53.017 --> 00:10:53.339 WALLACH: Oh yeah. 00:10:53.339 --> 00:10:55.240 WORTMAN VAUGHAN: … recruiting you to Microsoft here many years ago. 00:10:55.240 --> 00:10:57.600 WALLACH: Oh yeah. WORTMAN VAUGHAN: Yeah. I 00:10:57.600 --> 00:11:01.200 was really excited to have you join, too, though I think the timing actually worked 00:11:01.200 --> 00:11:04.640 out so that I missed your first couple of months because I was on maternity leave 00:11:04.640 --> 00:11:09.600 with my first daughter at the time. I should say I’ve got two daughters, 00:11:09.600 --> 00:11:14.000 and I'm very proud to share in the context of this podcast that they're both very interested 00:11:14.000 --> 00:11:19.440 in math and reading, as well. WALLACH: Yeah, they're both great. 00:11:19.440 --> 00:11:25.920 Um, so then we ended up working in the same place. But despite that, it still took us several years 00:11:25.920 --> 00:11:33.103 to end up actually collaborating on research. Do you remember how we ended up working together? 00:11:33.103 --> 00:11:38.400 WORTMAN VAUGHAN: Yeah. So I used to tell this story a lot. Actually, I was at this panel on 00:11:38.400 --> 00:11:46.720 AI in society back in, I think, it was probably 2016. It was taking place in DC. And someone on 00:11:46.720 --> 00:11:53.200 this panel

made this statement that soon our AI systems are just going to be so good that 00:11:53.200 --> 00:11:58.560 all of the uncertainty is going to be taken out of our decision-making, and something about this 00:11:58.560 --> 00:12:03.156 statement just, like, really set me off. I got so mad about it because I thought it was just … 00:12:03.156 --> 00:12:05.360 WALLACH: I remember. WORTMAN VAUGHAN: … such an irresponsible 00:12:05.360 --> 00:12:11.840 thing to be saying. So I came back to New York, and I think I was ranting to you about this in 00:12:11.840 --> 00:12:20.160 the lab, and this conversation ended up getting us started on this whole longer discussion about the 00:12:20.160 --> 00:12:26.080 importance of communicating uncertainty and about explaining the assumptions that are behind the 00:12:26.080 --> 00:12:31.840 predictions that you're making and all of this. WALLACH: So this was something … I was really 00:12:31.840 --> 00:12:36.080 excited about this because this was something that had really been drummed into me for years 00:12:36.080 --> 00:12:41.040 as a Bayesian. So Bayesian statistics, which forms a lot of the foundation of the type of machine 00:12:41.040 --> 00:12:47.440 learning that I was doing, is all about explicitly stating assumptions and quantifying uncertainty. 00:12:47.440 --> 00:12:52.800 So I just felt super strongly about this stuff. WORTMAN VAUGHAN: Yeah. So somehow all of these 00:12:52.800 --> 00:12:57.280 discussions we were having led us to read up on this literature that was 00:12:57.280 --> 00:13:01.760 coming out of the machine learning community on interpretability at the time. There are a 00:13:01.760 --> 00:13:05.600 bunch of these papers coming out that were making claims about models being 00:13:05.600 --> 00:13:11.680 interpretable without stopping to define who they were interpretable to or for what 00:13:11.680 --> 00:13:18.080 purpose. Never actually taking these models and putting them down in front of real people. 00:13:18.080 --> 00:13:24.320 And we wanted to do something about this. So we started running controlled experiments with real 00:13:24.320 --> 00:13:30.200 people and found that we often can't trust our intuition about what makes a model interpretable. 00:13:30.200 --> 00:13:34.880 WALLACH: Yeah, one of the things that came up a lot in that work was, sort of, 00:13:34.880 --> 00:13:41.120 how to measure these squishy abstract human concepts, like interpretability, 00:13:41.120 --> 00:13:46.783 that are really hard to define, let alone quantify and measure and stuff like that. 00:13:46.783 --> 00:13:50.800 WORTMAN VAUGHAN: Absolutely. So I think one of the first things that we really 00:13:50.800 --> 00:13:56.880 struggled with in this line of work was what it even means to be interpretable or 00:13:56.880 --> 00:14:01.440 intelligible or any of these terms that were getting thrown around at the time. 00:14:01.440 --> 00:14:07.573 Um, we ended up doing some research, which is still one

of my favorite papers, ... 00:14:07.573 --> 00:14:07.592 WALLACH: Me, too. 00:14:07.592 --> 00:14:11.600 WORTMAN VAUGHAN: ... with our colleagues Forough Poursabzi, Jake Hofman, 00:14:11.600 --> 00:14:19.680 and Dan Goldstein. And in this work, we found it really useful to think about interpretability as 00:14:19.680 --> 00:14:25.120 a latent property that can be, kind of, influenced by different properties of a 00:14:25.120 --> 00:14:29.840 model or system’s design. So things like the number of features the model has or 00:14:29.840 --> 00:14:34.720 whether the model's linear or even things like the user interface of the model. 00:14:34.720 --> 00:14:40.240 This was kind of a gateway project for me in the sense that it's one of the first projects that 00:14:40.240 --> 00:14:46.320 I got really excited about that was more of a human-computer interaction, or HCI, 00:14:46.320 --> 00:14:52.560 project rather than a theory project like I'd been working on in the past. And it just set 00:14:52.560 --> 00:14:59.040 off this huge spark of excitement in me. It felt to me at the time more important than 00:14:59.040 --> 00:15:03.920 other things that I was doing, and I just wanted to do more and more of this work. 00:15:03.920 --> 00:15:08.080 I would say the other project that had a really similar effect on me, 00:15:08.080 --> 00:15:11.680 which we also worked on together right around the same time, 00:15:11.680 --> 00:15:16.080 was our work with Ken Holstein mapping out challenges that industry practitioners were 00:15:16.080 --> 00:15:22.560 facing in the space of AI fairness. WALLACH: Oh yeah. OK, yep. That project, 00:15:22.560 --> 00:15:31.040 that was so fun, and I learned so much from it. If I recall correctly, we originally hired Ken, 00:15:31.040 --> 00:15:36.763 who I think was an HCI PhD student at CMU at the time, as an intern … 00:15:36.763 --> 00:15:38.480 WORTMAN VAUGHAN: Yep. WALLACH: … to work with us on creating, 00:15:38.480 --> 00:15:46.160 sort of, user experiences for fairness tools like the Fairlearn toolkit. And we started that 00:15:46.160 --> 00:15:52.000 project—so that was in collaboration with Miro Dudík and Hal Daumé—we started that project by 00:15:52.000 --> 00:15:58.640 having Ken talk to a whole bunch of practitioners at Microsoft but at other organizations, as well, 00:15:58.640 --> 00:16:04.960 to get a sense for how they were and weren't using fairness toolkits like Fairlearn. 00:16:04.960 --> 00:16:09.120 And I want to point out that at that point in time, the academic research community 00:16:09.120 --> 00:16:14.400 was super focused on all of these, like, simple quantitative metrics for assessing 00:16:14.400 --> 00:16:20.160 the fairness in the context of predictions and predictive machine learning models with this, 00:16:20.160 --> 00:16:25.040 kind of, understanding that these tools could then be built to help practitioners assess 00:16:25.040 --> 00:16:29.440 the fairness of their predictive models and maybe

even make fairer predictions. 00:16:29.440 --> 00:16:34.320 And so that's the kind of stuff that this Fairlearn toolkit was originally developed 00:16:34.320 --> 00:16:40.320 to do. So we ended up asking all of these practitioners originally just as, sort of, 00:16:40.320 --> 00:16:44.320 the precursor to what we thought we were going to end up doing with this project. 00:16:44.320 --> 00:16:47.920 We also asked these practitioners about their current practices and challenges 00:16:47.920 --> 00:16:54.720 around fairness in their work and about their additional needs for support. So where did 00:16:54.720 --> 00:16:58.400 they feel like they had the right tools and processes and practices and where did they 00:16:58.400 --> 00:17:04.560 feel like they were missing stuff. And this was really eye-opening because what we found 00:17:04.560 --> 00:17:10.480 was so different than what we were expecting. And there's two things that really stood out to us. 00:17:10.480 --> 00:17:15.040 So the first thing was that we found a much, much wider range of applications 00:17:15.040 --> 00:17:20.560 beyond prediction. So we'd come into this assuming that all these practitioners were 00:17:20.560 --> 00:17:24.880 doing stuff with predictive machine learning models, but in fact, we were finding they were 00:17:24.880 --> 00:17:28.400 doing all kinds of stuff. There was a bunch of unsupervised stuff; there was a bunch of, 00:17:28.400 --> 00:17:33.840 you know, language-based stuff—all of this kind of thing. And in hindsight, that probably doesn't 00:17:33.840 --> 00:17:39.120 sound very surprising nowadays because of the rise of generative AI, and really the 00:17:39.120 --> 00:17:44.080 entire machine learning and AI field is much less focused on prediction in that, kind of, 00:17:44.080 --> 00:17:49.520 narrow, kind of, classification-regression kind of way. But at the time, this was really surprising, 00:17:49.520 --> 00:17:56.800 especially in light of the academic literature's focus on predictions when thinking about fairness. 00:17:57.440 --> 00:18:01.760 The second thing that we found was that practitioners often struggled to use 00:18:01.760 --> 00:18:07.680 existing fairness research, in part because these quantitative metrics that were all the rage at 00:18:07.680 --> 00:18:14.240 that point in time, just weren't really amenable to the types of real-world complex scenarios that 00:18:14.240 --> 00:18:17.840 these practitioners were facing. And there was a bunch of different reasons for this, 00:18:17.840 --> 00:18:21.840 but one of the things that really stood out to us was that this wasn't so much 00:18:21.840 --> 00:18:26.080 about the underlying models and stuff like that, but it was actually that there were 00:18:26.080 --> 00:18:32.160 a variety of data challenges involved here around things like data collection, 00:18:32.160 --> 00:18:38.720 collection of sensitive attributes, which you need in order to actually use these fairness metrics. 00:18:38.720 --> 00:18:43.280 So putting all this together, the upshot of all this was that we never did what we 00:18:43.280 --> 00:18:46.800 originally set out to do with that [LAUGHS], that internship project. 00:18:47.600 --> 00:18:52.880 We … because we uncovered this really large gap between research and practice, 00:18:52.880 --> 00:18:57.600 we ended up publishing this paper that characterized this gap and then surfaced 00:18:57.600 --> 00:19:03.840 important directions for future research. The other thing that the paper did was emphasize 00:19:03.840 --> 00:19:10.000 the importance of doing this kind of qualitative work to actually understand what's happening in 00:19:10.000 --> 00:19:16.000 practice rather than just making assumptions about what practitioners are and aren't doing. 00:19:16.640 --> 00:19:22.480 The other thing that came out of it, of course, was that the four of us—so you, me, Miro and 00:19:22.480 --> 00:19:31.583 Hal—learned a ton about HCI and about qualitative research from Ken, which was just, uh, so fun. 00:19:31.583 --> 00:19:37.600 WORTMAN VAUGHAN: Yeah, and I started to be confronted with the fact that I 00:19:37.600 --> 00:19:42.320 could no longer reasonably ignore all of these messes of the real world because, 00:19:42.320 --> 00:19:47.600 you know, in some ways, responsible AI is really all about the messes. 00:19:47.600 --> 00:19:53.520 So I think this project was really a big shift for both of us. And in some ways, 00:19:53.520 --> 00:19:59.440 working on this and the interpretability work really led us to be active in these 00:19:59.440 --> 00:20:04.960 early efforts that were happening within Microsoft in the responsible AI space. Um, 00:20:04.960 --> 00:20:09.680 the research that we were doing was feeding directly into company policy, and it felt 00:20:09.680 --> 00:20:14.320 like it was just, like, a huge place where we could have some impact. So it was very exciting. 00:20:14.320 --> 00:20:19.800 So switching gears a bit. Hanna, do you remember how we first got the idea for WiML? 00:20:19.800 --> 00:20:30.640 WALLACH: Yes, I do. So we were at NeurIPS. This was back in 2005. It was a … so NeurIPS was a 00:20:30.640 --> 00:20:35.280 very different conference back then. Now it's like tens of thousands of people. It's held in 00:20:35.280 --> 00:20:39.760 a massive convention center. Yes, there are researchers there, but there's a variety of 00:20:39.760 --> 00:20:45.680 people from across the tech industry who attend, but that is not what it was like back then. 00:20:45.680 --> 00:20:53.040 So in around … in 2005, it was more like 600 people thereabouts in total, 00:20:53.040 --> 00:20:58.320 and the main conference would be held every year in Vancouver, and then everybody at the 00:20:58.320 --> 00:21:04.049 conference would pile onto these buses, and we would all head up to Whistler for the workshops. 00:21:04.049 --> 00:21:04.071 WORTMAN VAUGHAN: Yep. 00:21:04.071 --> 00:21:12.240 WALLACH: So super

different to what's happening nowadays. It was my third time. 00:21:12.240 --> 00:21:16.960 I think that's right. I think it was my third time attending the conference. But it was my 00:21:16.960 --> 00:21:22.560 first time sharing a hotel room with other women. And I remember up at the workshops, 00:21:22.560 --> 00:21:27.840 up in Whistler, there were five of us sitting around in a hotel room, and we were talking 00:21:27.840 --> 00:21:33.360 about how amazing it was that there were five of us sitting around talking, women. And we, 00:21:33.360 --> 00:21:36.880 kind of, couldn't believe there were five of us. We're all PhD students at the time. And so 00:21:36.880 --> 00:21:40.560 we decided to make this list, and we started trying to figure out who the other women in 00:21:40.560 --> 00:21:45.760 machine learning were. And we came up with about 10 names, and we were kind of amazed that there 00:21:45.760 --> 00:21:50.400 were even 10 women in machine learning. We thought this was a huge number. We were very excited. And 00:21:50.400 --> 00:21:55.520 we started talking about how it might be really fun to just bring them all together sometime. 00:21:55.520 --> 00:22:02.240 So we returned from NeurIPS, and you and I ended up getting lunch to strategize. 00:22:02.240 --> 00:22:07.280 I still remember walking out of the department together to go get lunch and you were walking 00:22:07.280 --> 00:22:10.720 ahead of me. I can visualize the coat you were wearing as you were walking in front of 00:22:10.720 --> 00:22:16.800 me. And so we strategized a bit and ended up deciding, along with one of the other women, 00:22:16.800 --> 00:22:22.160 Lisa Wainer, to submit a proposal to the Grace Hopper conference for a session in 00:22:22.160 --> 00:22:26.240 which women in machine learning would give short talks about their research. 00:22:27.120 --> 00:22:32.000 We reached out to the 10 names that we had written down in the hotel room and 00:22:32.000 --> 00:22:35.520 through that process actually ended up finding out about more women in machine 00:22:35.520 --> 00:22:41.360 learning and eventually had something like 25 women listed on the final proposal. I 00:22:41.360 --> 00:22:45.440 think there's an email somewhere where one or another of us is saying to the other one, 00:22:45.440 --> 00:22:48.720 “Oh my gosh! I can't believe there are so many women in machine learning.” 00:22:50.080 --> 00:22:55.040 So we submitted this proposal, and ultimately, the proposal was rejected by the Grace Hopper 00:22:55.040 --> 00:23:01.200 conference. But we were so excited about the idea and just really invested in it by that point that 00:23:01.200 --> 00:23:07.920 we decided to hold our own co-located event the day before the Grace Hopper conference. And I’ve 00:23:07.920 --> 00:23:14.160 got to say, you know, 20 years later, I don't know what we were thinking. Like, that was a bold move 00:23:14.160 --> 00:23:21.120 on the part of three PhD students. And it turned out

to be a huge amount of work that we had to do 00:23:21.120 --> 00:23:22.711 entirely ourselves, as well. WORTMAN VAUGHAN: Yeah. 00:23:22.711 --> 00:23:29.600 WALLACH: We had no idea what we were doing. But the Grace Hopper folks very nicely connected us 00:23:29.600 --> 00:23:35.680 with the venue that the conference was going to be held at, and somehow, we managed to pull it off. 00:23:36.880 --> 00:23:42.880 Ultimately, that first workshop had around 100 women, and there was this … rather than just, 00:23:42.880 --> 00:23:45.840 like, a single short session, which is what we'd originally had in mind, 00:23:45.840 --> 00:23:50.800 we had this full day's worth of talks. I actually have the booklet of abstracts 00:23:50.800 --> 00:23:57.600 from all of those talks at my desk in the office. I still have that today. 00:23:57.600 --> 00:24:04.240 And it was just an amazing experience. WORTMAN VAUGHAN: Yeah, it was. And, you know, 00:24:04.240 --> 00:24:10.640 you mentioned how bold we were. I just, I really don't think that any of us at the time realized 00:24:10.640 --> 00:24:16.320 how bold we were being here, getting this workshop rejected and then saying, you know, 00:24:16.320 --> 00:24:21.360 no, we think this is important. We're going to do it anyway. On our own. As grad students. 00:24:21.360 --> 00:24:26.560 So I've already talked a little bit about some of the spaces that I was in throughout 00:24:26.560 --> 00:24:31.200 my career where there just weren't a lot of women around in the room with me. 00:24:32.560 --> 00:24:38.640 How had you experienced a lack of community or network of women in machine learning before the 00:24:38.640 --> 00:24:43.200 founding of WiML? And, you know, why do you think it's important to have that kind of community? 00:24:43.200 --> 00:24:48.560 WALLACH: So I felt it in a number of different ways. I think I mentioned a few minutes ago that, 00:24:48.560 --> 00:24:52.560 like, it was my third time at NeurIPS but my first time sharing a hotel room 00:24:52.560 --> 00:24:57.920 with another woman. But there were many places over the years where I’d felt this. 00:24:57.920 --> 00:25:04.000 So first, as an undergraduate. Then, I did a lot of free and open-source software development, 00:25:04.000 --> 00:25:08.960 and I was pretty involved in stuff to do with the Debian Linux distribution. And back then, 00:25:08.960 --> 00:25:13.200 the percentage of women involved in free and open-source software development was 00:25:13.200 --> 00:25:18.960 about 1 percent, 1.5%, and the percentage involved actually in Debian was even less 00:25:18.960 --> 00:25:25.760 than that. So that had led me and some others to start this Debian Women Project. And then, 00:25:25.760 --> 00:25:27.680 again, of course, I faced this in machine learning. 00:25:27.680 --> 00:25:31.520 I just didn't know that many other women in machine learning. I didn't … there weren't a 00:25:31.520 --> 00:25:36.400 large number of senior women, for example, to

look up to as role models. There weren't a large number 00:25:36.400 --> 00:25:43.600 of female PhD students. And this, kind of, made me sad because I was really excited about machine 00:25:43.600 --> 00:25:50.160 learning, and I hoped to spend my entire career in it. But because I didn't see so many other 00:25:50.160 --> 00:25:54.960 women around, particularly more senior women, that really made me question whether that would 00:25:54.960 --> 00:26:00.640 even be possible, and I just didn't know. Um, I think, you know, thinking about this, 00:26:00.640 --> 00:26:05.760 and I've obviously reflected on this a lot over the years, but I think having a diverse 00:26:05.760 --> 00:26:10.560 community in any area, be it free and open-source software development, be it machine learning, 00:26:10.560 --> 00:26:16.000 any of these kinds of things, is just so important for so many reasons. And some of those reasons 00:26:16.000 --> 00:26:21.040 are little things like finding people that you would feel comfortable sharing a hotel room with. 00:26:21.040 --> 00:26:24.640 But many of these things are bigger things that can then have, like, even, 00:26:24.640 --> 00:26:29.280 kind of, knock-on cumulative effects. Like feeling valued in the community, 00:26:29.280 --> 00:26:33.360 feeling welcome in the community, having role models, being able to, sort of, 00:26:33.360 --> 00:26:38.320 see people and say, “Oh, I want to be kind of like that person when I grow up; I could do this.” 00:26:38.320 --> 00:26:44.560 And then even just representation of different perspectives in the work itself is so important. 00:26:45.440 --> 00:26:49.120 The flip side of that is that there are a whole bunch of things that can go wrong if you don't 00:26:49.120 --> 00:26:55.440 have a diverse community. You can end up with gatekeeping, with toxic or unsafe cultures, 00:26:56.000 --> 00:27:00.400 obviously attrition as people just leave these kinds of spaces because they feel that they're 00:27:00.400 --> 00:27:05.680 not welcome there and won't be valued there. And then to that point of having representation 00:27:05.680 --> 00:27:10.480 of different perspectives, with a really homogeneous community, you can end up with, 00:27:10.480 --> 00:27:15.503 kind of, blind spots around the technology itself, which can then lead to harms. 00:27:15.503 --> 00:27:20.880 WORTMAN VAUGHAN: 100%. So did you ever imagine during all of this that WiML would 00:27:20.880 --> 00:27:26.702 still be around 20 years later and we would be sitting here on a podcast talking about this? 00:27:26.702 --> 00:27:31.200 WALLACH: [LAUGHS] No, absolutely not. I didn't even think that WiML would 00:27:31.200 --> 00:27:34.320 necessarily be around for a second year. I thought it was probably going to be, like, 00:27:34.320 --> 00:27:38.880 a one-off event. And I certainly don't think that I thought that I would still 00:27:38.880 --> 00:27:45.440 be involved in the machine learning community 20 years later, as well. So very unexpected. 00:27:46.400 --> 00:27:52.143 I’ve got a question for you, though. What do you remember most about that first workshop? 00:27:52.143 --> 00:27:54.560 WORTMAN VAUGHAN: I remember a lot of things. I remember that, you know, 00:27:54.560 --> 00:27:59.840 when we were planning this, we always really wanted the focus to be the research. And, 00:27:59.840 --> 00:28:02.080 you know, if you think back to what this first workshop looked like, 00:28:02.080 --> 00:28:08.800 it was a lot of us just giving talks or presenting posters about our own research to other people. 00:28:08.800 --> 00:28:13.040 And, you know, I remember thinking at the poster session, like, the vibe was just so 00:28:13.040 --> 00:28:17.760 much different and better, healthier really than other poster sessions I had been to. 00:28:17.760 --> 00:28:24.800 Everyone was so supportive and encouraging, but it really was all about the research. I 00:28:24.800 --> 00:28:30.000 also remember being blown away just walking into that conference room in the morning and seeing all 00:28:30.000 --> 00:28:35.280 of these women gathered in one place and knowing that somehow, we had actually made this happen. 00:28:35.280 --> 00:28:40.480 Um, I remember we also faced some challenges with the workshop early 00:28:40.480 --> 00:28:44.560 on. What are the challenges that stand out to you most? 00:28:44.560 --> 00:28:48.160 WALLACH: Yeah, so a lot of people really got it, 00:28:48.160 --> 00:28:52.480 right. And they were super supportive. So, for example, folks at Penn totally got it, 00:28:52.480 --> 00:28:56.960 and they actually funded a bunch of that first workshop. But others in 00:28:56.960 --> 00:29:01.360 the community didn't get it and didn't see the point, didn't see why it was necessary. 00:29:01.360 --> 00:29:05.600 I remember having dinner with one machine learning researcher and him telling me 00:29:05.600 --> 00:29:09.520 that he didn't think this kind of workshop was necessary because women's experiences were no 00:29:09.520 --> 00:29:15.280 different to men's experiences. And then later on in the conversation, he talked about—like, 00:29:15.280 --> 00:29:19.120 you know, this is, like, an hour and a half later or something—he talked about how he 00:29:19.120 --> 00:29:26.080 and a friend of his had gone to the bar at an all-women’s college and he'd felt so awkward 00:29:26.080 --> 00:29:30.400 and out of place. And I ended up pointing out to him [LAUGHS] that he just, kind of, 00:29:30.400 --> 00:29:36.800 explained to himself why we needed WiML. So, yeah, there were some people who didn't get it, 00:29:36.800 --> 00:29:40.240 and it took a lot of, sort of, talking to people and, kind of, explaining. 00:29:40.240 --> 00:29:40.791 WORTMAN VAUGHAN: Yep. 00:29:40.791 --> 00:29:43.680 WALLACH: Another challenge was figuring out how to fund it in 00:29:43.680 --> 00:29:48.080 an ongoing manner once we decided that we wanted to do this more than once. 00:29:48.080 -->

00:29:52.720 So as I said, Penn funded a lot of that first workshop, but that wasn't a sustainable model, 00:29:52.720 --> 00:29:56.720 and it wasn't going to be realistic for Penn to keep funding it. So in the end, 00:29:56.720 --> 00:30:01.520 we worked with Amy Greenwald to obtain a National Science Foundation grant 00:30:01.520 --> 00:30:06.880 that would cover a lot of costs, and we also received donations from other organizations. 00:30:06.880 --> 00:30:12.320 Um, a third challenge was figuring out where to hold the workshop given that we did want 00:30:12.320 --> 00:30:17.840 that focus to be on research. So the first two times, we held the workshop at the Grace Hopper 00:30:17.840 --> 00:30:21.920 conference, but we started to feel that that wasn't really the right venue given that we 00:30:21.920 --> 00:30:27.360 wanted that focus to be on research. So we ended up moving it to NeurIPS, and this had a bunch of 00:30:27.360 --> 00:30:32.160 benefits, some of which I don't think we'd even fully thought through when we made that decision. 00:30:32.160 --> 00:30:38.080 So one of the benefits was that attendees’ WiML travel funding—so we would give them 00:30:38.080 --> 00:30:43.680 this travel funding to enable them to pay the cost of attending WiML, stay in hotel rooms, 00:30:43.680 --> 00:30:48.960 all this kind of stuff—this would actually enable them to attend NeurIPS, as well, 00:30:48.960 --> 00:30:51.680 if we co-located with NeurIPS. WORTMAN VAUGHAN: Yep. 00:30:51.680 --> 00:30:58.560 WALLACH: Another main benefit was that we held WiML on the day before NeurIPS. 00:30:58.560 --> 00:31:02.800 So then throughout the rest of the conference, WiML attendees would 00:31:02.800 --> 00:31:08.863 see familiar faces throughout the crowd and wouldn't necessarily feel so alone. 00:31:08.863 --> 00:31:13.280 WORTMAN VAUGHAN: So you're talking about these challenges. How have these challenges changed 00:31:13.280 --> 00:31:18.160 over time? Or, you know, more broadly, can you talk about how the workshop and Women 00:31:18.160 --> 00:31:22.640 in Machine Learning as an organization as a whole, kind of, evolved over the years? I know 00:31:22.640 --> 00:31:30.000 that you served a term as the WiML president. WALLACH: Yeah. So it's changed a lot. So first, 00:31:30.000 --> 00:31:33.680 obviously, most importantly, it evolved from being, kind of, this one-off event where we 00:31:33.680 --> 00:31:40.800 were just seeing what would happen to being really a robust organization. And the first step in that 00:31:40.800 --> 00:31:46.960 was creating the WiML board. And, as you just said, I served as the first president of that. 00:31:46.960 --> 00:31:52.560 But there have been a bunch of other steps since then. And one of the things I want to flag about 00:31:52.560 --> 00:31:57.600 the WiML board was that this was really important because the board members could focus on the 00:31:57.600 --> 00:32:03.520 long-term health of the organization and these, sort

of, like, you know, things that spanned 00:32:03.520 --> 00:32:08.560 multiple years, like how to get sustainable funding sources, this kind of thing, versus 00:32:08.560 --> 00:32:13.840 the actual workshop organizers, who would focus on things like running the call for submissions 00:32:13.840 --> 00:32:20.240 and stuff like that. And being able to separate those roles made it really just reduce the burden 00:32:20.240 --> 00:32:26.000 on the workshop organizers meant that we could take this, kind of, longer-term perspective. 00:32:26.000 --> 00:32:31.360 Another really important step was becoming, officially becoming a non-profit. So that 00:32:31.360 --> 00:32:35.760 happened a few years ago. And again, it was the natural thing to do at that point 00:32:35.760 --> 00:32:41.600 in time and just another step towards creating this, sort of, durable, robust organization. 00:32:42.160 --> 00:32:47.120 But it's really taken on a life of its own. I'm honestly not super actively involved nowadays, 00:32:47.120 --> 00:32:53.040 which I think is fantastic. The organization doesn't need me. That's great. It's also wild 00:32:53.040 --> 00:32:57.040 to me that because it's been around for 20 years at this point that there are 00:32:57.040 --> 00:33:02.160 women in the field who don't know what it's like to not have WiML. 00:33:02.160 --> 00:33:07.520 So a bunch of other affinity groups got created. So Timnit Gebru cofounded Black 00:33:07.520 --> 00:33:12.400 in AI when she was actually a postdoc at Microsoft Research New York City. So you 00:33:12.400 --> 00:33:17.680 and I got to actually see the founding of that affinity group up close. And then now 00:33:17.680 --> 00:33:22.960 there are a ton of other affinity groups. So there's LatinX in AI; there's Queer in AI, 00:33:22.960 --> 00:33:29.423 Muslims in ML, Indigenous in AI and ML, New In ML, just to name a few. 00:33:29.423 --> 00:33:33.360 WORTMAN VAUGHAN: Yeah, and all of these are growing, too, every year. 00:33:33.360 --> 00:33:38.320 You know, this year, WiML had over 400 submissions. They accepted 250 to be 00:33:38.320 --> 00:33:41.023 presented. It's amazing. WALLACH: That's wild. 00:33:41.023 --> 00:33:45.280 WORTMAN VAUGHAN: Yeah, yep. And there's going to be a WiML presence this year actually at all 00:33:45.280 --> 00:33:49.040 three of the NeurIPS venues. So there's going to be a presence in Mexico City, 00:33:49.040 --> 00:33:56.000 in Copenhagen, and, of course, in San Diego for the main workshop. So it's pretty great. 00:33:56.000 --> 00:33:59.840 And, you know, on top of that, I think the organization now, as you were saying, 00:33:59.840 --> 00:34:07.280 is able to do so much more than just the workshop alone. So for instance, WiML now runs this 00:34:07.840 --> 00:34:13.280 worldwide mentorship program for women and nonbinary individuals in machine learning, 00:34:13.280 --> 00:34:19.280 where they're matched with a mentor and they can participate in these one-to-one mentoring meetings 00:34:19.280 --> 00:34:25.520 and seminars and panel discussions, which happens all throughout the year. I think they have about 00:34:25.520 --> 00:34:31.600 50 mentors signing up each year, but I'm sure they could always use more. Um, so it's just 00:34:31.600 --> 00:34:40.320 really amazing to look back and see how much the WiML community has done and how much it's grown. 00:34:40.320 --> 00:34:44.080 And, you know, on the one hand, I think that honestly, like, 00:34:44.080 --> 00:34:49.920 founding WiML was one of the things that I've done over the course of my career, 00:34:49.920 --> 00:34:52.750 if not the thing, that I am most proud of … 00:34:52.750 --> 00:34:54.480 WALLACH: Oh yeah, me, too. WORTMAN VAUGHAN: … to this day, 00:34:54.480 --> 00:34:59.120 but at the same time, like, we can't take credit for any of this. It's, like, a community effort. 00:34:59.120 --> 00:34:59.192 WALLACH: No. 00:34:59.192 --> 00:35:01.966 WORTMAN VAUGHAN: It's the community that has really kept us going … 00:35:01.966 --> 00:35:03.084 WALLACH: Yes. WORTMAN VAUGHAN: … for the last 20 years, 00:35:03.084 --> 00:35:03.099 WALLACH: Yes. 00:35:03.099 --> 00:35:06.920 WORTMAN VAUGHAN: … so it's great. Going to stop gushing now, but it's amazing. 00:35:06.920 --> 00:35:10.640 WALLACH: And it's not just WiML that's changed over the years. 00:35:10.640 --> 00:35:14.640 The entire industry has changed a ton, as well. 00:35:14.640 --> 00:35:19.840 How has your research evolved as a result of these changes to the entire field of AI 00:35:19.840 --> 00:35:24.943 and machine learning and also from your own change from academia to industry? 00:35:24.943 --> 00:35:28.880 WORTMAN VAUGHAN: It's a great question. You know, we've touched on this a little bit, but 00:35:28.880 --> 00:35:34.160 our research paths really evolved differently but ended up in these very similar places 00:35:34.160 --> 00:35:38.880 where we’re working on responsible AI, we're advocating for interdisciplinary approaches, 00:35:38.880 --> 00:35:44.720 incorporating techniques from HCI, and so on. And I think that part of this was because of shifts 00:35:44.720 --> 00:35:50.320 of the community and also what's happening in industry. Working in responsible AI in industry, 00:35:50.320 --> 00:35:55.040 there's definitely not ever a shortage of interesting problems to solve, right. 00:35:55.040 --> 00:36:00.640 And I think that for both of us, our research interests in recent years really have been driven 00:36:00.640 --> 00:36:06.800 by these really practical challenges that we're seeing. We were both involved early on in defining 00:36:06.800 --> 00:36:13.040 what responsible AI means within Microsoft, shaping our internal Responsible AI Standard. 00:36:13.040 --> 00:36:18.880 I led this internal companywide working group on AI transparency, which was focused both on 00:36:18.880 --> 00:36:24.800 model interpretability like we were talking about earlier but also other forms of transparency

like 00:36:24.800 --> 00:36:30.240 data sheets for datasets and the transparency notes that Microsoft now releases with all 00:36:30.240 --> 00:36:36.000 of our products. And at the same time, you are leading this internal working group on fairness. 00:36:36.000 --> 00:36:42.000 WALLACH: Yeah, taking on that internal working group was, kind of, a big transition point in my 00:36:42.000 --> 00:36:48.320 career. You know, when I joined Microsoft, I was focusing on computational social science and I 00:36:48.320 --> 00:36:55.280 was also entirely doing research and wasn't really that involved in stuff in the rest of the company. 00:36:55.280 --> 00:37:02.160 Then at the end of my first year at Microsoft, I attended the first Fairness, Accountability, 00:37:02.160 --> 00:37:07.680 and Transparency in Machine Learning workshop, which was co-located with NeurIPS. It was one of 00:37:07.680 --> 00:37:11.360 the NeurIPS workshops. And I got really excited about that and thought, great, 00:37:11.360 --> 00:37:16.800 I'm going to spend like 20% of my time, maybe one day a week, doing research on topics in the space 00:37:16.800 --> 00:37:22.800 of fairness and accountability and transparency. Um, that is not what ended up happening. 00:37:22.800 --> 00:37:27.920 Over the next couple of years, I ended up doing more and more research on responsible AI, 00:37:27.920 --> 00:37:32.320 you know, as you said, on topics to do with fairness, to do with interpretability. And 00:37:32.320 --> 00:37:38.640 then in early 2018, I was asked to co-chair this internal working group on fairness, 00:37:38.640 --> 00:37:42.000 and that was the point where I started getting much more involved 00:37:42.000 --> 00:37:47.840 in responsible AI stuff across Microsoft, so outside of just Microsoft Research. 00:37:47.840 --> 00:37:51.760 And this was really exciting to me because responsible AI was so new, 00:37:51.760 --> 00:37:55.920 which meant that research had a really big role to play. It wasn't like this was kind 00:37:55.920 --> 00:38:01.920 of an established area where folks in engineering and policy knew exactly what they were doing. And 00:38:01.920 --> 00:38:07.600 so that meant that I got to branch out from this very, sort of, research-focused work into much 00:38:07.600 --> 00:38:13.440 more applied work in collaboration with folks from policy, from engineering, and so on. 00:38:14.320 --> 00:38:20.560 Now, in fact, as well as being a researcher, I actually run a small applied science team, 00:38:20.560 --> 00:38:26.000 the Sociotechnical Alignment Center, or STAC for short, within Microsoft Research 00:38:26.000 --> 00:38:31.903 that focuses specifically on bridging research and practice in responsible AI. 00:38:31.903 --> 00:38:33.120 WORTMAN VAUGHAN: Yeah. Do you think that 00:38:33.120 --> 00:38:36.520 your involvement in WiML has played a role in this work? 00:38:36.520 --> 00:38:43.600 WALLACH: Yes, definitely. [LAUGHS] Yeah, without a doubt. So particularly when

00:38:43.600 --> 00:38:48.560 working on topics related to fairness, I've ended up focusing a bunch on stuff 00:38:48.560 --> 00:38:53.920 to do with marginalized groups as part of my responsible AI work. 00:38:53.920 --> 00:38:58.000 So there's been this, sort of, you know, focus on marginalized groups, 00:38:58.000 --> 00:39:01.520 particularly women, in the context of machine learning and with my WiML, 00:39:01.520 --> 00:39:06.240 kind of, work and then in my research work thinking about fairness, as well. 00:39:07.040 --> 00:39:12.640 The other way that WiML has really, sort of, affected what I do is that I work with 00:39:12.640 --> 00:39:18.000 a much more varied group of people nowadays than I did back when I was just focusing on, 00:39:18.000 --> 00:39:22.160 kind of, machine learning, computational social science, and stuff like that. And 00:39:22.160 --> 00:39:26.783 many of my collaborators are people that I've met through WiML over the years. 00:39:26.783 --> 00:39:29.760 WORTMAN VAUGHAN: And, of course, there has been another big shift 00:39:29.760 --> 00:39:33.840 within industry recently, which is just all the excitement around generative AI. 00:39:33.840 --> 00:39:36.680 Can you say a bit about how that has changed your research? 00:39:36.680 --> 00:39:40.720 WALLACH: OK, yeah. So this is another big one. There are so many ways that 00:39:40.720 --> 00:39:46.800 this changed my work. One of the biggest ways, though, is that generative AI systems are now 00:39:46.800 --> 00:39:51.680 everywhere. They're being used all over the place for all kinds of things. And, 00:39:51.680 --> 00:39:55.120 you know, you see all these news headlines about GenAI systems, 00:39:55.760 --> 00:40:00.640 you know, diagnosing illnesses, solving math problems, and writing code, stuff like that. 00:40:00.640 --> 00:40:08.160 And also headlines about various different risks that can occur when you're using generative AI. 00:40:08.160 --> 00:40:13.920 So fabricating facts, memorizing copyrighted data, generating harmful content, you know, 00:40:13.920 --> 00:40:18.880 these kinds of things. And with all this attention, it's really natural to ask, what 00:40:18.880 --> 00:40:25.840 is the evidence behind these claims? So where is this evidence coming from, and should we trust it? 00:40:26.880 --> 00:40:32.640 It turns out that much of the evidence comes from GenAI evaluations that involve measuring 00:40:32.640 --> 00:40:37.920 the capabilities, the behaviors, and the impacts of GenAI systems, 00:40:37.920 --> 00:40:44.160 but the current evaluation practices that are often used in the space don't 00:40:44.160 --> 00:40:49.200 really have as much scientific rigor as we would like, and that's, kind of, a problem. 00:40:49.200 --> 00:40:54.240 So one of the biggest challenges is that the concepts of interest when people are, sort of, 00:40:54.240 --> 00:41:00.000 doing these GenAI evaluations—so things like diagnostic ability, memorization, 00:41:00.000 --> 00:41:06.400 harmful content, concepts like that—are much more abstract than the concepts like prediction 00:41:06.400 --> 00:41:12.960 accuracy that underpinned machine learning evaluations before the generative AI era. 00:41:12.960 --> 00:41:18.560 And when we look at these new concepts that we need to be able to focus on in order to evaluate 00:41:18.560 --> 00:41:23.680 GenAI systems, we see that they're actually much more reminiscent of these abstract contested 00:41:23.680 --> 00:41:27.920 concepts—these, kind of, fuzzy, squishy concepts—that are studied in the social 00:41:27.920 --> 00:41:34.480 sciences. So things like democracy and political science or personality traits and psychometrics. 00:41:34.480 --> 00:41:38.960 So there's really that, sort of, connection there to these, kind of, squishier things. 00:41:38.960 --> 00:41:42.960 So when I was focusing primarily on computational social science, 00:41:42.960 --> 00:41:48.480 most of my work was focused on developing machine learning methods to help social scientists measure 00:41:48.480 --> 00:41:54.560 abstract contested concepts. So then when GenAI started to be a big thing and I saw all of these 00:41:54.560 --> 00:41:59.760 evaluative claims involving measurements of abstract concepts, it seemed super clear to 00:41:59.760 --> 00:42:04.560 me that if we were going to actually be able to make meaningful claims about what AI can 00:42:04.560 --> 00:42:10.000 do and can't do, we're going to need to take a different approach to GenAI evaluation. 00:42:10.000 --> 00:42:14.560 And so I ended up, sort of, drawing on my computational social science work around 00:42:14.560 --> 00:42:22.880 measurement and I started advocating for adopting a variant of the framework that social scientists 00:42:22.880 --> 00:42:30.720 use for measuring abstract contested concepts. And my reason for doing this was that I believed—I 00:42:30.720 --> 00:42:37.440 still believe—that this is an important way to improve the scientific rigor of GenAI evaluations. 00:42:37.440 --> 00:42:42.800 You know all of this, of course, because you and I, along with a bunch of other collaborators at 00:42:42.800 --> 00:42:47.280 Microsoft Research and Stanford and the University of Michigan published 00:42:47.280 --> 00:42:53.600 a position paper on this framework entitled “Evaluating GenAI Systems is a Social Science 00:42:53.600 --> 00:42:58.480 Measurement Challenge” at ICML [International Conference on Machine Learning] this past summer. 00:42:59.440 --> 00:43:02.080 What are you excited about at the moment? 00:43:02.080 --> 00:43:07.920 WORTMAN VAUGHAN: Yeah, so lately, I have been spending a lot of time thinking about AI and 00:43:07.920 --> 00:43:14.320 critical thought: how can we design AI systems to support appropriate reliance, preserve 00:43:14.320 --> 00:43:19.840 human agency, and really encourage critical engagement on the part of the human, right? 00:43:19.840 --> 00:43:26.480 So this is an area where I

think we actually have a huge opportunity, but there are also huge risks. 00:43:26.480 --> 00:43:32.960 If I think about my most optimistic possible vision of the future of AI—which is not 00:43:32.960 --> 00:43:38.480 something that is easy for me to do, as I'm not a natural optimist, as you know—it 00:43:39.120 --> 00:43:47.200 would be a future in which AI helps people grow and flourish, in which it, kind of, enriches 00:43:47.200 --> 00:43:54.400 our own human capabilities. It deepens our own human thinking and safeguards our own agency. 00:43:54.400 --> 00:44:00.080 So in this future, you know, we could build AI systems that actually help us brainstorm and 00:44:00.080 --> 00:44:04.800 learn new knowledge and skills, both in formal educational settings and in our day-to-day work, 00:44:04.800 --> 00:44:09.520 as well. But I think we're not going to achieve this future by default. 00:44:09.520 --> 00:44:12.800 It's something that we really need to design for if we want to get there. 00:44:12.800 --> 00:44:18.063 WALLACH: You mentioned that there are risks. What are the risks that you can see here? 00:44:18.063 --> 00:44:22.480 WORTMAN VAUGHAN: Yeah, there's so much at stake here. You know, in the short term, there are 00:44:22.480 --> 00:44:27.760 things like overreliance—depending on the output of an AI system even when the system's wrong. This 00:44:27.760 --> 00:44:33.920 is something that I've worked on a bunch myself. There's a risk of loss of agency or the ability 00:44:33.920 --> 00:44:39.920 to make and execute independent decisions and to ensure that our outcomes of AI systems are 00:44:39.920 --> 00:44:46.240 aligned with personal or professional values of the humans who are using those systems. This is 00:44:46.240 --> 00:44:51.680 something that I've been looking at recently in the context of AI tools for journalism. There's 00:44:51.680 --> 00:44:58.720 diminished innovation, by which I mean a loss of creativity or diversity of ideas. 00:44:58.720 --> 00:45:04.640 You know, longer term, we risk atrophied skills—people just losing or simply never 00:45:04.640 --> 00:45:10.080 developing helpful skills for their career or their life because of prolonged use of AI 00:45:10.080 --> 00:45:15.760 systems. The famous example that people often bring up here is pilots losing the ability 00:45:15.760 --> 00:45:21.040 to perform certain actions in flight because of dependence on autopilot systems. And I think we're 00:45:21.040 --> 00:45:28.160 already starting to see the same sort of thing happen across all sorts of fields because of AI. 00:45:28.160 --> 00:45:32.720 And, you know, finally, another risk that I'll mention that seems to resonate with 00:45:32.720 --> 00:45:36.240 a lot of folks I talk to is what I would just call loss of joy, 00:45:36.240 --> 00:45:41.920 right. What happens when we are delegating to AI systems the parts of our activities 00:45:41.920 --> 00:45:47.320 that we really take pleasure and find this satisfaction in doing ourselves. 00:45:47.320 --> 00:45:52.880 WALLACH: So then as a community, what should we be doing if we're worried about these risks? 00:45:52.880 --> 00:45:55.200 WORTMAN VAUGHAN: Yeah, I mean, I think this is going to have 00:45:55.200 --> 00:45:59.200 to be a big community effort if we want to achieve this. This is a big 00:45:59.200 --> 00:46:03.280 goal. But there are a few places I think we especially need work. 00:46:03.280 --> 00:46:10.320 So I think we need generalized principles and practices for AI system builders for how they 00:46:10.320 --> 00:46:17.120 can build AI systems in ways that promote human agency and encourage critical thought. We also 00:46:17.120 --> 00:46:23.280 need principles and practices for system users. So how do we teach the general population to 00:46:23.280 --> 00:46:29.760 use AI in ways that amplify their skills and capabilities and help them learn new things? 00:46:30.480 --> 00:46:35.520 And then, you know, close to your heart, I'm sure, I think that we need more work on 00:46:35.520 --> 00:46:42.080 measurement and evaluation, right. We are once again back to these squishy human properties. 00:46:42.640 --> 00:46:47.840 You know, I mentioned I've done some work on overreliance in generative AI systems, 00:46:47.840 --> 00:46:54.160 and I started there because on the grand scale of risks here, overreliance is something that 00:46:54.160 --> 00:46:59.920 is relatively easy to measure, at least in the short term. But how do we start 00:46:59.920 --> 00:47:05.040 thinking about measuring people's critical thinking when using AI across all sorts of 00:47:05.040 --> 00:47:11.120 contexts and at scale and over long-time horizons, right? How do we measure the, 00:47:11.120 --> 00:47:17.200 sort of, longitudinal effect of AI systems just on our critical thought as a population? 00:47:17.200 --> 00:47:20.640 And by the way, if anyone listening is going to be at the WiML workshop, 00:47:20.640 --> 00:47:24.800 I'll actually be giving a keynote on this topic. And this is something I'm 00:47:24.800 --> 00:47:30.800 just incredibly excited about because first, I'm incredibly excited about this topic, but also, 00:47:30.800 --> 00:47:36.000 in the whole 20 years of WiML, I've given opening remarks and similar several times, 00:47:36.000 --> 00:47:38.880 but this is actually the very first time that I will be talking about 00:47:38.880 --> 00:47:44.080 my own research there. So this is like my dream. I'm thrilled that this is happening. 00:47:44.080 --> 00:47:48.720 WALLACH: That's awesome. Oh, that's so exciting. Excellent. 00:47:48.720 --> 00:47:53.280 So one last question for you. If you could go back and talk to yourself 00:47:53.280 --> 00:47:57.023 20 years ago and give yourself some advice, what would you say? 00:47:57.023 --> 00:48:01.280 WORTMAN VAUGHAN: Yeah, OK, I've thought about this one a bit over the past week, 00:48:01.280 --> 00:48:04.640 and there are three things here I want to mention.

00:48:05.520 --> 00:48:11.920 So first, I would tell myself to be brave about speaking up. You know I'm about as introverted as 00:48:11.920 --> 00:48:17.920 it gets and I'm naturally very shy, and this has always held me back. It still holds me back now. 00:48:18.880 --> 00:48:23.680 It was really embarrassingly late in my career that I decided to do something about this and 00:48:23.680 --> 00:48:28.960 start to develop strategies to help myself speak up more. And eventually, it started to grow into 00:48:28.960 --> 00:48:34.000 something that's a little bit more natural. WALLACH: What kind of, um, 00:48:34.000 --> 00:48:35.823 what kind of strategies? 00:48:35.823 --> 00:48:41.280 WORTMAN VAUGHAN: Yeah, so you know, one example is I use a lot of notes. For this podcast, 00:48:41.280 --> 00:48:46.720 I have a lot of notes here. I'm a big notes person, and things like that really help me. 00:48:46.720 --> 00:48:50.400 The second thing that I would tell myself is to, you know, 00:48:50.400 --> 00:48:54.800 work on the problems that you really want to see solved. As researchers, 00:48:54.800 --> 00:49:01.360 we have this amazing freedom to choose our own direction. And early on, you know, a lot of the 00:49:01.360 --> 00:49:05.920 problems that I worked on were problems that I really enjoyed thinking about on a day-to-day 00:49:05.920 --> 00:49:12.480 basis. It was a lot of fun. They were like little math puzzles to me. But I often found that, 00:49:12.480 --> 00:49:16.880 you know, when I would be at conferences and people would ask me about my work, I didn't 00:49:16.880 --> 00:49:22.640 really want to talk about these problems. I just in some sense, you know, I had fun doing it, but 00:49:22.640 --> 00:49:27.840 I didn't really care. I wasn't passionate about it. I didn't care that I had solved the problem. 00:49:28.400 --> 00:49:32.880 And so once, many years ago now, when I was thinking about my research agenda, 00:49:32.880 --> 00:49:36.800 I got some good advice from our former lab director, Jennifer Chayes, 00:49:36.800 --> 00:49:42.800 who suggested that I go through my recent projects and sort them into projects where 00:49:42.800 --> 00:49:47.760 I really liked working on them—it was a fun experience day-to-day—and projects 00:49:47.760 --> 00:49:52.080 that I liked talking about after the fact and, kind of, felt good about the results 00:49:52.080 --> 00:49:56.560 and then see where the overlap is. And this is something that, like, it kind of sounds, 00:49:56.560 --> 00:50:01.360 kind of, obvious when I say it now, but at the time, it was really eye-opening for me. 00:50:01.360 --> 00:50:05.680 WALLACH: That's so cool. And now I, kind of, want to do that with all of my projects, 00:50:06.400 --> 00:50:11.840 particularly at the moment. I actually just took five months, as you know, five months off of work 00:50:11.840 --> 00:50:16.400 for parental leave because I just had a baby. And so I'm, sort of, taking a big, kind of, inventory 00:50:16.400 --> 00:50:22.000 of everything as I get back into all of this now, and I love this idea. I think this is really cool. 00:50:22.000 --> 00:50:25.680 WORTMAN VAUGHAN: It's changed really my whole approach to research. Like, 00:50:25.680 --> 00:50:28.640 you know, we were talking about this, but most of the work I do now is more 00:50:28.640 --> 00:50:33.600 HCI than machine learning because I found that the problems that really motivate me, 00:50:33.600 --> 00:50:38.640 that I want to be talking to people about at conferences, are the people problems. 00:50:38.640 --> 00:50:42.160 The third piece of advice I would give myself 00:50:42.160 --> 00:50:47.520 is that you should bring more people into your work, right. 00:50:47.520 --> 00:50:55.040 So there's this kind of vision on the outside of research being this solo endeavor, 00:50:55.040 --> 00:51:02.880 and it can feel so competitive at times, right. We all feel this. But time and time again, I've seen 00:51:02.880 --> 00:51:07.440 that the best research comes from collaborations and from bringing people together with diverse 00:51:07.440 --> 00:51:14.080 perspectives who can challenge each other in a way that is respectful but makes the work better. 00:51:14.080 --> 00:51:17.560 Is there advice that you would give to your former self of 20 years ago? 00:51:17.560 --> 00:51:20.560 WALLACH: Yeah. OK. So I've also been thinking about this a bunch 00:51:20.560 --> 00:51:25.680 over the past week. There's actually a lot of advice I think I would give my former self, 00:51:25.680 --> 00:51:28.480 [LAUGHS] but there are three things that I keep coming back to. 00:51:28.480 --> 00:51:33.680 OK, so first—and this is similar to your second point—push for doing the work that you find to 00:51:33.680 --> 00:51:40.160 be most fulfilling even if that means taking a nontraditional path. So in my case, I've always 00:51:40.160 --> 00:51:44.960 been interested in the social sciences. Back when I was a student, you know, even when I was a PhD 00:51:44.960 --> 00:51:50.960 student, doing research that combined computer science and the social sciences just wasn't 00:51:50.960 --> 00:51:55.520 really a thing. And so as a result, it would have been really easy for me to just be like, 00:51:55.520 --> 00:51:59.840 “Oh well, I guess that isn't possible. I'll just focus on traditional computer science problems.” 00:51:59.840 --> 00:52:04.800 But that's not what I ended up doing. Instead, and often in ways that made my career, 00:52:04.800 --> 00:52:09.600 kind of, harder than it probably would have been otherwise, I ended up pushing. 00:52:09.600 --> 00:52:12.480 I kept pushing, and in fact, I keep pushing, even nowadays, 00:52:12.480 --> 00:52:17.680 to bring these things together—computer science and the social sciences—in an interdisciplinary 00:52:17.680 --> 00:52:23.280 fashion. And this hasn't been easy. But cumulatively, the effect has been that I've 00:52:23.280 --> 00:52:28.560 been able to do much

more impactful work than I think I would have been able to do otherwise, 00:52:28.560 --> 00:52:35.200 and the work I've done, I’ve just enjoyed so much more than would otherwise have been the case. 00:52:35.200 --> 00:52:41.680 OK, so second, be brave and share your work. So this is actually advice for my current self 00:52:41.680 --> 00:52:46.783 and my former self, as this is something that I definitely still struggle with. 00:52:46.783 --> 00:52:49.680 WORTMAN VAUGHAN: As do I, you know, and actually, I think it's funny to hear 00:52:49.680 --> 00:52:53.160 you say this because I would say that you are much better at this than I am. 00:52:53.160 --> 00:52:59.520 WALLACH: I still, I think I have a lot of work to do on this one. Yeah, it's hard. It's really hard. 00:52:59.520 --> 00:53:02.240 As you know, I am a perfectionist, and this is good in some ways, 00:53:02.240 --> 00:53:06.400 but this is also bad in other ways. And one way in which this is bad is that I tend to be 00:53:06.400 --> 00:53:12.800 really anxious about sharing and publicizing my work, especially when I feel it's not perfect. 00:53:12.800 --> 00:53:19.920 So as an example, I wrote this massive tutorial on computational social science for ICML in 2015, 00:53:19.920 --> 00:53:24.000 but I never put the slides … and I wrote a whole script for it … I never put the slides or the 00:53:24.000 --> 00:53:29.040 script online as a resource for others because I felt it needed more work. And I actually went 00:53:29.040 --> 00:53:34.240 back and looked at it earlier this year, when we were working on the ICML paper, 00:53:34.240 --> 00:53:38.800 and I was stunned because it's great. Why didn't I put this online? All these things that I thought 00:53:38.800 --> 00:53:43.280 were problems 10 years ago, no, they're not a big deal. I should have just shared it. 00:53:43.280 --> 00:53:48.480 As another example, STAC, my applied science team, was using LLMs as part of our approach 00:53:48.480 --> 00:53:55.520 to GenAI evaluation back in 2022, way before the sort of “LLM-as-a-judge” paradigm was widespread. 00:53:55.520 --> 00:54:00.000 But I was really worried that others would think negatively of us for doing this, so we 00:54:00.000 --> 00:54:04.400 didn't share that much about what we were doing, and I regret that because we missed out on an 00:54:04.400 --> 00:54:11.440 opportunity to kick off an industrywide discussion about this, sort of, LLM-as-a-judge paradigm. 00:54:11.440 --> 00:54:17.680 OK, so then my third point is that the social side of research is just as valuable as the technical 00:54:17.680 --> 00:54:22.480 side. And by this, I'm actually not talking about social science and computer science. 00:54:22.480 --> 00:54:26.880 I actually mean that the how of doing research, including who you talk to, who you collaborate 00:54:26.880 --> 00:54:33.120 with, and how you approach those interactions is just as important as the research itself. 00:54:33.120 --> 00:54:38.320 As a PhD student, I felt really bad about spending time socializing with other researchers, 00:54:38.320 --> 00:54:42.640 especially at conferences, because I thought that I was supposed to be listening to talks, 00:54:42.640 --> 00:54:48.080 reading papers, and discussing technical topics with researchers and not socializing. But in 00:54:48.080 --> 00:54:52.960 hindsight, I think that was wrong. Many of those social connections have ended up 00:54:52.960 --> 00:54:58.000 being incredibly valuable to my research, both because I've ended up collaborating with and 00:54:58.000 --> 00:55:02.560 in some cases even hiring the people who I first got to know socially ... 00:55:02.560 --> 00:55:02.867 WORTMAN VAUGHAN: Yeah. 00:55:02.867 --> 00:55:04.800 WALLACH: … but also because the friendships that I've built, 00:55:04.800 --> 00:55:09.520 like our friendship, for example, have served as a crucial support network over the years, 00:55:09.520 --> 00:55:12.863 especially when things have felt particularly challenging. 00:55:12.863 --> 00:55:17.280 WORTMAN VAUGHAN: Yeah, absolutely. I agree with all of that so much. 00:55:17.280 --> 00:55:23.040 And with that, I will say thank you so much for doing this podcast with me today. 00:55:23.040 --> 00:55:24.063 WALLACH: Thank you. 00:55:24.063 --> 00:55:28.000 WORTMAN VAUGHAN: It was a lot of fun to reflect on the last 20 years of WiML, but also the last 00:55:28.000 --> 00:55:34.480 20 years of our careers and friendship and all of this, so it's great, and I never would have agreed 00:55:34.480 --> 00:55:40.240 to do this if it had been with anyone but you. WALLACH: Likewise. [LAUGHS] 00:55:40.240 --> 00:55:43.760 So thank you, everybody, for listening to us, 00:55:43.760 --> 00:55:49.840 and hopefully some of you will join for the 20th annual workshop for Women in Machine Learning, 00:55:49.840 --> 00:55:55.360 which is taking place on Dec. 2. And of course, Jenn and I will both be there in 00:55:55.360 --> 00:56:00.640 person. We'll also be at NeurIPS afterwards. So feel free to reach out to us if you want 00:56:00.640 --> 00:56:06.880 to chat with us or to learn more about anything that we covered here today. 00:56:06.880 --> 00:56:10.421 [MUSIC] 00:56:10.421 --> 00:56:13.520 OUTRO: 00:56:13.520 --> 00:56:17.520 You’ve been listening to Ideas, a Microsoft Research Podcast. Find 00:56:17.520 --> 00:56:22.960 more episodes of the podcast at aka.ms/researchpodcast. 00:56:22.960 --> 00:56:23.739 [MUSIC FADES]