Transcript
Announcer: Hello, and welcome to Screaming in the Cloud with your host, Cloud Economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of Cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.
Corey: This episode is sponsored in part byLaunchDarkly. Take a look at what it takes to get your code into production. I’m going to just guess that it’s awful because it’s always awful. No one loves their deployment process. What if launching new features didn’t require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren’t what you expect? LaunchDarkly does exactly this. To learn more, visitlaunchdarkly.com and tell them Corey sent you, and watch for the wince.
Corey: If your mean time to WTF for a security alert is more than a minute, it's time to look at Lacework. Lacework will help you get your security act together for everything from compliance service configurations to container app relationships, all without the need for PhDs in AWS to write the rules. If you're building a secure business on AWS with compliance requirements, you don't really have time to choose between antivirus or firewall companies to help you secure your stack. That's why Lacework is built from the ground up for the Cloud: low effort, high visibility and detection. To learn more, visit lacework.com.
Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by Laura Thompson, VP of platform engineering at Fastly. Laura, thank you for joining me.
Laura: Thank you for having me on.
Corey: So, Fastly is generally considered to be one of the definitive names in the CDN world. Is that an accurate description of what Fastly is, or is it perceived internally as, “Well, we are a CDN, but there's also much more to it than that.” I always want to make sure that my understanding of the company isn't based upon things that are no longer completely true.
Laura: So, we like to describe ourselves as an edge cloud platform because we've gone beyond the CDN. We are now shipping these products like Compute@Edge—which is essentially serverless only really, really fast because it runs at the edge—and a suite of security products. So it's more than just a CDN. And I think we're becoming more and more general as cloud platforms go.
Corey: Credit where due, I had a customer a while back who was using Fastly as a CDN and had been using a bunch of these edge compute things, and they said, “Oh, yeah, we can't move that off on to a competitor at this point because of all that logic that's there.” And I said, “Oh, so it's a locked-in story?” And they looked at me as if I were simple and said, “No. Because it's awesome, and nothing else is quite like it.” “Well, yeah, I guess there is a certain lock-in story around building something awesome that other people haven’t.”
Laura: That's actually really interesting. We've been talking about—this the product folks that work because the perception that I have is that people choose a particular cloud provider—whether it's general compute, or whether it's CDN, or whatever it is, edge cloud stuff—for the unique features. For the most part, people aren’t really interested in the lowest common denominator stuff, unless they have a very straightforward use case, and people have less and less straightforward use cases over time. So, they want to use the unique features; they want to know what's special, what can I only do here, and that's the basis of their purchasing decision a lot of the time.
Corey: Absolutely. Any sort of cloud comparison analysis between which vendor we're going to pick that breaks down to, “Well, what size of instance is going to cost how much?” And then trying to equate that, like, that is so far from the relevant part of the story when you're doing vendor selection in a modern era.
Laura: Right, exactly. It's interesting for me because I've been on the other side of the desk a lot. My last job at Mozilla, I was partially responsible for making CDN buying choices, so I have pretty good insight into what goes into that. It's interesting to think about.
Corey: It really is. In fact, to that end, you were—in fact, you are currently a member of the board of trustees of the Internet Society. What is the Internet Society, first off?
Laura: So, the Internet Society is about making sure that people have access, equality of opportunity, and that standards are kept open. So, the Internet Society essentially funds the IETF, which is responsible for most of the standards that run the internet. And that is sort of the primary mission.
Corey: I feel like on some level, the default response to that is, “Wait a second, the internet has standards? Since when?”
Laura: [laugh]. Right. Well, it does. I mean, TLS and HTTP would be the two things that people would think of, I would expect. And if those were not standardized, we wouldn't be here having this conversation.
Corey: Exactly. For those who didn't grow up in the ’80s ’90s, et cetera, watching these things evolve, the things that we take for granted today, like different networks can talk to one another all comes down to interoperability around shared standards. But I remember the dark ages, where if you were on CompuServe, you couldn't talk to the people who were on AOL directly. And over time, those walls started breaking down, and a lot of work of the standards bodies like the Internet Society and the IETF are the reason why. It's one of those boring governance things that happens underneath the hood so no one has to think about it. But the reason no one has to think about it is because people like you are doing the hard work of making it that way.
Laura: That’s right. And standards is a thing I've been passionate about for a long time. It is one of the reasons that I was at Mozilla for so long. It's one of the reasons I'm at Fastly, which has, I think, a similar role on the other side. You know, if Mozilla is thinking about how do we make clients use open standards, Fastly is thinking about how do we keep the internet open, and standard, and useful, and safe for everybody?
And [ISoc 00:06:00] and IETF are obviously, in the position of figuring out how things talk to each other. And it seems like maybe this is a solved problem, except it's not. One of the things that's happened on the internet—which as you can see over the last few years is that we're actually getting less and less open. So, we have more walled gardens, we have the internet being dominated by a few really large vendors, and they have a lot of power and standards. If you have the vast majority of the market share in a particular vertical, then you get to dictate what the standards are, and you can do that in a fairly anti-competitive way. I'm deliberately not naming any names here.
Corey: Oh, absolutely. I wouldn't expect you to, but I sure can. There's a reason that I have a podcast here where I control the RSS feed that I can point wherever I need it to live; there's a reason I have an email newsletter; there's a reason I don't have a Facebook page, to be very direct, where it's not about de-platforming in the censorship sense, but more along the lines of if I have built an audience on a platform, and then that platform decides that its business strategy is going to shift—something like Medium, for example—then suddenly, I am beholden to the whims of that provider unless I want to start over from scratch again. That's why it's always been so important for me to build my audience on something that was much more agnostic, not because I'm posting garbage that should be taken off the internet—well, most weeks—but rather because I don't want other people making my business decisions for me.
Laura: Yeah, so my interest in the internet is not about de-platforming, it's about platforming. It's making sure that people have access and the ability to build awesome things as much as they can.
Corey: Absolutely. But you're also a VP of engineering, which I feel like if I had asked you a year and a half ago, “What does it mean to be a VP of engineering?” You would have given me an answer. And now I feel like if I asked that same question in these uncertain times, or these unprecedented times, depending upon framing, you might have a different answer from that. Is that accurate?
Laura: Yeah, I think it's definitely been—I’m really tired of the word unprecedented. It's been a year. [laugh]. It's been a heck of a year.
Corey: It really ha—it feels like it's been longer than that.
Laura: Yeah. So, I started at Fastly on the ninth of March, 2020, which was a week after Fastly closed the offices for COVID. So, I have never set foot in an office, other than in the interview process. And I haven't met almost any of my coworkers. It's super strange, and it's really hard to build those trust relationships without having met anybody.
So that's kind of the first thing, and obviously, you would have a different experience if you'd been there and seeing the change, but this is all I have known in this role. So it's actually pretty interesting.
Corey: I know that there's a lot of zeitgeist awareness around what it's like to be an employee in a pandemic now and being full remote and the rest, but something that doesn't get discussed very much is, how does leadership change when suddenly you aren't able to gather people in a room and have a conversation and hash something out when everyone's remote?
Laura: Right. So, the funny thing is, that part hasn't changed very much. And I will say—for me, I've been a remote employee for about 15 years. And this is not what remote is normally like. I think it's part of the, sort of, important takeaway for people who might be doing it for the first time.
There's a big difference between choosing it and having it thrust upon you, for one thing. When I give people advice on how to work remotely successfully, it's always about, have good boundaries, have a dedicated space, and make sure that you start work at a particular time, finish work at a particular time so that you can walk away and de-stress. And because a lot of people are kind of trapped in the houses, they can’t have as good boundaries. They might have a one-bedroom apartment in San Francisco, and it's all in one room, and their spouse is there trying to work as well, and maybe they have a dog and a baby. It is not a typical remote working experience.
Similarly, it's not the typical remote leadership experience because remote is great. It also is important to meet with people once in a while, and not for meetings—not all sitting around a whiteboard talking through a bunch of dot points--but for the part that is really hard to do remote, which is the relationship-building. To me, you are trying to get to know people so that when things go wrong, they say, “Oh, I know Laura. I know what she's like. I trust her to do this right.”
And when I’ve never met you, it’s a little harder for people to do that; everybody has a little work a little harder. And when I say a little harder that's in the face of all of the cognitive load that we're already under because it's been, as you say, a heck of a year.
Corey: Right. And there are a lot of negative examples about how all this stuff looks terrible when people get it wrong. For example, “Oh, you're not spending an hour commuting every day; now you can spend that time working.” Or, “We have a policy of turning your webcam on, which is a fancy way of saying I'm inviting myself into your home so I can critique it. Excuse me, it's not acceptable that your infant is crying.” At some point, you hear these stories, and it's the biggest gap we have is the ability to strangle people over the internet now because that's horrifying.
Laura: I know. We have done a lot, I think, to make that comfortable for people. I think part of it is, Fastly has been at least sort of, half remote for a long time. [unintelligible 00:10:50], like remote-first companies are good to work for I will say, in general. But making sure that people know that it's okay if you have to have your camera off, or if you have to have a baby on your lap.
In fact, the thing I found is there is nothing quite like a meeting that has a cat, or a baby, or a puppy. And it lightens the mood, it helps people talk to each other. So we're all surrounded by our emotional service babies, and cats, and dogs. It's quite funny, really. I'm going to tell a story, which is—people talk about CEOs, and we have a CEO named Joshua Bixby, who has a really lovely human being. And when the pandemic started, he started a weekly meeting where he would read picture books to people's kids.
Corey: That's an amazing idea and in fact, I’m debating stealing it and claiming I came up with it myself, except that we're doing this on a recording, and suddenly my team will know when this comes out.
Laura: Yeah. I just thought it was an incredibly nice thing to do. And making sure that people knew that we're all in it with our families, we're all stuck at home with kids and whatever, and let's try and make the best of it and get to know each other a little bit better.
Corey: On the one hand, I absolutely agree with the sentiment and the place it comes from. On the other, I've got to say I have an anti-authority streak in a big way. My single biggest stumbling block, along with my personality, back when I was an employee. I mean, my last job was at a regulated finance company. You can imagine how well that worked out for me.
But authority is a problem for me, so whenever I hear, “Oh, go ahead and bring your family into this social gathering we're doing,” my immediate knee-jerk response is, “Is this required?” And that's not helpful as far as the sentiment goes, but it was there. That was my initial flare-up reaction. And I'm always hyper-aware of—now that I managed before myself of being sure to never present as anything other than if you want to.
Laura: Yes. Yes, it's very much been that way here. I too have an anti-authority streak, which is a funny thing to say when you end up in leadership. But that's why, by the way. And I think sort of compulsive joining-ness annoys a lot of people. I'm not one of those people that it annoys. I like hanging out with other people. I'm really extroverted, which may surprise you in an engineer and someone who chooses to work remote, but I like people, right?
I like hanging out with people; happy to hang out with everybody, but I know that not everybody wants to and that is 100 percent okay and you have to be so clear that that's okay. Everybody is kind of walking their own road through this thing. And for some people it’s… [sigh] some people are actually pretty happy being on their own, doing their own thing, so that's pretty important to notice as well and not try and invade people's privacy, and keep it strictly work if that's what you need. And if you need to, sort of, get to know people, then that's okay, too. Part of, I think, being able to lead well is to code switch to what people need. So, one style of management doesn't work for everybody. You have to figure out what works for each individual person and work with them in that way.
Corey: Impedance matching, almost. I periodically reference on this show a boss I once had, who, as I described him, spoke only in metaphor. Where—
Laura: Oh my.
Corey: —that's great. I don't understand what the hell you're talking about. Should I be doing more of this thing or more of that thing? “As the boulder crashes down the mountain through the stream”—it’s like, “Okay. I'm sorry, go write haiku in your own time. I'm trying to figure out exactly what needs to happen. Am I doing well? Am I about to be fired? It'd be really nice to know where I stand with you.” And I never got a clear answer.
Laura: Wow, that's rough. It's funny how those things work out. I once had a boss who had very little in the way of facial expressions. Just the way that their psychology worked; I would venture to guess that they're probably not neurotypical. And at the beginning, I found this, like, super intimidating.
And then I figured out that it didn't matter if you couldn't read his face because he would tell you exactly what he was thinking. And without emotion; it would just be all very factual. But there was no sketching around the issue, there was no trying to figure out what he meant. He would just tell you. And that was actually very relaxing once I figured that out. It's strange because if someone had said, “Would you like to work for someone like this?” I would not have said, “Yes.” But it was actually kind of refreshing.
Corey: There's something to be said about having a sense of—I know people talk about this a fair bit—of psychological safety in employment. And you need that sense of psychological safety, but you also need a sense of job safety. I know that even now, four years into running my own company with my business partner, whenever one of us sends the other a message of, “Can we talk?” And then goes quiet, it's, “Oh, am I about to be fired?” is the instinctive, immediate reaction every time. Even though neither one of us can be fired, it's still—that's a trauma that leaves scars.
Laura: It’s actually really terrifying, I think. I totally agree with you. And having had to do that, reasonably often. When you just need to ask somebody a question. And I think, I end up putting something in Slack that'll be like, “Hey, I need to ask you something quickly, but this is nothing bad. Don't worry, this is not a scary VP coming to be scary.” I find myself qualifying it like that because it's not my goal to make anybody's heart rate spike. [laugh]. They can watch a horror movie if they want that, but they probably don't. Nobody needs any extra stress right now.
Corey: Exactly.
Laura: Making sure I only inflict stress intentionally, and that's very rare.
Corey: So, one of the things that you spoke on, back when conference speaking was a thing—was a periodic focus on ‘Minimum Viable Bureaucracy,’ and as I mentioned, for someone who has a problem with authority, just the very phrase is appealing. Tell me more.
Laura: The reason I started with Minimum Viable Bureaucracy is that I, too, have a problem with authority. I have a problem with process. I'm really goal-oriented, and I don't really mind how we get there. And it's been hard for me to learn as an adult that you actually do need some process. So, figuring out what the balance is of what is the level of process that is the smallest amount required for things to run smoothly, to be predictable, without driving all the people that you work with bonkers. Because there's a lot of engineers who don't like authority, but there are people who also need process, so what's the balance? And there's a few basic principles to that.
One of them is, first of all, to push decision-making down to the edges so that people who know the most about something can make a decision about that thing. That's really important to me. A second principle is that you should always iterate on your process like you would on your code, or in your infrastructure, or in your products. You don't expect to ship v1 and walk away and never improve it; you don't expect to ship with version 1 of your config for your infrastructure and walk away and never improve it, but we tend to get stuck on process. If you are doing something and it seems terrible, then stop and don't do it; do something else instead. I think the ability to change something on a day-to-day basis, to iterate quickly, essentially, to continuously deploy the way that you work is super important to happiness.
Corey: There's also the risk aversion approach where whenever something breaks into a particular way, “Ah. We're going to add a process in to make sure that never happens that way again.” And keep iterating forward, and eventually, you're at a point where there's six thousand things that need to be verified at every point and it becomes unwieldy. It becomes almost ossified and mired in that process.
Laura: Yeah, that's absolutely true. And knee-jerk process is the worst. I think we actually have a really good team here that does incident management, and part of their job is to figure out, well, what parts of this actually require a change? Other changes we could make, other changes we shouldn't make, and to be really strategic about that. And it's super exciting to work with them on that.
Having said that, I think you can target things that need process. And the way I always say this is you should make the boring things boring, and that covers everything from the promotions process: everybody should know how it works. “How do I get promoted?” Everyone knows. It's straightforward.
It's the same every time. Maybe there's some iteration, but in general, it's well understood. It's true for deploying code as well. It should be boring, it shouldn't be exciting. I want the exciting things to be exciting, like the new thing that we're shipping, or the new tech that we're working on, or hiring somebody awesome. And I want the things that should be the same every time to be almost invisible.
Corey: You're the engineering leader I'm not, so you will almost certainly have a way better take on his than I will, but it feels that some organizations approach process and procedure from a position of if we just put enough process around this, we can finally not have the creative expensive types do these jobs, and instead just wind up turning it down to someone who follows a script and that's it. Is that the actual intention? Is that how it just manifests, or am I completely missing something fundamental?
Laura: The thing you've described is not valueless; there's one thing I think it's important to note, so I'll come back to that in a second. But in a place where you don't have any process—everything is terribly artisanal and bespoke and so on, you end up in a situation where you can only have really, really senior people working there, people who have the experience, and judgment, and know how to improvise. And what you have done then is you've made it impossible to have junior engineers, or interns, or people who do process. So, you can go too far the other way. And, to me, it's super important to have up-and-coming people, like people who are relatively new to the industry because they bring new ideas and also, who's going to run it when we all retire?
It's super important to have junior people coming up. And if you have, sort of, absolutely no standard ways of doing anything, it's going to be really, really hard for them to be successful. So that's actually perhaps a really strange reason for having processes, but that's one of the reasons. It's certainly not unreasonable to say, “Let's have the expensive people do the hard things.” Like that's certainly one way of thinking about it. But it's also so that not everybody has to be stressed out by not knowing how things work all the time.
Corey: And that's very fair. I want to be very clear: here at The Duckbill Group, we fixed the horrifying AWS bill, and originally everything was bespoke because it was just me as an independent consultant. And, yeah, I can keep it all in my head. Why not? As we started hiring people, we built out processes and procedures around how these engagements go, how the analysis looks, but it's still nowhere near the point where someone who's not conversant with the relevant technologies, and the relevant terms of art, and the relevant financial strategic requirements of businesses would be able to perform effectively in that.
So it's the, let's get some standards around here and let's make sure that we're not missing things, but it's also never been aimed at driving down what it takes in order to deliver an engagement successfully to a point where we can start having fundamentally unqualified people in those roles.
Laura: Or people that you're trying to train. You want to be able to have an apprentice, or a Padawan learner, or whatever you want to call it.
Corey: Oh, absolutely. Every person we've hired into this role has gone through a training, and onboarding, and upskill approach because no one else does this quite this way. But there's foundational prerequisite knowledge that works. I mean, it’s—the idea of what it takes to operate in large-scale environments is a key example here. You can't teach that without giving someone a high-scale environment to work within. And it turns out, we don't have a lot of those right now.
Laura: Yeah, that's really true. There are some things you can try and there are some things you can't try, and some things are harder to learn than others, I think, too. And not just scale. Because scale, as long as you can get somewhere that has it, you'll pick it up. You'll have to.
There are some things that are much, much harder to learn. And one of those is, I think, really good troubleshooting. A great way to learn that is by shadowing someone working or working with someone who's really good at it, as long as they can talk through what they're doing. Someone who's really good at troubleshooting and can communicate. Another one is, sort of, responding to ops situations.
And I think—what I mean there is incidents, and outages, and firefighting. That's actually kind of an interesting thing. I remember I once toured a fire station in Atlanta. It was actually a heavy rescue station. If you don’t know the difference between firefighting and heavy rescue, firefighting is putting out fires, heavy rescue is running into burning buildings to pull people out.
And so I had asked this fire chief, “What makes someone really good at heavy rescue?” And he said, “Someone who thinks the day we get to run into a burning building is the best day of their life.” And some people in ops are like that. Some people love it. Like, it’s—
Corey: Oh, the adrenaline hit of the firefighting? Oh, yeah. I'm right there with you.
Laura: Yeah, I love that stuff, and some people find that incredibly stressful. That's one of the things I'm not even sure that you can learn it, by the way. I mean, I like to think you can learn anything if you set out to, but it might not be good for you to do it [laugh] honestly. If you find it incredibly stressful, maybe you should take something more—a little further from the fire. Dispatch or something, or R&D. But, you know, some things are hard to learn. And that's one of them.
Corey: This episode is sponsored by ExtraHop. ExtraHop provides threat detection and response for the Enterprise (not the starship). On-prem security doesn’t translate well to cloud or multi-cloud environments, and that’s not even counting IoT. ExtraHop automatically discovers everything inside the perimeter, including your cloud workloads and IoT devices, detects these threats up to 35 percent faster, and helps you act immediately. Ask for a free trial of detection and response for AWS today at extrahop.com/trial.
Corey: What do you think is changing? Is it even still possible to have that shadow approach in a virtualized environment like we're all working within now? I mean, having someone shadow in the olden days was a tried and true method of getting someone up to speed on various engagements. It's changing now. And I don't want to ever be the kind of company that can't manage to hire junior people: “Oh, everyone here must be super senior.” Great, then where does the next generation come from if that's your approach?
Laura: Right, exactly. I think it is possible. One of the things that makes this interesting is—you know, I have an 11-year-old daughter, and she's doing remote schooling right now. And it is interesting to me how much better at learning without being in person, people who have grown up doing that than people who have grown up doing it in person. One of our friends make jokes about, “Haha, digital natives.”
And I'm like, “Yeah, that's it.” So, I see this kid sitting on a call explaining to an adult, walking them through how to set up that Discord server. Which I think is awesome, by the way. I'm super happy with that. But they don't see anything weird about it.
So, I think that is a thing that, the longer you spend doing it, the easier it becomes. It is hard to visualize how that works if you've done it in person your whole life, but the longer you spend doing it, the longer you start figuring out the tricks. To me, it is harder because you can't necessarily tell when somebody is struggling. So, there has to be a certain level of trust, and building that is probably the hardest thing. Comes back to trust, again.
Corey: Every company claims that they've nailed this. “Our employees love us. We have high trust among our staff.” And it holds still and it makes sense right up until the point where you talk to their staff directly.
Laura: Yeah. I think that's really true. It's really a hard change for people who haven't done it before and who wouldn't choose it. There's a lot of people who have had it thrust upon them this year.
Corey: So, a recurring theme of this show has been where does the next generation come from? And it's pretty clear that Fastly has done a phenomenal job of finding and recruiting extremely capable senior talent. But what are you doing to bring up the next generation? Because it's clear you don't just hire senior people, which is good. I'm just curious how you wind up developing those folks into the same level of amazing that some of your senior folks are.
Laura: So, we actually have some programs here I'm really excited about. To begin with, we hire people from all different backgrounds: we hire them from boot camps, we have people that are self-taught, we have people who have PhDs in computer science. But internally, we have a couple different programs that I am excited about. One is that we have a system where folks who work in our customer support teams can actually do a rotation through engineering where they can work in engineering one or two days a week for a few months. And if they like it, then they can transfer over and work full time in engineering as a junior engineer.
And that's been a really successful program, we've had a number of graduates from that, and some of the most awesome people on our teams. I’m really excited about some of those folks. The second thing, which is a new thing that we've just started trying, we have a kind of a weird team here called resilience engineering. That's something that I started that I'm super, super happy with. I’ll talk about that briefly, and then I'll talk about the apprenticeship program.
So, as you pointed out, we have a lot of senior folks. One of the things that's often the case with senior folks when you hire them because you—you know, they work on a standard, or they're famous for a thing, is that they tend to be specialists. People who know the most in the world about some technology X, whether it's some kind of network thing, or TLS, or whatever it is. And we didn't have a lot of folks looking at the whole system end to end: what are the weakest parts of the system? How can we make it better?
What can we do to make it more resilient? So, we started this team called resilience engineering. And it’s an interesting team. It has a bunch of really senior folks on it; some of them are pretty well known. But one of the things we thought about was that we weren't really training any new systems engineers that way.
So we've actually just rotated in our first junior engineer—well, they're not junior. They're an engineer, engineer—and I think this will be a good pathway for them to work their way up to being a principal engineer by looking at every system at Fastly, and understanding how things work, and understanding that the complexities that you get from having complex systems with huge scale. They don't necessarily behave in predictable, deterministic ways, it has emergent behavior. And there's that old story about the engineer who—knowing where to tap: this is all about knowing where to tap. So I'm pretty excited about that program.
Yeah, we're trying new things. We do not currently have an internship program, but that's something that we would like to do in the next couple of years. They’re all approaches we're getting to get junior people in.
Corey: Internships are always hard because, on the one hand, it apparently has to be about education. Two, if you're bringing interns in and not paying them, don't do that.
Laura: Oh no. Don’t do that.
Corey: That’s garbage. [laugh].
Laura: Don’t do that.
Corey: Yeah.
Laura: Yeah. Don’t.
Corey: Oh, that's not for you, that’s for a couple of companies who are feeling shame when they hear this, and they should because that's monstrous. But a lot of companies view intern programs as a backdoor recruiting funnel—which is fine, thrilled to do it. Until I watched them try to talk people into dropping out of school their final year and come into work full time instead, which feels a little weird. I've got to admit.
Laura: Yeah. No, absolutely not. Try really hard not to do that. So, we talked about starting one this year, but we didn't because of COVID. I was pretty involved at the internship program at Mozilla, so let me talk about that and some of the things we did there because I'm pretty proud of the work that we did there.
Corey: Wonderful.
Laura: So, obviously we had college interns, and we made some changes to that program because for a while we had gone after, sort of, Stanford people and MIT people, and you know exactly what I'm talking about, right? And the thing that we noticed was that there's a certain lack of diversity in folks like that, which is probably completely unsurprising to you. So, we started looking for interns from a different set of colleges and universities, and that was incredibly helpful. So, we did some deliberate recruiting to historically black colleges and universities, to colleges that had lots of professors that were interesting open-source and things like that. And those things were actually just a way to get some different types of folks.
We did another thing, too, which was a non-traditional internship program, and this was for people from any background. One of the people we had that came into it was a chef, previously. And the criteria for application were separate for each particular internship. So, for some of it might be: to apply for this internship, write us a piece of documentation; or, to apply for this internship, write a test for this. So, it was sort of a very open-source approach to things.
And some of the people we got through that program were incredibly brilliant folks from really unusual backgrounds. There was one particular person that I worked with who is one of the smartest engineers I've ever worked with. Generally, you hire somebody and you are constantly blown away by their insights and how hard they work.
Corey: I'm very fortunate to be able to say, “Yes.”
Laura: Yes, exactly. And this person had no background in computer anything. They had a background in mechanical engineering, believe it or not. They were also from a non-traditional background; I think the first person from their entire family to go to college. And they had actually been building museum exhibits for science museums, which is a really, really cool thing to do, by the way, but obviously nothing to do with programming.
The problem with those kinds of jobs is a lot of them are seasonal or contract. And they were looking around saying, “Well, I’d really like something a little bit longer term, and it seems like jobs in tech seem to have those criteria, so I will go and apply for this internship and along the way, hopefully, I will learn how to code,” and ended up being an incredibly brilliant engineer who built some really amazing things. So, yeah, I'm really open to bringing people in. My goal in working with people on the internet is for everyone to have the opportunity to build things they’re passionate about. And not everybody starts from the same point of opportunity, so finding different paths for people to get in is really important to me. I have kind of a non-traditional path at some point in my career, so I’m very empathetic to that.
Corey: I think that most people who have thrived in their career can look back at various points in their career and specific people that they reported to and say, “That person had a profound impact on my career.” Ideally, they'll even say that in a positive way, rather than negative, but I guess my goal has always tried to be one of those people that they say that about, someday. And it's easier said than done because the payoff is far in the future, you'll never know if you succeed, in some cases ever, and in other cases, not for another 15 years. But it's a good aspirational way to aim for, at least in my fumbling attempts at management. What tips would you have for folks who are aspiring to either become managers at all, or—in other words—become better managers than the ones that they had inflicted upon them?
Laura: So, I think the hardest part of any kind of leadership is managing yourself. Before you attempt to manage other people, you have to try and get a handle on yourself. And by that I mean let's imagine that you're at work, and you are really stressed. If I am an individual engineer, let's say I'm really stressed about something going on in my personal life: maybe one of my relatives has COVID, and I'm freaking out about that. Oh, I don't know how I'm going to manage childcare this year, there's so much going on, I can come up with a million examples.
And maybe my work suffers. And maybe my manager comes to talk to me about it. Now, if I'm a manager, or a director, or a VP, I come to work, and I have that leaking out all over the place, it is likely that that gets taken out on the people who work for me. Or at least, if they see that you are freaked out about something or if you're angry about something, people always jump to the worst conclusion. It’s the thing you mentioned earlier, “Can we have a quick chat about something?”
It's that if I see that my manager is super, super grumpy about something, and maybe writing a grumpy email, or the tone is not there, it has a profound negative trickle-down effect. It's not to say that you can't be human and can't have feelings, but you have to have an emotionally mature way of dealing with things. And that's really hard, by the way. Like that's non-trivial, obviously, but I encourage people, if you want to be a good manager, make sure that you have somebody to talk to about it. And that can be your boss; it can be trusted peers outside work; maybe you maybe hang out a social Slack. In a normal year, maybe you would go to a conference and have a cup of coffee with somebody. But I think it's really important to have ways to let off steam that do not involve your employees because it's just not fair to them.
Corey: Yeah. One of the hardest lessons for me to learn was that you can never complain to your directs, or in some cases, your peers. And that becomes a very difficult thing, especially when you're working in companies that aren't aligned in all the ways you wish they were, where there are things you aren't allowed to tell your staff, there are things that you think that your staff is right on, but you're not allowed to communicate in various directions because politics always strike you down. It was never a game I was particularly good at, and my approach was ultimately not to play, which. Really it's not winning; that's abdicating. What's the old line about office politics, where, you're not opting out, you're forfeiting?
Laura: Oh, ow. Ow. That's really painful, but yes, you're right. I think the second piece of advice is related to this, which is that I think you should try to be as transparent with people as you can. And when you can't be, it's okay to say, like, “I can't talk to you about this,” but being transparent by admitting that you can't talk about something.
Like, “There's something going on here; I can't talk about it now.” That's one set of things. And the other thing is to be really direct, if you can. I tell folks that I work with that my goal for communication is to be kind, direct, and prompt. So, be direct, say a thing that needs to be said, even if it's not really a positive thing, but be kind about it; there's no need to be a jerk about it, especially if it's negative feedback.
And to be prompt. So, if there's something that I need to tell you, like, “You just did a great job with this podcast, Corey,” I should tell you today. On the other hand, if it was, like, “Corey, you were a complete jerk. Why did you speak to me like that?” I should tell you today. I shouldn't wait. Three months later, until you've done the thing 20 times.
Corey: Oh, my God. The annual review or whatnot. It’s, “Well, eight months ago, you said something dumb, and we're going to ding you for it.” It’s, “What? What is this?”
Laura: Yeah, exactly. And that's all about managing your own psychology, too because it's really easy for people to want to avoid conflict. But learning to have constructive conflict is an incredibly important skill here, too.
Corey: Oh, it absolutely is. Thank you so much for taking the time to speak with me today about a wide variety of different topics all tied back to engineering leadership in some form or another. If people want to learn more about what you're up to, where can they find you?
Laura: Probably the best way to find me is on Twitter, which sounds terrible, doesn't it? But blogs, and websites, and things all have fallen by the wayside because I have had less and less time to think about anything longer than 140 characters. So, you can find me on Twitter @lxt.
Corey: And I presume you're hiring as well?
Laura: So we've had a very busy year, and as a result, we're hiring a lot of folks. So, please reach out if you're interested in working here.
Corey: Excellent. And we'll of course put links to that in the [show notes 00:35:40]. Thank you so much for taking the time to speak with me today. I appreciate it.
Laura: Of course. Anytime. It was lovely talking to you, as well.
Corey: Laura Thompson, VP of platform engineering at Fastly. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with a comment saying why a CDN isn't necessary and even if it were, you could build your own in the course of a weekend.
Announcer: This has been this week’s episode of Screaming in the Cloud. You can also find more Corey at screaminginthecloud.com, or wherever fine snark is sold.
This has been a HumblePod production. Stay humble.
Announcer: Hello, and welcome to Screaming in the Cloud with your host, Cloud Economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of Cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.
Corey: This episode is sponsored in part byLaunchDarkly. Take a look at what it takes to get your code into production. I’m going to just guess that it’s awful because it’s always awful. No one loves their deployment process. What if launching new features didn’t require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren’t what you expect? LaunchDarkly does exactly this. To learn more, visitlaunchdarkly.com and tell them Corey sent you, and watch for the wince.
Corey: If your mean time to WTF for a security alert is more than a minute, it's time to look at Lacework. Lacework will help you get your security act together for everything from compliance service configurations to container app relationships, all without the need for PhDs in AWS to write the rules. If you're building a secure business on AWS with compliance requirements, you don't really have time to choose between antivirus or firewall companies to help you secure your stack. That's why Lacework is built from the ground up for the Cloud: low effort, high visibility and detection. To learn more, visit lacework.com.
Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by Laura Thompson, VP of platform engineering at Fastly. Laura, thank you for joining me.
Laura: Thank you for having me on.
Corey: So, Fastly is generally considered to be one of the definitive names in the CDN world. Is that an accurate description of what Fastly is, or is it perceived internally as, “Well, we are a CDN, but there's also much more to it than that.” I always want to make sure that my understanding of the company isn't based upon things that are no longer completely true.
Laura: So, we like to describe ourselves as an edge cloud platform because we've gone beyond the CDN. We are now shipping these products like Compute@Edge—which is essentially serverless only really, really fast because it runs at the edge—and a suite of security products. So it's more than just a CDN. And I think we're becoming more and more general as cloud platforms go.
Corey: Credit where due, I had a customer a while back who was using Fastly as a CDN and had been using a bunch of these edge compute things, and they said, “Oh, yeah, we can't move that off on to a competitor at this point because of all that logic that's there.” And I said, “Oh, so it's a locked-in story?” And they looked at me as if I were simple and said, “No. Because it's awesome, and nothing else is quite like it.” “Well, yeah, I guess there is a certain lock-in story around building something awesome that other people haven’t.”
Laura: That's actually really interesting. We've been talking about—this the product folks that work because the perception that I have is that people choose a particular cloud provider—whether it's general compute, or whether it's CDN, or whatever it is, edge cloud stuff—for the unique features. For the most part, people aren’t really interested in the lowest common denominator stuff, unless they have a very straightforward use case, and people have less and less straightforward use cases over time. So, they want to use the unique features; they want to know what's special, what can I only do here, and that's the basis of their purchasing decision a lot of the time.
Corey: Absolutely. Any sort of cloud comparison analysis between which vendor we're going to pick that breaks down to, “Well, what size of instance is going to cost how much?” And then trying to equate that, like, that is so far from the relevant part of the story when you're doing vendor selection in a modern era.
Laura: Right, exactly. It's interesting for me because I've been on the other side of the desk a lot. My last job at Mozilla, I was partially responsible for making CDN buying choices, so I have pretty good insight into what goes into that. It's interesting to think about.
Corey: It really is. In fact, to that end, you were—in fact, you are currently a member of the board of trustees of the Internet Society. What is the Internet Society, first off?
Laura: So, the Internet Society is about making sure that people have access, equality of opportunity, and that standards are kept open. So, the Internet Society essentially funds the IETF, which is responsible for most of the standards that run the internet. And that is sort of the primary mission.
Corey: I feel like on some level, the default response to that is, “Wait a second, the internet has standards? Since when?”
Laura: [laugh]. Right. Well, it does. I mean, TLS and HTTP would be the two things that people would think of, I would expect. And if those were not standardized, we wouldn't be here having this conversation.
Corey: Exactly. For those who didn't grow up in the ’80s ’90s, et cetera, watching these things evolve, the things that we take for granted today, like different networks can talk to one another all comes down to interoperability around shared standards. But I remember the dark ages, where if you were on CompuServe, you couldn't talk to the people who were on AOL directly. And over time, those walls started breaking down, and a lot of work of the standards bodies like the Internet Society and the IETF are the reason why. It's one of those boring governance things that happens underneath the hood so no one has to think about it. But the reason no one has to think about it is because people like you are doing the hard work of making it that way.
Laura: That’s right. And standards is a thing I've been passionate about for a long time. It is one of the reasons that I was at Mozilla for so long. It's one of the reasons I'm at Fastly, which has, I think, a similar role on the other side. You know, if Mozilla is thinking about how do we make clients use open standards, Fastly is thinking about how do we keep the internet open, and standard, and useful, and safe for everybody?
And [ISoc 00:06:00] and IETF are obviously, in the position of figuring out how things talk to each other. And it seems like maybe this is a solved problem, except it's not. One of the things that's happened on the internet—which as you can see over the last few years is that we're actually getting less and less open. So, we have more walled gardens, we have the internet being dominated by a few really large vendors, and they have a lot of power and standards. If you have the vast majority of the market share in a particular vertical, then you get to dictate what the standards are, and you can do that in a fairly anti-competitive way. I'm deliberately not naming any names here.
Corey: Oh, absolutely. I wouldn't expect you to, but I sure can. There's a reason that I have a podcast here where I control the RSS feed that I can point wherever I need it to live; there's a reason I have an email newsletter; there's a reason I don't have a Facebook page, to be very direct, where it's not about de-platforming in the censorship sense, but more along the lines of if I have built an audience on a platform, and then that platform decides that its business strategy is going to shift—something like Medium, for example—then suddenly, I am beholden to the whims of that provider unless I want to start over from scratch again. That's why it's always been so important for me to build my audience on something that was much more agnostic, not because I'm posting garbage that should be taken off the internet—well, most weeks—but rather because I don't want other people making my business decisions for me.
Laura: Yeah, so my interest in the internet is not about de-platforming, it's about platforming. It's making sure that people have access and the ability to build awesome things as much as they can.
Corey: Absolutely. But you're also a VP of engineering, which I feel like if I had asked you a year and a half ago, “What does it mean to be a VP of engineering?” You would have given me an answer. And now I feel like if I asked that same question in these uncertain times, or these unprecedented times, depending upon framing, you might have a different answer from that. Is that accurate?
Laura: Yeah, I think it's definitely been—I’m really tired of the word unprecedented. It's been a year. [laugh]. It's been a heck of a year.
Corey: It really ha—it feels like it's been longer than that.
Laura: Yeah. So, I started at Fastly on the ninth of March, 2020, which was a week after Fastly closed the offices for COVID. So, I have never set foot in an office, other than in the interview process. And I haven't met almost any of my coworkers. It's super strange, and it's really hard to build those trust relationships without having met anybody.
So that's kind of the first thing, and obviously, you would have a different experience if you'd been there and seeing the change, but this is all I have known in this role. So it's actually pretty interesting.
Corey: I know that there's a lot of zeitgeist awareness around what it's like to be an employee in a pandemic now and being full remote and the rest, but something that doesn't get discussed very much is, how does leadership change when suddenly you aren't able to gather people in a room and have a conversation and hash something out when everyone's remote?
Laura: Right. So, the funny thing is, that part hasn't changed very much. And I will say—for me, I've been a remote employee for about 15 years. And this is not what remote is normally like. I think it's part of the, sort of, important takeaway for people who might be doing it for the first time.
There's a big difference between choosing it and having it thrust upon you, for one thing. When I give people advice on how to work remotely successfully, it's always about, have good boundaries, have a dedicated space, and make sure that you start work at a particular time, finish work at a particular time so that you can walk away and de-stress. And because a lot of people are kind of trapped in the houses, they can’t have as good boundaries. They might have a one-bedroom apartment in San Francisco, and it's all in one room, and their spouse is there trying to work as well, and maybe they have a dog and a baby. It is not a typical remote working experience.
Similarly, it's not the typical remote leadership experience because remote is great. It also is important to meet with people once in a while, and not for meetings—not all sitting around a whiteboard talking through a bunch of dot points--but for the part that is really hard to do remote, which is the relationship-building. To me, you are trying to get to know people so that when things go wrong, they say, “Oh, I know Laura. I know what she's like. I trust her to do this right.”
And when I’ve never met you, it’s a little harder for people to do that; everybody has a little work a little harder. And when I say a little harder that's in the face of all of the cognitive load that we're already under because it's been, as you say, a heck of a year.
Corey: Right. And there are a lot of negative examples about how all this stuff looks terrible when people get it wrong. For example, “Oh, you're not spending an hour commuting every day; now you can spend that time working.” Or, “We have a policy of turning your webcam on, which is a fancy way of saying I'm inviting myself into your home so I can critique it. Excuse me, it's not acceptable that your infant is crying.” At some point, you hear these stories, and it's the biggest gap we have is the ability to strangle people over the internet now because that's horrifying.
Laura: I know. We have done a lot, I think, to make that comfortable for people. I think part of it is, Fastly has been at least sort of, half remote for a long time. [unintelligible 00:10:50], like remote-first companies are good to work for I will say, in general. But making sure that people know that it's okay if you have to have your camera off, or if you have to have a baby on your lap.
In fact, the thing I found is there is nothing quite like a meeting that has a cat, or a baby, or a puppy. And it lightens the mood, it helps people talk to each other. So we're all surrounded by our emotional service babies, and cats, and dogs. It's quite funny, really. I'm going to tell a story, which is—people talk about CEOs, and we have a CEO named Joshua Bixby, who has a really lovely human being. And when the pandemic started, he started a weekly meeting where he would read picture books to people's kids.
Corey: That's an amazing idea and in fact, I’m debating stealing it and claiming I came up with it myself, except that we're doing this on a recording, and suddenly my team will know when this comes out.
Laura: Yeah. I just thought it was an incredibly nice thing to do. And making sure that people knew that we're all in it with our families, we're all stuck at home with kids and whatever, and let's try and make the best of it and get to know each other a little bit better.
Corey: On the one hand, I absolutely agree with the sentiment and the place it comes from. On the other, I've got to say I have an anti-authority streak in a big way. My single biggest stumbling block, along with my personality, back when I was an employee. I mean, my last job was at a regulated finance company. You can imagine how well that worked out for me.
But authority is a problem for me, so whenever I hear, “Oh, go ahead and bring your family into this social gathering we're doing,” my immediate knee-jerk response is, “Is this required?” And that's not helpful as far as the sentiment goes, but it was there. That was my initial flare-up reaction. And I'm always hyper-aware of—now that I managed before myself of being sure to never present as anything other than if you want to.
Laura: Yes. Yes, it's very much been that way here. I too have an anti-authority streak, which is a funny thing to say when you end up in leadership. But that's why, by the way. And I think sort of compulsive joining-ness annoys a lot of people. I'm not one of those people that it annoys. I like hanging out with other people. I'm really extroverted, which may surprise you in an engineer and someone who chooses to work remote, but I like people, right?
I like hanging out with people; happy to hang out with everybody, but I know that not everybody wants to and that is 100 percent okay and you have to be so clear that that's okay. Everybody is kind of walking their own road through this thing. And for some people it’s… [sigh] some people are actually pretty happy being on their own, doing their own thing, so that's pretty important to notice as well and not try and invade people's privacy, and keep it strictly work if that's what you need. And if you need to, sort of, get to know people, then that's okay, too. Part of, I think, being able to lead well is to code switch to what people need. So, one style of management doesn't work for everybody. You have to figure out what works for each individual person and work with them in that way.
Corey: Impedance matching, almost. I periodically reference on this show a boss I once had, who, as I described him, spoke only in metaphor. Where—
Laura: Oh my.
Corey: —that's great. I don't understand what the hell you're talking about. Should I be doing more of this thing or more of that thing? “As the boulder crashes down the mountain through the stream”—it’s like, “Okay. I'm sorry, go write haiku in your own time. I'm trying to figure out exactly what needs to happen. Am I doing well? Am I about to be fired? It'd be really nice to know where I stand with you.” And I never got a clear answer.
Laura: Wow, that's rough. It's funny how those things work out. I once had a boss who had very little in the way of facial expressions. Just the way that their psychology worked; I would venture to guess that they're probably not neurotypical. And at the beginning, I found this, like, super intimidating.
And then I figured out that it didn't matter if you couldn't read his face because he would tell you exactly what he was thinking. And without emotion; it would just be all very factual. But there was no sketching around the issue, there was no trying to figure out what he meant. He would just tell you. And that was actually very relaxing once I figured that out. It's strange because if someone had said, “Would you like to work for someone like this?” I would not have said, “Yes.” But it was actually kind of refreshing.
Corey: There's something to be said about having a sense of—I know people talk about this a fair bit—of psychological safety in employment. And you need that sense of psychological safety, but you also need a sense of job safety. I know that even now, four years into running my own company with my business partner, whenever one of us sends the other a message of, “Can we talk?” And then goes quiet, it's, “Oh, am I about to be fired?” is the instinctive, immediate reaction every time. Even though neither one of us can be fired, it's still—that's a trauma that leaves scars.
Laura: It’s actually really terrifying, I think. I totally agree with you. And having had to do that, reasonably often. When you just need to ask somebody a question. And I think, I end up putting something in Slack that'll be like, “Hey, I need to ask you something quickly, but this is nothing bad. Don't worry, this is not a scary VP coming to be scary.” I find myself qualifying it like that because it's not my goal to make anybody's heart rate spike. [laugh]. They can watch a horror movie if they want that, but they probably don't. Nobody needs any extra stress right now.
Corey: Exactly.
Laura: Making sure I only inflict stress intentionally, and that's very rare.
Corey: So, one of the things that you spoke on, back when conference speaking was a thing—was a periodic focus on ‘Minimum Viable Bureaucracy,’ and as I mentioned, for someone who has a problem with authority, just the very phrase is appealing. Tell me more.
Laura: The reason I started with Minimum Viable Bureaucracy is that I, too, have a problem with authority. I have a problem with process. I'm really goal-oriented, and I don't really mind how we get there. And it's been hard for me to learn as an adult that you actually do need some process. So, figuring out what the balance is of what is the level of process that is the smallest amount required for things to run smoothly, to be predictable, without driving all the people that you work with bonkers. Because there's a lot of engineers who don't like authority, but there are people who also need process, so what's the balance? And there's a few basic principles to that.
One of them is, first of all, to push decision-making down to the edges so that people who know the most about something can make a decision about that thing. That's really important to me. A second principle is that you should always iterate on your process like you would on your code, or in your infrastructure, or in your products. You don't expect to ship v1 and walk away and never improve it; you don't expect to ship with version 1 of your config for your infrastructure and walk away and never improve it, but we tend to get stuck on process. If you are doing something and it seems terrible, then stop and don't do it; do something else instead. I think the ability to change something on a day-to-day basis, to iterate quickly, essentially, to continuously deploy the way that you work is super important to happiness.
Corey: There's also the risk aversion approach where whenever something breaks into a particular way, “Ah. We're going to add a process in to make sure that never happens that way again.” And keep iterating forward, and eventually, you're at a point where there's six thousand things that need to be verified at every point and it becomes unwieldy. It becomes almost ossified and mired in that process.
Laura: Yeah, that's absolutely true. And knee-jerk process is the worst. I think we actually have a really good team here that does incident management, and part of their job is to figure out, well, what parts of this actually require a change? Other changes we could make, other changes we shouldn't make, and to be really strategic about that. And it's super exciting to work with them on that.
Having said that, I think you can target things that need process. And the way I always say this is you should make the boring things boring, and that covers everything from the promotions process: everybody should know how it works. “How do I get promoted?” Everyone knows. It's straightforward.
It's the same every time. Maybe there's some iteration, but in general, it's well understood. It's true for deploying code as well. It should be boring, it shouldn't be exciting. I want the exciting things to be exciting, like the new thing that we're shipping, or the new tech that we're working on, or hiring somebody awesome. And I want the things that should be the same every time to be almost invisible.
Corey: You're the engineering leader I'm not, so you will almost certainly have a way better take on his than I will, but it feels that some organizations approach process and procedure from a position of if we just put enough process around this, we can finally not have the creative expensive types do these jobs, and instead just wind up turning it down to someone who follows a script and that's it. Is that the actual intention? Is that how it just manifests, or am I completely missing something fundamental?
Laura: The thing you've described is not valueless; there's one thing I think it's important to note, so I'll come back to that in a second. But in a place where you don't have any process—everything is terribly artisanal and bespoke and so on, you end up in a situation where you can only have really, really senior people working there, people who have the experience, and judgment, and know how to improvise. And what you have done then is you've made it impossible to have junior engineers, or interns, or people who do process. So, you can go too far the other way. And, to me, it's super important to have up-and-coming people, like people who are relatively new to the industry because they bring new ideas and also, who's going to run it when we all retire?
It's super important to have junior people coming up. And if you have, sort of, absolutely no standard ways of doing anything, it's going to be really, really hard for them to be successful. So that's actually perhaps a really strange reason for having processes, but that's one of the reasons. It's certainly not unreasonable to say, “Let's have the expensive people do the hard things.” Like that's certainly one way of thinking about it. But it's also so that not everybody has to be stressed out by not knowing how things work all the time.
Corey: And that's very fair. I want to be very clear: here at The Duckbill Group, we fixed the horrifying AWS bill, and originally everything was bespoke because it was just me as an independent consultant. And, yeah, I can keep it all in my head. Why not? As we started hiring people, we built out processes and procedures around how these engagements go, how the analysis looks, but it's still nowhere near the point where someone who's not conversant with the relevant technologies, and the relevant terms of art, and the relevant financial strategic requirements of businesses would be able to perform effectively in that.
So it's the, let's get some standards around here and let's make sure that we're not missing things, but it's also never been aimed at driving down what it takes in order to deliver an engagement successfully to a point where we can start having fundamentally unqualified people in those roles.
Laura: Or people that you're trying to train. You want to be able to have an apprentice, or a Padawan learner, or whatever you want to call it.
Corey: Oh, absolutely. Every person we've hired into this role has gone through a training, and onboarding, and upskill approach because no one else does this quite this way. But there's foundational prerequisite knowledge that works. I mean, it’s—the idea of what it takes to operate in large-scale environments is a key example here. You can't teach that without giving someone a high-scale environment to work within. And it turns out, we don't have a lot of those right now.
Laura: Yeah, that's really true. There are some things you can try and there are some things you can't try, and some things are harder to learn than others, I think, too. And not just scale. Because scale, as long as you can get somewhere that has it, you'll pick it up. You'll have to.
There are some things that are much, much harder to learn. And one of those is, I think, really good troubleshooting. A great way to learn that is by shadowing someone working or working with someone who's really good at it, as long as they can talk through what they're doing. Someone who's really good at troubleshooting and can communicate. Another one is, sort of, responding to ops situations.
And I think—what I mean there is incidents, and outages, and firefighting. That's actually kind of an interesting thing. I remember I once toured a fire station in Atlanta. It was actually a heavy rescue station. If you don’t know the difference between firefighting and heavy rescue, firefighting is putting out fires, heavy rescue is running into burning buildings to pull people out.
And so I had asked this fire chief, “What makes someone really good at heavy rescue?” And he said, “Someone who thinks the day we get to run into a burning building is the best day of their life.” And some people in ops are like that. Some people love it. Like, it’s—
Corey: Oh, the adrenaline hit of the firefighting? Oh, yeah. I'm right there with you.
Laura: Yeah, I love that stuff, and some people find that incredibly stressful. That's one of the things I'm not even sure that you can learn it, by the way. I mean, I like to think you can learn anything if you set out to, but it might not be good for you to do it [laugh] honestly. If you find it incredibly stressful, maybe you should take something more—a little further from the fire. Dispatch or something, or R&D. But, you know, some things are hard to learn. And that's one of them.
Corey: This episode is sponsored by ExtraHop. ExtraHop provides threat detection and response for the Enterprise (not the starship). On-prem security doesn’t translate well to cloud or multi-cloud environments, and that’s not even counting IoT. ExtraHop automatically discovers everything inside the perimeter, including your cloud workloads and IoT devices, detects these threats up to 35 percent faster, and helps you act immediately. Ask for a free trial of detection and response for AWS today at extrahop.com/trial.
Corey: What do you think is changing? Is it even still possible to have that shadow approach in a virtualized environment like we're all working within now? I mean, having someone shadow in the olden days was a tried and true method of getting someone up to speed on various engagements. It's changing now. And I don't want to ever be the kind of company that can't manage to hire junior people: “Oh, everyone here must be super senior.” Great, then where does the next generation come from if that's your approach?
Laura: Right, exactly. I think it is possible. One of the things that makes this interesting is—you know, I have an 11-year-old daughter, and she's doing remote schooling right now. And it is interesting to me how much better at learning without being in person, people who have grown up doing that than people who have grown up doing it in person. One of our friends make jokes about, “Haha, digital natives.”
And I'm like, “Yeah, that's it.” So, I see this kid sitting on a call explaining to an adult, walking them through how to set up that Discord server. Which I think is awesome, by the way. I'm super happy with that. But they don't see anything weird about it.
So, I think that is a thing that, the longer you spend doing it, the easier it becomes. It is hard to visualize how that works if you've done it in person your whole life, but the longer you spend doing it, the longer you start figuring out the tricks. To me, it is harder because you can't necessarily tell when somebody is struggling. So, there has to be a certain level of trust, and building that is probably the hardest thing. Comes back to trust, again.
Corey: Every company claims that they've nailed this. “Our employees love us. We have high trust among our staff.” And it holds still and it makes sense right up until the point where you talk to their staff directly.
Laura: Yeah. I think that's really true. It's really a hard change for people who haven't done it before and who wouldn't choose it. There's a lot of people who have had it thrust upon them this year.
Corey: So, a recurring theme of this show has been where does the next generation come from? And it's pretty clear that Fastly has done a phenomenal job of finding and recruiting extremely capable senior talent. But what are you doing to bring up the next generation? Because it's clear you don't just hire senior people, which is good. I'm just curious how you wind up developing those folks into the same level of amazing that some of your senior folks are.
Laura: So, we actually have some programs here I'm really excited about. To begin with, we hire people from all different backgrounds: we hire them from boot camps, we have people that are self-taught, we have people who have PhDs in computer science. But internally, we have a couple different programs that I am excited about. One is that we have a system where folks who work in our customer support teams can actually do a rotation through engineering where they can work in engineering one or two days a week for a few months. And if they like it, then they can transfer over and work full time in engineering as a junior engineer.
And that's been a really successful program, we've had a number of graduates from that, and some of the most awesome people on our teams. I’m really excited about some of those folks. The second thing, which is a new thing that we've just started trying, we have a kind of a weird team here called resilience engineering. That's something that I started that I'm super, super happy with. I’ll talk about that briefly, and then I'll talk about the apprenticeship program.
So, as you pointed out, we have a lot of senior folks. One of the things that's often the case with senior folks when you hire them because you—you know, they work on a standard, or they're famous for a thing, is that they tend to be specialists. People who know the most in the world about some technology X, whether it's some kind of network thing, or TLS, or whatever it is. And we didn't have a lot of folks looking at the whole system end to end: what are the weakest parts of the system? How can we make it better?
What can we do to make it more resilient? So, we started this team called resilience engineering. And it’s an interesting team. It has a bunch of really senior folks on it; some of them are pretty well known. But one of the things we thought about was that we weren't really training any new systems engineers that way.
So we've actually just rotated in our first junior engineer—well, they're not junior. They're an engineer, engineer—and I think this will be a good pathway for them to work their way up to being a principal engineer by looking at every system at Fastly, and understanding how things work, and understanding that the complexities that you get from having complex systems with huge scale. They don't necessarily behave in predictable, deterministic ways, it has emergent behavior. And there's that old story about the engineer who—knowing where to tap: this is all about knowing where to tap. So I'm pretty excited about that program.
Yeah, we're trying new things. We do not currently have an internship program, but that's something that we would like to do in the next couple of years. They’re all approaches we're getting to get junior people in.
Corey: Internships are always hard because, on the one hand, it apparently has to be about education. Two, if you're bringing interns in and not paying them, don't do that.
Laura: Oh no. Don’t do that.
Corey: That’s garbage. [laugh].
Laura: Don’t do that.
Corey: Yeah.
Laura: Yeah. Don’t.
Corey: Oh, that's not for you, that’s for a couple of companies who are feeling shame when they hear this, and they should because that's monstrous. But a lot of companies view intern programs as a backdoor recruiting funnel—which is fine, thrilled to do it. Until I watched them try to talk people into dropping out of school their final year and come into work full time instead, which feels a little weird. I've got to admit.
Laura: Yeah. No, absolutely not. Try really hard not to do that. So, we talked about starting one this year, but we didn't because of COVID. I was pretty involved at the internship program at Mozilla, so let me talk about that and some of the things we did there because I'm pretty proud of the work that we did there.
Corey: Wonderful.
Laura: So, obviously we had college interns, and we made some changes to that program because for a while we had gone after, sort of, Stanford people and MIT people, and you know exactly what I'm talking about, right? And the thing that we noticed was that there's a certain lack of diversity in folks like that, which is probably completely unsurprising to you. So, we started looking for interns from a different set of colleges and universities, and that was incredibly helpful. So, we did some deliberate recruiting to historically black colleges and universities, to colleges that had lots of professors that were interesting open-source and things like that. And those things were actually just a way to get some different types of folks.
We did another thing, too, which was a non-traditional internship program, and this was for people from any background. One of the people we had that came into it was a chef, previously. And the criteria for application were separate for each particular internship. So, for some of it might be: to apply for this internship, write us a piece of documentation; or, to apply for this internship, write a test for this. So, it was sort of a very open-source approach to things.
And some of the people we got through that program were incredibly brilliant folks from really unusual backgrounds. There was one particular person that I worked with who is one of the smartest engineers I've ever worked with. Generally, you hire somebody and you are constantly blown away by their insights and how hard they work.
Corey: I'm very fortunate to be able to say, “Yes.”
Laura: Yes, exactly. And this person had no background in computer anything. They had a background in mechanical engineering, believe it or not. They were also from a non-traditional background; I think the first person from their entire family to go to college. And they had actually been building museum exhibits for science museums, which is a really, really cool thing to do, by the way, but obviously nothing to do with programming.
The problem with those kinds of jobs is a lot of them are seasonal or contract. And they were looking around saying, “Well, I’d really like something a little bit longer term, and it seems like jobs in tech seem to have those criteria, so I will go and apply for this internship and along the way, hopefully, I will learn how to code,” and ended up being an incredibly brilliant engineer who built some really amazing things. So, yeah, I'm really open to bringing people in. My goal in working with people on the internet is for everyone to have the opportunity to build things they’re passionate about. And not everybody starts from the same point of opportunity, so finding different paths for people to get in is really important to me. I have kind of a non-traditional path at some point in my career, so I’m very empathetic to that.
Corey: I think that most people who have thrived in their career can look back at various points in their career and specific people that they reported to and say, “That person had a profound impact on my career.” Ideally, they'll even say that in a positive way, rather than negative, but I guess my goal has always tried to be one of those people that they say that about, someday. And it's easier said than done because the payoff is far in the future, you'll never know if you succeed, in some cases ever, and in other cases, not for another 15 years. But it's a good aspirational way to aim for, at least in my fumbling attempts at management. What tips would you have for folks who are aspiring to either become managers at all, or—in other words—become better managers than the ones that they had inflicted upon them?
Laura: So, I think the hardest part of any kind of leadership is managing yourself. Before you attempt to manage other people, you have to try and get a handle on yourself. And by that I mean let's imagine that you're at work, and you are really stressed. If I am an individual engineer, let's say I'm really stressed about something going on in my personal life: maybe one of my relatives has COVID, and I'm freaking out about that. Oh, I don't know how I'm going to manage childcare this year, there's so much going on, I can come up with a million examples.
And maybe my work suffers. And maybe my manager comes to talk to me about it. Now, if I'm a manager, or a director, or a VP, I come to work, and I have that leaking out all over the place, it is likely that that gets taken out on the people who work for me. Or at least, if they see that you are freaked out about something or if you're angry about something, people always jump to the worst conclusion. It’s the thing you mentioned earlier, “Can we have a quick chat about something?”
It's that if I see that my manager is super, super grumpy about something, and maybe writing a grumpy email, or the tone is not there, it has a profound negative trickle-down effect. It's not to say that you can't be human and can't have feelings, but you have to have an emotionally mature way of dealing with things. And that's really hard, by the way. Like that's non-trivial, obviously, but I encourage people, if you want to be a good manager, make sure that you have somebody to talk to about it. And that can be your boss; it can be trusted peers outside work; maybe you maybe hang out a social Slack. In a normal year, maybe you would go to a conference and have a cup of coffee with somebody. But I think it's really important to have ways to let off steam that do not involve your employees because it's just not fair to them.
Corey: Yeah. One of the hardest lessons for me to learn was that you can never complain to your directs, or in some cases, your peers. And that becomes a very difficult thing, especially when you're working in companies that aren't aligned in all the ways you wish they were, where there are things you aren't allowed to tell your staff, there are things that you think that your staff is right on, but you're not allowed to communicate in various directions because politics always strike you down. It was never a game I was particularly good at, and my approach was ultimately not to play, which. Really it's not winning; that's abdicating. What's the old line about office politics, where, you're not opting out, you're forfeiting?
Laura: Oh, ow. Ow. That's really painful, but yes, you're right. I think the second piece of advice is related to this, which is that I think you should try to be as transparent with people as you can. And when you can't be, it's okay to say, like, “I can't talk to you about this,” but being transparent by admitting that you can't talk about something.
Like, “There's something going on here; I can't talk about it now.” That's one set of things. And the other thing is to be really direct, if you can. I tell folks that I work with that my goal for communication is to be kind, direct, and prompt. So, be direct, say a thing that needs to be said, even if it's not really a positive thing, but be kind about it; there's no need to be a jerk about it, especially if it's negative feedback.
And to be prompt. So, if there's something that I need to tell you, like, “You just did a great job with this podcast, Corey,” I should tell you today. On the other hand, if it was, like, “Corey, you were a complete jerk. Why did you speak to me like that?” I should tell you today. I shouldn't wait. Three months later, until you've done the thing 20 times.
Corey: Oh, my God. The annual review or whatnot. It’s, “Well, eight months ago, you said something dumb, and we're going to ding you for it.” It’s, “What? What is this?”
Laura: Yeah, exactly. And that's all about managing your own psychology, too because it's really easy for people to want to avoid conflict. But learning to have constructive conflict is an incredibly important skill here, too.
Corey: Oh, it absolutely is. Thank you so much for taking the time to speak with me today about a wide variety of different topics all tied back to engineering leadership in some form or another. If people want to learn more about what you're up to, where can they find you?
Laura: Probably the best way to find me is on Twitter, which sounds terrible, doesn't it? But blogs, and websites, and things all have fallen by the wayside because I have had less and less time to think about anything longer than 140 characters. So, you can find me on Twitter @lxt.
Corey: And I presume you're hiring as well?
Laura: So we've had a very busy year, and as a result, we're hiring a lot of folks. So, please reach out if you're interested in working here.
Corey: Excellent. And we'll of course put links to that in the [show notes 00:35:40]. Thank you so much for taking the time to speak with me today. I appreciate it.
Laura: Of course. Anytime. It was lovely talking to you, as well.
Corey: Laura Thompson, VP of platform engineering at Fastly. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with a comment saying why a CDN isn't necessary and even if it were, you could build your own in the course of a weekend.
Announcer: This has been this week’s episode of Screaming in the Cloud. You can also find more Corey at screaminginthecloud.com, or wherever fine snark is sold.
This has been a HumblePod production. Stay humble.
