John Hubbard 00:00
support for the blueprint podcast comes from the sans institute if you like the topics covered in this podcast and would like to learn more about blue team fundamentals such as hosting network data collection threat detection alert triage threat intelligence and more check out my new course sek 450 blue team fundamentals this course is designed to bring attendees the information that every sock analyst and blue team member needs to know to hit the ground running including 15 labs to get you hands on with tools for threat intelligence sim incident management automation and much much more this course is everything you need to launch your blue team career check out the details at sans url.com slash 450 hope to see you in class
00:40
this is the blueprint podcast bringing you the latest in cyber defense and security operations from top blue team leaders blueprint is brought to you by the sans institute hosted by sans certified instructor john houbara now here's your host john koba
John Hubbard 01:02
in this episode of the blueprint podcast we speak with tanya janka of wiac purple about app sec devops and dev sec ops we discussed what they are why you need to know about them and the consequences of ignoring these new and critical fields even if you aren't a programmer these technologies are quickly paving the way into the future of it operations and not knowing what they are means you'll quickly be left behind which of course no one wants so buckle in and let's discuss your role as a blue teamer in the new world of dev sec ops today on the blueprint podcast all right so today on the podcast we have founder and ceo of the we have purple academy accomplished application security author podcast host dev sec ops expert tanya janka tanya thank you so much for making time in what i'm sure is a very busy schedule to be on the podcast today
Tanya Janca 01:54
thanks for having me john
John Hubbard 01:57
so as i have hinted at with the intro here you have your hands in a ton of different subjects and expertise is could you give our listeners a little bit about your background and your expertise
Tanya Janca 02:09
sure i was a dev for a really long time i think around 17 years and then i met a hacker and he said you should work in security and i was like no security is lame writing software is the best i've also been a professional musician for most of my life and he was in a band too so then obviously our bands had to play together and we became friends and for a year and a half he just broke me down slowly until i agree to try to become a pen tester very opposite of most people's interaction with how to get into security and then i was briefly a pen tester before i discovered that there's this field called application security which is sort of halfway between the security team and the dev team if that makes sense and then and also that you got to be social you got to hang out with dabs you still got to smash things i think this is the best of all the worlds and so i've been doing application security for quite a few years now and just totally loving it
John Hubbard 03:14
very very cool so yeah everyone always has an interesting story on how they get an infosec i live here in the various routes that people take so absecon and devops in this kind of world that you live in is not something we've covered on this podcast before so i wanted to make sure before we kind of dive into the meat of it for those listeners who might not also be in this world and i'll start off by saying i'm not really in that world specifically so excuse if i have dumb questions app sec devops and all of that but let's start off with what is application security to you and like a day to day basis
Tanya Janca 03:49
yeah so application security the idea of it is it's all the things that you do to try to make sure that the custom software or even the software you buy is actually secure or it's add a security posture that you feel safe using it and so that can mean for me teaching a whole bunch of devs about a mistake that i keep seeing in their code it can be about adding some extra security tests to their pipeline it can mean me doing some security testing or automating a bunch of tests to go out and then like reviewing the results talking to the dev about the results and then usually quite a bit of negotiation about i know you have this deadline friday but there's one critical that really has me concerned so there's like a lot of social stuff and talking and persuasion so you might be awesome at sales if you're awesome at apps so it's kind of that job where you're the one that you know kind of make sure all the steps happen during the system development lifecycle so that your app is in good shape at the end and you sort of try to teach the devs and enable them to make secure app
John Hubbard 05:00
gotcha so i'm sure i know the answer to this but to make it explicit what are the consequences or issues that arise if you do not pay attention to this what's the common like big impacts you see for failures
Tanya Janca 05:13
so i've seen a lot i've seen some where there's going to be an acquisition and merger and then the company looks at all the apps are really old and legacy and awful and they're just totally dumpster fires of security problems and then the acquisition doesn't happen i've seen it where just so many companies are dealing with security incidents basically all the time their staff super stressed out and then that means they can't do all the projects and normal stuff they're supposed to be doing i've seen quite often we've heard of ransomware we've heard of you know malware and stuff so there are different types of attacks that happen on software than that but basically where there are attacks where people can try to steal the data out of your app or they can try to take the data and change it or they can bring your app down so i can't get back up stealing secrets basically like all the badness that could happen with a computer can definitely happen with software
John Hubbard 06:12
so i was expecting the technical like obviously security breaches john answer but you took it much further than that right like your business you know m&a is might be broken for bad apps like like that's a serious high level like full business visibility problem
Tanya Janca 06:25
yeah i consult at a place and so we would advise on acquisitions and mergers and they actually took $10 million off of the price because of how much app sec they were going to have to do they're like there's no app sec program where you work none of your staff have ever been trained all your apps are just like we ran a scanner on it and it looked like a christmas tree turning on and we're just so it has financial consequences and also consequences for real people so it can result in for instance someone attacked a water system in the united states and poisoned i think 15,000 people recently and that could have resulted in death it's absolutely terrifying until all of this is software
John Hubbard 07:13
yeah yeah absolutely very very large consequences clearly with app sec what is it that makes getting this right so difficult is it this is a newer field that people haven't been really considering in the past because they're like oh let's just make it work and then we'll make it secure you know what is brought about these failures
Tanya Janca 07:31
i would say there's a couple things so i took a computer science in the 90s and there was no security class and you would think by now that there will be tons of security class in universities or college no so when i talk to students i guess lecture at universities quite often and i'll ask them so who here has heard of application security before and no hands go up and like who here has heard of dev sec ops before or devops and no hands go up oh boy and then yeah and then i'm like who here has heard of the confidentiality integrity and availability the cia the mandate of every security team and there's like one cautious and the back and that upsets me i know i'm like throwing the gauntlet down a bit but academia just has they're teaching and graduating like a million computer science and software engineering students around the planet every year approximately and teaching them how to make insecure apps like if you look at the boot camps i know that you're not like an app tech person you're hardcore security in general person but like the hello world app that every single boot camp every single new language they teach you and the first thing they teach you is to put stuff on the screen that says hello world with the second lesson is always how can we read input from the screen and so they teach everyone to read the input not validated or check it to see if it's dangerous and then they teach them to output it right onto the screen with output encoding and that is the exact recipe for cross site scripting and then we're like why do we have cross site scripting in every app i'm like because we've done well with the hello world but
John Hubbard 09:15
yeah i mean i've done that myself right i went to see in various coding classes and yeah i was definitely never taught any of that stuff what do you think that is is that because it's the mindset is like hey let's just show them how to make it work first and then later on we'll teach security but that day never comes like where would you put this in a curriculum for a coder going through you know a university or whatever
Tanya Janca 09:35
i would say it totally makes sense to not teach them in the very first lesson about security absolutely and teach them the idea of input and output and all of those things but i would intertwine little tiny lessons from the beginning because i'm very biased and i love arcsec but also because teaching someone how to do something wrong repeatedly for a long time means that Their muscle memory is like, I don't know if you've ever played an instrument, but I play drums and guitar. And so like, once I learned how to do basic drum beats, then I could just sing at the same time. But if I needed to learn, like a more complicated drum beat, I couldn't sing anymore. And so if we roll into them the wrong way, for a long time, their muscle memory, we'll keep going back to that, unfortunately. So again, like me throwing the gauntlet down with academia, I feel like it's a big pyramid scheme. So the way that you know, you have to be a PhD to teach to become tenured, and only some people become tenured. And then if you aren't willing to go through their big scheme, then you teach as an adjunct professor, and do you know how much adjunct professors get paid? It's basically volunteer work. I've had all these universities reach out to me, and they're like, Oh, yeah, we want you to teach appsec for us. Great. That sounds awesome. I have an amazing course. I have a bunch of courses, I wrote a book. I'm like, yeah, I'm down. And then they're like, we would like to pay you 4000 Canadian dollars, which is approximately 3000 American dollars. And I'm like, okay, and they're like, so you're going to teach two times per week. And then mark courses, plus, there's a lab time, so it'll be around 1215 hours per week for 16 weeks. And I'm like, Oh, so the $4,000 is per week? And they're like, no, that's for the whole thing. And I'm just, I can work at McDonald's and make better buddy, are you kidding? And they're like, well, you don't have a PhD. So you're not valid as a human being, because we have this pyramid scheme. And you haven't gone through it, Tanya. And so it's like, they're trying to really hard to protect this old system, rather than doing what's actually right for their students, which would be making sure that they're being taught new things. Like, I literally talked to a bunch of computer science students, I guess, lectured in person the last time in January of 2020. And I said, Okay, so you know, I'm going to explain app sack and dev sec ops. And then I was like, does anyone know what DevOps is? And they had never heard of it? And then I said, Does anyone know what agile is? And one of them said, I heard about it on my co op term. I'm like, are they teaching you waterfall? And they're like, yeah, and then I looked at the teacher, and I must have given like, the desks there, because he actually went like this. Like, so the teacher, he was the cyber security teacher. And he actually is the seaso for the province that I live in, in Canada, and has been basically volunteering, to teach at the university. So they can learn some security. And so he's covering CIA identity and all those things. And he's like, I basically volunteer at the university, because I just can't stand by and not have this happen. And he's like, I would teach them DevOps. If I knew that. I know, I know. But we can't expect that people are just going to volunteer their time for universities that make billions of dollars, Hey, could you come do this for free? And do that for free? I'm like, No, you're rich. I know. Because they're like, could you come in and do this for free? And that for free? And I'm like, No, I'm gonna do that for community group instead, where they like everyone attend for free you in charge the students like 5000 bucks per year, you can pay me so I have some feelings, obviously.
John Hubbard 13:27
So it's it. Yeah, it's a multi faceted problem, right?
Tanya Janca 13:31
Some of the problem is, is that we're not sharing information, maybe as much as we could because of capitalism. So like, I feel like, for instance, the Canadian government has thesis and thesis is full of our thesis and CSC. So CSAC, the Canadian security establishment, the Canadian aviation security status for Canada, I'm so embarrassed, because I know so many people don't work there. It's full of like, these super brilliant geniuses that know all about it. And they teach Canadians and they're like it, we're not a school, like we have to secure Canada, we're kind of busy like that. You're smart.
John Hubbard 14:13
Yeah. And then it kind of falls into that, like, you know, you just have this subset of people who have the time and are willing to teach, you know, for maybe a less than they could make otherwise. And so, we have an underserved segment of students, right, with all these people out there that are taking different opportunities. Yeah, that's it's an interesting kind of perspective and highlights, I think, you know, that's a problem. That's probably common in a lot of different areas. Right. Moving on to DevOps. How would you summarize what DevOps is and how that leads into dev sec ops and where we are today with this, you know, the modern way of creating an application and making sure you're doing it right and efficiently and quickly.
Tanya Janca 14:55
So DevOps is a modern way of making software. So it's partially about culture so you can't really do devops properly unless you also have a culture around devops it's where dev and ops don't work in silos anymore so ops being the people that support the apps and the operating systems and all of the platforms basically that the software lives on and the devs making the software and maintaining like writing bug fixes etc and so they've sort of joined forces usually into one team and the idea is is that they automate as much as possible they get feedback as quickly as possible wherever they do they work to try to make the entire system faster as opposed to just their part because i remember back in the day being like i made an app toss it over the wall to ops that's their problem now but now we don't do that now ops is our friends and you follow so i follow the practices of jean kim patrick dubois all those folks that wrote the devops handbook accelerate the phoenix project the unicorn project with the idea that there are three ways of devops so the first way speed of the entire system so not just your part very fast feedback which for security is awesome because we want to give them security feedback as soon as we are able to and then continuous learning and making time to improve your daily work so previously what we did with software like waterfall is we would talk to the client and ask them what they want and then we would proceed to build for six months nine months a year and never get feedback like no you can't see yeah it's not done it's not done and then you know we would show it to them they'd be like what is this that's not what we asked for and if you look online you can see stats anywhere from 30 to 70% of waterfall software projects fail yeah and devops projects do significantly better and so yeah devops quite often you like to do devops you use an automation to release your software and automation for testing your software automation for tracking every single thing you can and it just it makes life a lot more fun automation like making things automated is super fun and then not doing boring repetitive work is also super awesome and there's less errors so you can tell i really liked devops
John Hubbard 17:25
yeah i mean we're all about that in any kind of security team as well it's like we don't want to click the buttons we don't have to click right like let the machines do the machine work let the people do the people work right and so maybe that's a good segue into then bringing the word second to it right because that's devops right and so obviously that sec ops then is bringing in the security team and how do all of those three teams then work together to form this new world of now dev sec ops
Tanya Janca 17:51
exactly and also basically it's adjusting whatever your application security activities are to fit into the processes with the devops teams right so i've seen organizations where the security team is like they're over there doing their dev ops and i wanted to do a code freeze for three weeks and they laughed it's like yeah because if dev and ops are doing sprints your work needs to fit into sprint's if dev and ops are using a pipeline some of your work needs to be in a pipeline and you need to get to know that pipeline and so if they have these processes that are working really well you want to weave your security through them and that's dev sec ops the automation of application security and the reconfiguring of our brains so that we're supporting the way they're doing stuff as opposed to kind of like fighting and clashing with them
John Hubbard 18:42
yeah as opposed to like trying to you know at the end when they think they're done and be like oh also what about security and then everything just kind of breaks and it delays problems right i think the word that's used in the terminology is shifting left pushing left whatever you want to call it right it's a way of phrasing that
Tanya Janca 18:58
we want to start security earlier in the system development lifecycle and be there every step of the way with you in a friendly way not a creepy way
John Hubbard 19:09
so in a practical sense if a you know development team has been given here we need this new application new code for a web app or whatever it is how does the security team get involved in like every step of the way could you give us an example of what that would actually look like
Tanya Janca 19:24
yeah so if there's a project starting i'm assuming COVID wasn't a thing i would walk on over to the project manager be like hi i'm tanya i'm your security person for this project what's up i would like to come to your project kickoff meeting and then i would announce that the kickoff meeting i'm your guy if you need any security things i'm here for you i'm going to add some security requirements to your project so that from the beginning you know what i desired to have happen and then when you go through your design i want to come hang out with you for one of your white board sessions. And then I want to point out potential security problems. And then I want us to fix them together. And then, you know, when you're doing secure, like coding, I want to enforce them secure coding things with you. So I'm going to talk about like, hey, these are some things I'd like to see. And oh, can I guess that? Or can I show you this cool tool. And I also want to talk to the project manager about adding a security sprint, depending upon how long the project is. So if it's a six month project, maybe one security sprint around two thirds through, so I'll have found a bunch of bugs and things by then. And it's like, I want you to just fix bugs for two weeks for me. But if it's not in the schedule, it's not going to happen, right. So I feel like app sex a lot about relationships and talking to people and influence and not annoying the crap out of depths. Because I had a meeting just yesterday with one of my clients. And, you know, we're meeting with one of the super high up devs. And I was explaining to him like, Listen, this is a negotiation, I don't have the power to actually force you to fix the things I have seen devs go in and mark all my bugs as fixed. And I know they're not fixed. So I want to have a real conversation with you about what can actually happen and what can happen later. Because I know you have a deadline. But you know that I'm worried about my look, there's nothing worse as a security consultant if your company gets hit, and it was something you knew about that you did not fix. I'm like, you know, we were a different client, we installed checkmarks and scanned everything, and it found 40,000 problems. Yeah. So he, we all laughed, and then I was explaining to him. So I'm going to like tear through a bunch of these and come up with things that actually worry me and ask you to fix them. And I'm like, and then in four weeks from now, I'm going to come and see you again and be like, Oh, yeah, there's this other thing. And he's like, but it's the same thing you already found. I'm like, I know. But if I can't you 25 things and ask you to fix them, you're going to get stressed out because when I was a Deb, I didn't like that. So I'm going to ask you about the most important three things, give you time to fix them, test them that they're okay. And then I'm going to go and give you three more things. And I'm just going to keep slowly giving them to you. Rather than giving you this giant report that I know you can't do because this app is in maintenance. This isn't a thing that you open and fix every day. But every time you open it, oh, I want you to fix one of my bugs, like scary than the last.
John Hubbard 22:34
If I can summarize that it's making your presence known, like from the start, right and saying we are security people, you have security questions, we are here to answer those questions, maybe helping them with those unknown unknowns, right? Like you don't know what you don't know about security, because they're not security people. Right? So just saying like, Hey, have you thought of this, you know, in that right kind of spot, and then facilitating what I assume is testing and automation with security tools, and then maybe helping them understand and interpret those results? Is that how that goes?
Tanya Janca 23:04
Yeah, I would also say, it's a lot of places I worked. When I was a dev, there was no guidance about what the security team wanted from us. So they'd always say, No, not that you suck. What am I supposed to do? You should know. And so I try really hard to do the opposite of that. So literally, every new project, here's your security requirements, every single new design and like here's design concepts that I'd like to see applied, can you pick one that we can put in like defense in depth, or least privilege etc? Like I see an opportunity here. I want a threat model, even if it's just super informal. And it's one hour with the dev and business person to talk about what we can see whiteboard it, is there encryption here. Do you have authorization and authentication there? Are you validating the input from that API that's from our third party provider? Because like, what if that's not good? And I just I have like a list of questions I just asked. And then we fixed those things. So like every step of the way, so for when they're coding secure coding guidelines, like I need these 17 things from you as a minimum. And so every single step, I have something for them in sdlc. And I know that people say, well, DevOps, no, it's continuous. Yeah, I know. I know. So, but you still need requirements. And so if you're going to add a new feature, I'll add security requirements to your feature requirements. Right? And
John Hubbard 24:26
my shipping.
Tanya Janca 24:28
Yes, if you have templates for your org, then it's a lot of I used to joke, it was a basket. I'm like, oh, you're doing a file upload of this is in your basket. Now, what else are you doing? And then I would just, like toss things in. And then I'd be like, here's your requirements, and they're like, okay, but eventually then we had code samples to share too, right? And so like, once your auric has done a thing and like, Oh, actually, this other teams already implemented that. So you can just copy their code because we pen tested it last month, so you don't even have to write it again. I just saved you like two weeks.
John Hubbard 25:01
Very cool. So we a security team, I guess might be involved in would be not only of course being present from the start, but in like the build pipeline, or you're like putting in all of your security bag of tricks, right. And you know, your bank at compile any version that runs through code analysis, it goes through your volume scanner, and all that kind of stuff is happening like every single time,
Tanya Janca 25:21
I would say it depends. So every work has a different level of security they need and they have a different level of speed that they need. So if a team releases once a week, I'm gonna put every single big thing in there. But if a team releases 12 times per day, it's like, okay, so I'm going to do a full SAS scan, like static application security testing every Friday. And I'm going to look at the results on Monday and tell you, but I'm just going to test the little part that you just changed on every release, right? Because with SAS to get a good picture, you want to do a full scan, but a full scan every time means devs, don't like me anymore. It can also be so for instance, you can do things before the pipeline as well. So let's say someone's going to check in code, you can do something called a pre commit hook. So as you're checking it in, you can have it scan the new part and say, You know what, that looks like a secret. It says connection string, and then it says password equals, and I'm uncomfortable with this. So I'm going to just spit your code out and not check it in until you know, this looks like a secret, could you please check yourself. And so then you can first of all stop a security incident, but also it never even got to the pipeline. So we just saved a whole bunch of time, if that makes sense. Yeah. And so there's like things you can do kind of in the pipeline, there's things you can do in the code repository. So like the codes just sitting there. And I'm like, Oh, I could just run a SAS scan against that once a week and not put it in a pipeline, and then not slow anyone down. Or if you're doing dynamic application security testing. So apps like burp suite, you know, neuroligin, or like netsparker, actionetics, etc, you can have your QA team. So there's like a lot of awesome QA teams, and I usually try to be their friends, because they're an awesome ally. Also, like, I like tech people. But specifically because I'm like, whoo, I can just do security tricks. And you can give me some of your automation files. So they'll record all these super cool gooey tests, they're all automated in something called a HAR file ha are, you can feed that in to a das scanner to a lot of them, they're on the market, and then it will just perfectly follow exactly. They're testing. And you can put that on a pipeline and spend 10 minutes with your dad scanner. And that's it. It's very directed at specific things. And then it skips everything else. And then you can test that outside the pipeline. Because sometimes basket wast. And they'll just be like scanning all over the place. And like it, especially if you're doing any sort of dynamic pages in, it's like, Oh, I got lost, I'm just going to scan forever. And it spends a lot of time, it gives you more false positives that way. And so yeah, I tend to talk to clients and see how much patients that their dev team has for them putting things in, and if they have a QA team that's already doing these awesome automation files, because that means it'll keep them up to date. Right. And so it's about like what they already have, and how well you can serve them. The hope that makes sense.
John Hubbard 28:32
Yeah, yeah, definitely. So So it sounds like it's a lot of I want to use the word social engineering, but to some extent, like working with people in the way that meeting them in the middle of what they're willing to deal with. And also finding that balance between how much can you test how frequently and you know, like, this is a short test, we'll do that every time versus this is a longer test, but we don't want to slow it down. But we still need to do it. When is the right frequency to balance with that?
Tanya Janca 28:56
Yeah. And also a lot of working with depth to help them actually fix these things. Yeah, so I've worked at a ton of places where they're doing amazing testing, like absolutely awesome. Like they have seven tools doing amazing things, no depth look at the results. Nothing's getting actioned. They're like we need 1000 patches. And so I'm like, so that's what we actually want to talk about. Like, I know, you're like Tonya likes making fancy pipelines. And I do that's fun. But I'm like, what we need to talk about is like, why aren't they fixing the things? Are we not giving them the time? Do they have no idea what these results mean? Are these results going out into some weird third party dashboard? That's not where they look at their bugs? Can we connect those two? Like what are our options here? Quite often, it's that the devs either given literally zero time to work on it, or they just don't know what to do. And like, I remember having someone you know, send me an automated scan tool report and none of it had been validated is this big mess and it was like in my receiving opinion unit felt like your app sucks here's your report just explaining to them this is what this means and like triaging the results with them okay so like that's actually not a sensitive value so we don't need to freak out that it's in the code all suppress that result from now on thanks for telling me right and so yeah you have to there's a social aspect to it that i really like because i'm a oddly extroverted computer science person
John Hubbard 30:29
so one of the the other things that i caught on they said was you know preventing issues and incidents like kind of before they happen and just you know obviously a little bit of prevention is going to save a ton of money downstream if you didn't do it one of the big incidents in the news right now the solar winds supply chain breach right we're gonna see a ton of stuff probably like this in the future because honestly everyone's getting better at security and it is getting harder to hack people right we're getting in zero trust we're getting into all of these awesome dev sec ops pipelines and it's getting harder so they're like i got to go upstream and infect the vendor or whatever and push it out as a trusted program how might a dev sec ops pipeline or philosophy or strategy set up in an ideal organization prevent that kind of thing from occurring you know before the point where it's you know that the backdoor has been installed and pushed out to people
Tanya Janca 31:22
so part of solar winds was that they are so crucial to so many other systems that they were a very very very high value target so they need to have a significantly higher security posture than the average company so a lot of my clients are like put solar winds and like we are so unimportant compared to solar winds you don't want to spend $25 million protecting something that costs $1 million unless it's state secrets but with that aside i think solar winds had two major problems that i'm aware of and one was that their private key was stolen and so that's about secret management right and so we briefly talked about scanning your code for secrets i don't know if the key was in the coke wash i hope not resale part of that is like secret management and how to do that really well and then the other part was that it was a supply chain attack so they put something very bad into a component that they knew that solar wind needed to use to build their software and then solar winds is used to build all these other things and so we can protect the software supply chain in a few ways so one is by using software composition analysis tools or sca and so what they do is they look at all the different components that's in your software so all the code that you did not write but that you are still putting in your app and therefore is still your risk and they basically compare it to a bad list so each sca company has like a bunch of software security researchers that will look at a bunch of the things and then they also go by like exploit db and the cve libraries and all of those and they create this stuff's bad list and then they look at oh tanya has this new get package and that one looks very bad it has these known vulnerabilities so that's one way that you can do it so i recommend that to everyone that's making software that you verify your third party components are okay with an automated tool the tools are expensive not gonna lie but they're highly accurate so it'll tell you like you have this version and we know there's this wrong with it but there's a lot of false negatives if you have something like solar wind where you need to have a very very high level of assurance you can review the components yourself which i know sounds absolutely awful and a lot of work so there's some companies like sona type and j frog where they'll actually they have a security team of security researchers that review in depth certain packages and then you can proxy basically download all your packages online from them and then basically you only get like 10% of all packages in the world to play with to build your software but they know they're secure so that's an option or you can literally review them yourself from what i understand ibm does that ibm so they can afford that and they literally as of a few years ago reviewed 100% of them themselves the average company that's just absolutely out of reach right but if you're microsoft or amazon you're some you can actually review them yourselves but for the average company i would say if you can buy a software composition analysis tool or even like manually check your repositories to let's say you're like okay so i'm a dotnet developer i have a bunch of new get packages i'm going to manually look them up on mitre.com and there's cve database and just see if they're there you will miss a lot of things but you will find something's wrong and so if you have absolutely no budget and for some reason you have tons of time to have security professionals do manual toil that's an option as well and so for instance i've seen companies in countries where the cost of labor is very very low doing things like that so there's a bunch of options that could help if i had more details specifically about how the private key was stolen we could talk about that but i don't have it and i wish the solar teams incident response team a lot of hugs
John Hubbard 35:39
actually needed
35:42
for them really do
John Hubbard 35:44
yeah and i mean to me it's really less about that specific breach and more so just that we're going into a world where everything is going to be you know infrastructure is code and serverless and containers and like all these buzzwords right all of it's ultimately going to be like you're representing what you're building and running and deploying it all the time in code and so something's got to check that and you want security part of the process right
Tanya Janca 36:08
infrastructure as code is so awesome because as a dev i'm like oh my gosh infrastructure is mine now and so you can scan it for vulnerabilities just like you can regular code you can check it into your code library and also you can do security as code so you can actually code in and make templates for all the different security configurations that you want to then you can enforce them it's beautiful really infrastructures code is cool
John Hubbard 36:34
yeah it's a real like mind shift but it's so incredibly cool and efficient like yeah i love it as a concept to just being like i can just write this text file and the whole network will spring into existence in a couple minutes that's insane right i hope it's secure
Tanya Janca 36:51
i was working at microsoft i went on microsoft ignite the tour so they did like 19 countries and i did like 10 of them and then someone else had to do my security talks at the other nine countries and so i made infrastructure as code to release out all night so like i put like garbage into azure it was like sql injection cross site scripting now where i stood like load vomit into azure and then like i showed them how to like trigger the things so that they could go investigate on stage and stuff and they're like yeah to set up tanya's app i just like click these three buttons and it set up my demo
John Hubbard 37:31
so with all this stuff you know coming out very very quickly in the last couple years obviously the pace of invention here is is super high where do you see this going in the next maybe three to five years
Tanya Janca 37:41
i see a lot more education happening i see a lot of companies investing in their employees so that they stay i see a lot of companies modernizing because otherwise the best employees will leave so that they can go do devops right like i've seen people quit and they're like i want to go work there it's like a $10,000 pay cut but they're doing cool stuff and i want to do that and so i see a lot of companies in order to keep their awesome resources like training them more and like letting them do the cool new stuff i also see a lot more breaches i'm really hopeful that the governments of our world will take up some governance on this and start making like certain demands so gdpr was a nice step the privacy stuff from california that's good canada has actually amazing privacy laws i'm hoping one day we actually ever enforced them that would be super cool we're like super pro like we're like oh you broke the law on stuff please don't do that we enforce the law yeah so i see the government's slowly catching up i hope that they'll talk to the security industry and have us weigh in rather than just have a couple staff members come together to build something so i hope they reach out and let us contribute because there's a lot of big brains in industry and not all of them work for the government but yeah these are my hopes
John Hubbard 39:10
yep there's a lot of work to be done it sounds like and plenty of help needed in all respects for those who are interested in this and want to get into it where can we find additional information resources on this and where can we find all of your stuff online
Tanya Janca 39:23
okay so i wrote a book called alice and bob learn application security and it is basically for any dev or anyone who works in it you can read it and then understand all the basics of app sec and i wrote it to i don't know if you know john you do not make money when you write a book but i was just like i need people to know this so if you're a dev out there listening to this please read my book i run a training academy on like how to do application security and it's called we hack purple so if you go to we hack purple calm there's that every monday on twitter i run a program i guess called cyber mentoring monday so if you use the hashtag cyber mentoring monday every monday a whole bunch of us now across the industry offer to connect people with professional mentors so if you have two years or more experience with what you're doing you could be a mentor even to just tell someone what it's like to do your job so they understand if they want that job or not or recommend a book or meet with someone online and have a virtual coffee with them and talk about you know why you chose your career because that might be the information that helps them decide so if someone wants to get into this they should use that hashtag to try to meet people in the industry who will help bring them in my book at the end of each chapter there's questions like if you are going to secure a car operating system what would you do and so there's an answer key but it's very minimal and so starting march 20 i am going to do free online open discussions for three hours once a month and i'm going to invite a bunch of guests from the industry to discuss the questions and the public are invited to watch and interact with us via youtube so if you want an invite and it's all free because i'm really good at absecon not very good at business but basically go to alice and bob learn calm and then you know give me your email and i'm going to send you invites to all 11 sessions so we're going to finish around a week before christmas this year and we're going to go through all of them and discuss every single question if you read the book you'll understand better but if you don't you'll still learn a lot because basically i'm going to invite you know lots of people from class people from just different areas based on whatever chapter it is so we can talk about it and hopefully help spread the word of secure software
John Hubbard 41:54
fantastic well thank you very much so we've got a book we've got we have purple the website we've got the mentoring anything else you want to shout out before we close it up here
Tanya Janca 42:04
um my company's giving away a free mini course with app sec so there's always like one or two or three app sec people and then 1000s of devs and so you have to scale your team so i made like this little mini course that we're giving away for free about how to scale your security program without extra security dollars so if you go to newsletter.we hack purple.com like you join our mailing list and we're sending out invites and then next month we're going to run the free instant response course as well and it's how to respond to incidents that have to do with software because it's different and like how to prepare prevent incidents and yeah marketing teams with we just gave a lot away for free and like i thought we were already doing that
John Hubbard 42:51
awesome well very very cool thank you so much for all the insight on app sec and def sec ops hopefully we've inspired some folks to chase this path as a career sounds like it's going to be something that's going to be needed far into the future so thanks very much again for appearing on the podcast and we will see you around
43:10
thank you so much john
John Hubbard 43:11
alright catch you later hey blue teamers hope you enjoyed today's episode of blueprint if you've got a second and wants to help support the podcast please subscribe and leave us a review on apple podcasts it would be really really meaningful to us and if you have any ideas or suggestions i would love to hear them your views are going to be one of the best ways to help others find this podcast so anything you could do would be a big help as always thank you for listening you can connect to me on social at sec hub sec hq bb on twitter or on linkedin so until next time thank you for listening to the blueprint podcast
support for the blueprint podcast comes from the sans institute if you like the topics covered in this podcast and would like to learn more about blue team fundamentals such as hosting network data collection threat detection alert triage threat intelligence and more check out my new course sek 450 blue team fundamentals this course is designed to bring attendees the information that every sock analyst and blue team member needs to know to hit the ground running including 15 labs to get you hands on with tools for threat intelligence sim incident management automation and much much more this course is everything you need to launch your blue team career check out the details at sans url.com slash 450 hope to see you in class
00:40
this is the blueprint podcast bringing you the latest in cyber defense and security operations from top blue team leaders blueprint is brought to you by the sans institute hosted by sans certified instructor john houbara now here's your host john koba
John Hubbard 01:02
in this episode of the blueprint podcast we speak with tanya janka of wiac purple about app sec devops and dev sec ops we discussed what they are why you need to know about them and the consequences of ignoring these new and critical fields even if you aren't a programmer these technologies are quickly paving the way into the future of it operations and not knowing what they are means you'll quickly be left behind which of course no one wants so buckle in and let's discuss your role as a blue teamer in the new world of dev sec ops today on the blueprint podcast all right so today on the podcast we have founder and ceo of the we have purple academy accomplished application security author podcast host dev sec ops expert tanya janka tanya thank you so much for making time in what i'm sure is a very busy schedule to be on the podcast today
Tanya Janca 01:54
thanks for having me john
John Hubbard 01:57
so as i have hinted at with the intro here you have your hands in a ton of different subjects and expertise is could you give our listeners a little bit about your background and your expertise
Tanya Janca 02:09
sure i was a dev for a really long time i think around 17 years and then i met a hacker and he said you should work in security and i was like no security is lame writing software is the best i've also been a professional musician for most of my life and he was in a band too so then obviously our bands had to play together and we became friends and for a year and a half he just broke me down slowly until i agree to try to become a pen tester very opposite of most people's interaction with how to get into security and then i was briefly a pen tester before i discovered that there's this field called application security which is sort of halfway between the security team and the dev team if that makes sense and then and also that you got to be social you got to hang out with dabs you still got to smash things i think this is the best of all the worlds and so i've been doing application security for quite a few years now and just totally loving it
John Hubbard 03:14
very very cool so yeah everyone always has an interesting story on how they get an infosec i live here in the various routes that people take so absecon and devops in this kind of world that you live in is not something we've covered on this podcast before so i wanted to make sure before we kind of dive into the meat of it for those listeners who might not also be in this world and i'll start off by saying i'm not really in that world specifically so excuse if i have dumb questions app sec devops and all of that but let's start off with what is application security to you and like a day to day basis
Tanya Janca 03:49
yeah so application security the idea of it is it's all the things that you do to try to make sure that the custom software or even the software you buy is actually secure or it's add a security posture that you feel safe using it and so that can mean for me teaching a whole bunch of devs about a mistake that i keep seeing in their code it can be about adding some extra security tests to their pipeline it can mean me doing some security testing or automating a bunch of tests to go out and then like reviewing the results talking to the dev about the results and then usually quite a bit of negotiation about i know you have this deadline friday but there's one critical that really has me concerned so there's like a lot of social stuff and talking and persuasion so you might be awesome at sales if you're awesome at apps so it's kind of that job where you're the one that you know kind of make sure all the steps happen during the system development lifecycle so that your app is in good shape at the end and you sort of try to teach the devs and enable them to make secure app
John Hubbard 05:00
gotcha so i'm sure i know the answer to this but to make it explicit what are the consequences or issues that arise if you do not pay attention to this what's the common like big impacts you see for failures
Tanya Janca 05:13
so i've seen a lot i've seen some where there's going to be an acquisition and merger and then the company looks at all the apps are really old and legacy and awful and they're just totally dumpster fires of security problems and then the acquisition doesn't happen i've seen it where just so many companies are dealing with security incidents basically all the time their staff super stressed out and then that means they can't do all the projects and normal stuff they're supposed to be doing i've seen quite often we've heard of ransomware we've heard of you know malware and stuff so there are different types of attacks that happen on software than that but basically where there are attacks where people can try to steal the data out of your app or they can try to take the data and change it or they can bring your app down so i can't get back up stealing secrets basically like all the badness that could happen with a computer can definitely happen with software
John Hubbard 06:12
so i was expecting the technical like obviously security breaches john answer but you took it much further than that right like your business you know m&a is might be broken for bad apps like like that's a serious high level like full business visibility problem
Tanya Janca 06:25
yeah i consult at a place and so we would advise on acquisitions and mergers and they actually took $10 million off of the price because of how much app sec they were going to have to do they're like there's no app sec program where you work none of your staff have ever been trained all your apps are just like we ran a scanner on it and it looked like a christmas tree turning on and we're just so it has financial consequences and also consequences for real people so it can result in for instance someone attacked a water system in the united states and poisoned i think 15,000 people recently and that could have resulted in death it's absolutely terrifying until all of this is software
John Hubbard 07:13
yeah yeah absolutely very very large consequences clearly with app sec what is it that makes getting this right so difficult is it this is a newer field that people haven't been really considering in the past because they're like oh let's just make it work and then we'll make it secure you know what is brought about these failures
Tanya Janca 07:31
i would say there's a couple things so i took a computer science in the 90s and there was no security class and you would think by now that there will be tons of security class in universities or college no so when i talk to students i guess lecture at universities quite often and i'll ask them so who here has heard of application security before and no hands go up and like who here has heard of dev sec ops before or devops and no hands go up oh boy and then yeah and then i'm like who here has heard of the confidentiality integrity and availability the cia the mandate of every security team and there's like one cautious and the back and that upsets me i know i'm like throwing the gauntlet down a bit but academia just has they're teaching and graduating like a million computer science and software engineering students around the planet every year approximately and teaching them how to make insecure apps like if you look at the boot camps i know that you're not like an app tech person you're hardcore security in general person but like the hello world app that every single boot camp every single new language they teach you and the first thing they teach you is to put stuff on the screen that says hello world with the second lesson is always how can we read input from the screen and so they teach everyone to read the input not validated or check it to see if it's dangerous and then they teach them to output it right onto the screen with output encoding and that is the exact recipe for cross site scripting and then we're like why do we have cross site scripting in every app i'm like because we've done well with the hello world but
John Hubbard 09:15
yeah i mean i've done that myself right i went to see in various coding classes and yeah i was definitely never taught any of that stuff what do you think that is is that because it's the mindset is like hey let's just show them how to make it work first and then later on we'll teach security but that day never comes like where would you put this in a curriculum for a coder going through you know a university or whatever
Tanya Janca 09:35
i would say it totally makes sense to not teach them in the very first lesson about security absolutely and teach them the idea of input and output and all of those things but i would intertwine little tiny lessons from the beginning because i'm very biased and i love arcsec but also because teaching someone how to do something wrong repeatedly for a long time means that Their muscle memory is like, I don't know if you've ever played an instrument, but I play drums and guitar. And so like, once I learned how to do basic drum beats, then I could just sing at the same time. But if I needed to learn, like a more complicated drum beat, I couldn't sing anymore. And so if we roll into them the wrong way, for a long time, their muscle memory, we'll keep going back to that, unfortunately. So again, like me throwing the gauntlet down with academia, I feel like it's a big pyramid scheme. So the way that you know, you have to be a PhD to teach to become tenured, and only some people become tenured. And then if you aren't willing to go through their big scheme, then you teach as an adjunct professor, and do you know how much adjunct professors get paid? It's basically volunteer work. I've had all these universities reach out to me, and they're like, Oh, yeah, we want you to teach appsec for us. Great. That sounds awesome. I have an amazing course. I have a bunch of courses, I wrote a book. I'm like, yeah, I'm down. And then they're like, we would like to pay you 4000 Canadian dollars, which is approximately 3000 American dollars. And I'm like, okay, and they're like, so you're going to teach two times per week. And then mark courses, plus, there's a lab time, so it'll be around 1215 hours per week for 16 weeks. And I'm like, Oh, so the $4,000 is per week? And they're like, no, that's for the whole thing. And I'm just, I can work at McDonald's and make better buddy, are you kidding? And they're like, well, you don't have a PhD. So you're not valid as a human being, because we have this pyramid scheme. And you haven't gone through it, Tanya. And so it's like, they're trying to really hard to protect this old system, rather than doing what's actually right for their students, which would be making sure that they're being taught new things. Like, I literally talked to a bunch of computer science students, I guess, lectured in person the last time in January of 2020. And I said, Okay, so you know, I'm going to explain app sack and dev sec ops. And then I was like, does anyone know what DevOps is? And they had never heard of it? And then I said, Does anyone know what agile is? And one of them said, I heard about it on my co op term. I'm like, are they teaching you waterfall? And they're like, yeah, and then I looked at the teacher, and I must have given like, the desks there, because he actually went like this. Like, so the teacher, he was the cyber security teacher. And he actually is the seaso for the province that I live in, in Canada, and has been basically volunteering, to teach at the university. So they can learn some security. And so he's covering CIA identity and all those things. And he's like, I basically volunteer at the university, because I just can't stand by and not have this happen. And he's like, I would teach them DevOps. If I knew that. I know, I know. But we can't expect that people are just going to volunteer their time for universities that make billions of dollars, Hey, could you come do this for free? And do that for free? I'm like, No, you're rich. I know. Because they're like, could you come in and do this for free? And that for free? And I'm like, No, I'm gonna do that for community group instead, where they like everyone attend for free you in charge the students like 5000 bucks per year, you can pay me so I have some feelings, obviously.
John Hubbard 13:27
So it's it. Yeah, it's a multi faceted problem, right?
Tanya Janca 13:31
Some of the problem is, is that we're not sharing information, maybe as much as we could because of capitalism. So like, I feel like, for instance, the Canadian government has thesis and thesis is full of our thesis and CSC. So CSAC, the Canadian security establishment, the Canadian aviation security status for Canada, I'm so embarrassed, because I know so many people don't work there. It's full of like, these super brilliant geniuses that know all about it. And they teach Canadians and they're like it, we're not a school, like we have to secure Canada, we're kind of busy like that. You're smart.
John Hubbard 14:13
Yeah. And then it kind of falls into that, like, you know, you just have this subset of people who have the time and are willing to teach, you know, for maybe a less than they could make otherwise. And so, we have an underserved segment of students, right, with all these people out there that are taking different opportunities. Yeah, that's it's an interesting kind of perspective and highlights, I think, you know, that's a problem. That's probably common in a lot of different areas. Right. Moving on to DevOps. How would you summarize what DevOps is and how that leads into dev sec ops and where we are today with this, you know, the modern way of creating an application and making sure you're doing it right and efficiently and quickly.
Tanya Janca 14:55
So DevOps is a modern way of making software. So it's partially about culture so you can't really do devops properly unless you also have a culture around devops it's where dev and ops don't work in silos anymore so ops being the people that support the apps and the operating systems and all of the platforms basically that the software lives on and the devs making the software and maintaining like writing bug fixes etc and so they've sort of joined forces usually into one team and the idea is is that they automate as much as possible they get feedback as quickly as possible wherever they do they work to try to make the entire system faster as opposed to just their part because i remember back in the day being like i made an app toss it over the wall to ops that's their problem now but now we don't do that now ops is our friends and you follow so i follow the practices of jean kim patrick dubois all those folks that wrote the devops handbook accelerate the phoenix project the unicorn project with the idea that there are three ways of devops so the first way speed of the entire system so not just your part very fast feedback which for security is awesome because we want to give them security feedback as soon as we are able to and then continuous learning and making time to improve your daily work so previously what we did with software like waterfall is we would talk to the client and ask them what they want and then we would proceed to build for six months nine months a year and never get feedback like no you can't see yeah it's not done it's not done and then you know we would show it to them they'd be like what is this that's not what we asked for and if you look online you can see stats anywhere from 30 to 70% of waterfall software projects fail yeah and devops projects do significantly better and so yeah devops quite often you like to do devops you use an automation to release your software and automation for testing your software automation for tracking every single thing you can and it just it makes life a lot more fun automation like making things automated is super fun and then not doing boring repetitive work is also super awesome and there's less errors so you can tell i really liked devops
John Hubbard 17:25
yeah i mean we're all about that in any kind of security team as well it's like we don't want to click the buttons we don't have to click right like let the machines do the machine work let the people do the people work right and so maybe that's a good segue into then bringing the word second to it right because that's devops right and so obviously that sec ops then is bringing in the security team and how do all of those three teams then work together to form this new world of now dev sec ops
Tanya Janca 17:51
exactly and also basically it's adjusting whatever your application security activities are to fit into the processes with the devops teams right so i've seen organizations where the security team is like they're over there doing their dev ops and i wanted to do a code freeze for three weeks and they laughed it's like yeah because if dev and ops are doing sprints your work needs to fit into sprint's if dev and ops are using a pipeline some of your work needs to be in a pipeline and you need to get to know that pipeline and so if they have these processes that are working really well you want to weave your security through them and that's dev sec ops the automation of application security and the reconfiguring of our brains so that we're supporting the way they're doing stuff as opposed to kind of like fighting and clashing with them
John Hubbard 18:42
yeah as opposed to like trying to you know at the end when they think they're done and be like oh also what about security and then everything just kind of breaks and it delays problems right i think the word that's used in the terminology is shifting left pushing left whatever you want to call it right it's a way of phrasing that
Tanya Janca 18:58
we want to start security earlier in the system development lifecycle and be there every step of the way with you in a friendly way not a creepy way
John Hubbard 19:09
so in a practical sense if a you know development team has been given here we need this new application new code for a web app or whatever it is how does the security team get involved in like every step of the way could you give us an example of what that would actually look like
Tanya Janca 19:24
yeah so if there's a project starting i'm assuming COVID wasn't a thing i would walk on over to the project manager be like hi i'm tanya i'm your security person for this project what's up i would like to come to your project kickoff meeting and then i would announce that the kickoff meeting i'm your guy if you need any security things i'm here for you i'm going to add some security requirements to your project so that from the beginning you know what i desired to have happen and then when you go through your design i want to come hang out with you for one of your white board sessions. And then I want to point out potential security problems. And then I want us to fix them together. And then, you know, when you're doing secure, like coding, I want to enforce them secure coding things with you. So I'm going to talk about like, hey, these are some things I'd like to see. And oh, can I guess that? Or can I show you this cool tool. And I also want to talk to the project manager about adding a security sprint, depending upon how long the project is. So if it's a six month project, maybe one security sprint around two thirds through, so I'll have found a bunch of bugs and things by then. And it's like, I want you to just fix bugs for two weeks for me. But if it's not in the schedule, it's not going to happen, right. So I feel like app sex a lot about relationships and talking to people and influence and not annoying the crap out of depths. Because I had a meeting just yesterday with one of my clients. And, you know, we're meeting with one of the super high up devs. And I was explaining to him like, Listen, this is a negotiation, I don't have the power to actually force you to fix the things I have seen devs go in and mark all my bugs as fixed. And I know they're not fixed. So I want to have a real conversation with you about what can actually happen and what can happen later. Because I know you have a deadline. But you know that I'm worried about my look, there's nothing worse as a security consultant if your company gets hit, and it was something you knew about that you did not fix. I'm like, you know, we were a different client, we installed checkmarks and scanned everything, and it found 40,000 problems. Yeah. So he, we all laughed, and then I was explaining to him. So I'm going to like tear through a bunch of these and come up with things that actually worry me and ask you to fix them. And I'm like, and then in four weeks from now, I'm going to come and see you again and be like, Oh, yeah, there's this other thing. And he's like, but it's the same thing you already found. I'm like, I know. But if I can't you 25 things and ask you to fix them, you're going to get stressed out because when I was a Deb, I didn't like that. So I'm going to ask you about the most important three things, give you time to fix them, test them that they're okay. And then I'm going to go and give you three more things. And I'm just going to keep slowly giving them to you. Rather than giving you this giant report that I know you can't do because this app is in maintenance. This isn't a thing that you open and fix every day. But every time you open it, oh, I want you to fix one of my bugs, like scary than the last.
John Hubbard 22:34
If I can summarize that it's making your presence known, like from the start, right and saying we are security people, you have security questions, we are here to answer those questions, maybe helping them with those unknown unknowns, right? Like you don't know what you don't know about security, because they're not security people. Right? So just saying like, Hey, have you thought of this, you know, in that right kind of spot, and then facilitating what I assume is testing and automation with security tools, and then maybe helping them understand and interpret those results? Is that how that goes?
Tanya Janca 23:04
Yeah, I would also say, it's a lot of places I worked. When I was a dev, there was no guidance about what the security team wanted from us. So they'd always say, No, not that you suck. What am I supposed to do? You should know. And so I try really hard to do the opposite of that. So literally, every new project, here's your security requirements, every single new design and like here's design concepts that I'd like to see applied, can you pick one that we can put in like defense in depth, or least privilege etc? Like I see an opportunity here. I want a threat model, even if it's just super informal. And it's one hour with the dev and business person to talk about what we can see whiteboard it, is there encryption here. Do you have authorization and authentication there? Are you validating the input from that API that's from our third party provider? Because like, what if that's not good? And I just I have like a list of questions I just asked. And then we fixed those things. So like every step of the way, so for when they're coding secure coding guidelines, like I need these 17 things from you as a minimum. And so every single step, I have something for them in sdlc. And I know that people say, well, DevOps, no, it's continuous. Yeah, I know. I know. So, but you still need requirements. And so if you're going to add a new feature, I'll add security requirements to your feature requirements. Right? And
John Hubbard 24:26
my shipping.
Tanya Janca 24:28
Yes, if you have templates for your org, then it's a lot of I used to joke, it was a basket. I'm like, oh, you're doing a file upload of this is in your basket. Now, what else are you doing? And then I would just, like toss things in. And then I'd be like, here's your requirements, and they're like, okay, but eventually then we had code samples to share too, right? And so like, once your auric has done a thing and like, Oh, actually, this other teams already implemented that. So you can just copy their code because we pen tested it last month, so you don't even have to write it again. I just saved you like two weeks.
John Hubbard 25:01
Very cool. So we a security team, I guess might be involved in would be not only of course being present from the start, but in like the build pipeline, or you're like putting in all of your security bag of tricks, right. And you know, your bank at compile any version that runs through code analysis, it goes through your volume scanner, and all that kind of stuff is happening like every single time,
Tanya Janca 25:21
I would say it depends. So every work has a different level of security they need and they have a different level of speed that they need. So if a team releases once a week, I'm gonna put every single big thing in there. But if a team releases 12 times per day, it's like, okay, so I'm going to do a full SAS scan, like static application security testing every Friday. And I'm going to look at the results on Monday and tell you, but I'm just going to test the little part that you just changed on every release, right? Because with SAS to get a good picture, you want to do a full scan, but a full scan every time means devs, don't like me anymore. It can also be so for instance, you can do things before the pipeline as well. So let's say someone's going to check in code, you can do something called a pre commit hook. So as you're checking it in, you can have it scan the new part and say, You know what, that looks like a secret. It says connection string, and then it says password equals, and I'm uncomfortable with this. So I'm going to just spit your code out and not check it in until you know, this looks like a secret, could you please check yourself. And so then you can first of all stop a security incident, but also it never even got to the pipeline. So we just saved a whole bunch of time, if that makes sense. Yeah. And so there's like things you can do kind of in the pipeline, there's things you can do in the code repository. So like the codes just sitting there. And I'm like, Oh, I could just run a SAS scan against that once a week and not put it in a pipeline, and then not slow anyone down. Or if you're doing dynamic application security testing. So apps like burp suite, you know, neuroligin, or like netsparker, actionetics, etc, you can have your QA team. So there's like a lot of awesome QA teams, and I usually try to be their friends, because they're an awesome ally. Also, like, I like tech people. But specifically because I'm like, whoo, I can just do security tricks. And you can give me some of your automation files. So they'll record all these super cool gooey tests, they're all automated in something called a HAR file ha are, you can feed that in to a das scanner to a lot of them, they're on the market, and then it will just perfectly follow exactly. They're testing. And you can put that on a pipeline and spend 10 minutes with your dad scanner. And that's it. It's very directed at specific things. And then it skips everything else. And then you can test that outside the pipeline. Because sometimes basket wast. And they'll just be like scanning all over the place. And like it, especially if you're doing any sort of dynamic pages in, it's like, Oh, I got lost, I'm just going to scan forever. And it spends a lot of time, it gives you more false positives that way. And so yeah, I tend to talk to clients and see how much patients that their dev team has for them putting things in, and if they have a QA team that's already doing these awesome automation files, because that means it'll keep them up to date. Right. And so it's about like what they already have, and how well you can serve them. The hope that makes sense.
John Hubbard 28:32
Yeah, yeah, definitely. So So it sounds like it's a lot of I want to use the word social engineering, but to some extent, like working with people in the way that meeting them in the middle of what they're willing to deal with. And also finding that balance between how much can you test how frequently and you know, like, this is a short test, we'll do that every time versus this is a longer test, but we don't want to slow it down. But we still need to do it. When is the right frequency to balance with that?
Tanya Janca 28:56
Yeah. And also a lot of working with depth to help them actually fix these things. Yeah, so I've worked at a ton of places where they're doing amazing testing, like absolutely awesome. Like they have seven tools doing amazing things, no depth look at the results. Nothing's getting actioned. They're like we need 1000 patches. And so I'm like, so that's what we actually want to talk about. Like, I know, you're like Tonya likes making fancy pipelines. And I do that's fun. But I'm like, what we need to talk about is like, why aren't they fixing the things? Are we not giving them the time? Do they have no idea what these results mean? Are these results going out into some weird third party dashboard? That's not where they look at their bugs? Can we connect those two? Like what are our options here? Quite often, it's that the devs either given literally zero time to work on it, or they just don't know what to do. And like, I remember having someone you know, send me an automated scan tool report and none of it had been validated is this big mess and it was like in my receiving opinion unit felt like your app sucks here's your report just explaining to them this is what this means and like triaging the results with them okay so like that's actually not a sensitive value so we don't need to freak out that it's in the code all suppress that result from now on thanks for telling me right and so yeah you have to there's a social aspect to it that i really like because i'm a oddly extroverted computer science person
John Hubbard 30:29
so one of the the other things that i caught on they said was you know preventing issues and incidents like kind of before they happen and just you know obviously a little bit of prevention is going to save a ton of money downstream if you didn't do it one of the big incidents in the news right now the solar winds supply chain breach right we're gonna see a ton of stuff probably like this in the future because honestly everyone's getting better at security and it is getting harder to hack people right we're getting in zero trust we're getting into all of these awesome dev sec ops pipelines and it's getting harder so they're like i got to go upstream and infect the vendor or whatever and push it out as a trusted program how might a dev sec ops pipeline or philosophy or strategy set up in an ideal organization prevent that kind of thing from occurring you know before the point where it's you know that the backdoor has been installed and pushed out to people
Tanya Janca 31:22
so part of solar winds was that they are so crucial to so many other systems that they were a very very very high value target so they need to have a significantly higher security posture than the average company so a lot of my clients are like put solar winds and like we are so unimportant compared to solar winds you don't want to spend $25 million protecting something that costs $1 million unless it's state secrets but with that aside i think solar winds had two major problems that i'm aware of and one was that their private key was stolen and so that's about secret management right and so we briefly talked about scanning your code for secrets i don't know if the key was in the coke wash i hope not resale part of that is like secret management and how to do that really well and then the other part was that it was a supply chain attack so they put something very bad into a component that they knew that solar wind needed to use to build their software and then solar winds is used to build all these other things and so we can protect the software supply chain in a few ways so one is by using software composition analysis tools or sca and so what they do is they look at all the different components that's in your software so all the code that you did not write but that you are still putting in your app and therefore is still your risk and they basically compare it to a bad list so each sca company has like a bunch of software security researchers that will look at a bunch of the things and then they also go by like exploit db and the cve libraries and all of those and they create this stuff's bad list and then they look at oh tanya has this new get package and that one looks very bad it has these known vulnerabilities so that's one way that you can do it so i recommend that to everyone that's making software that you verify your third party components are okay with an automated tool the tools are expensive not gonna lie but they're highly accurate so it'll tell you like you have this version and we know there's this wrong with it but there's a lot of false negatives if you have something like solar wind where you need to have a very very high level of assurance you can review the components yourself which i know sounds absolutely awful and a lot of work so there's some companies like sona type and j frog where they'll actually they have a security team of security researchers that review in depth certain packages and then you can proxy basically download all your packages online from them and then basically you only get like 10% of all packages in the world to play with to build your software but they know they're secure so that's an option or you can literally review them yourself from what i understand ibm does that ibm so they can afford that and they literally as of a few years ago reviewed 100% of them themselves the average company that's just absolutely out of reach right but if you're microsoft or amazon you're some you can actually review them yourselves but for the average company i would say if you can buy a software composition analysis tool or even like manually check your repositories to let's say you're like okay so i'm a dotnet developer i have a bunch of new get packages i'm going to manually look them up on mitre.com and there's cve database and just see if they're there you will miss a lot of things but you will find something's wrong and so if you have absolutely no budget and for some reason you have tons of time to have security professionals do manual toil that's an option as well and so for instance i've seen companies in countries where the cost of labor is very very low doing things like that so there's a bunch of options that could help if i had more details specifically about how the private key was stolen we could talk about that but i don't have it and i wish the solar teams incident response team a lot of hugs
John Hubbard 35:39
actually needed
35:42
for them really do
John Hubbard 35:44
yeah and i mean to me it's really less about that specific breach and more so just that we're going into a world where everything is going to be you know infrastructure is code and serverless and containers and like all these buzzwords right all of it's ultimately going to be like you're representing what you're building and running and deploying it all the time in code and so something's got to check that and you want security part of the process right
Tanya Janca 36:08
infrastructure as code is so awesome because as a dev i'm like oh my gosh infrastructure is mine now and so you can scan it for vulnerabilities just like you can regular code you can check it into your code library and also you can do security as code so you can actually code in and make templates for all the different security configurations that you want to then you can enforce them it's beautiful really infrastructures code is cool
John Hubbard 36:34
yeah it's a real like mind shift but it's so incredibly cool and efficient like yeah i love it as a concept to just being like i can just write this text file and the whole network will spring into existence in a couple minutes that's insane right i hope it's secure
Tanya Janca 36:51
i was working at microsoft i went on microsoft ignite the tour so they did like 19 countries and i did like 10 of them and then someone else had to do my security talks at the other nine countries and so i made infrastructure as code to release out all night so like i put like garbage into azure it was like sql injection cross site scripting now where i stood like load vomit into azure and then like i showed them how to like trigger the things so that they could go investigate on stage and stuff and they're like yeah to set up tanya's app i just like click these three buttons and it set up my demo
John Hubbard 37:31
so with all this stuff you know coming out very very quickly in the last couple years obviously the pace of invention here is is super high where do you see this going in the next maybe three to five years
Tanya Janca 37:41
i see a lot more education happening i see a lot of companies investing in their employees so that they stay i see a lot of companies modernizing because otherwise the best employees will leave so that they can go do devops right like i've seen people quit and they're like i want to go work there it's like a $10,000 pay cut but they're doing cool stuff and i want to do that and so i see a lot of companies in order to keep their awesome resources like training them more and like letting them do the cool new stuff i also see a lot more breaches i'm really hopeful that the governments of our world will take up some governance on this and start making like certain demands so gdpr was a nice step the privacy stuff from california that's good canada has actually amazing privacy laws i'm hoping one day we actually ever enforced them that would be super cool we're like super pro like we're like oh you broke the law on stuff please don't do that we enforce the law yeah so i see the government's slowly catching up i hope that they'll talk to the security industry and have us weigh in rather than just have a couple staff members come together to build something so i hope they reach out and let us contribute because there's a lot of big brains in industry and not all of them work for the government but yeah these are my hopes
John Hubbard 39:10
yep there's a lot of work to be done it sounds like and plenty of help needed in all respects for those who are interested in this and want to get into it where can we find additional information resources on this and where can we find all of your stuff online
Tanya Janca 39:23
okay so i wrote a book called alice and bob learn application security and it is basically for any dev or anyone who works in it you can read it and then understand all the basics of app sec and i wrote it to i don't know if you know john you do not make money when you write a book but i was just like i need people to know this so if you're a dev out there listening to this please read my book i run a training academy on like how to do application security and it's called we hack purple so if you go to we hack purple calm there's that every monday on twitter i run a program i guess called cyber mentoring monday so if you use the hashtag cyber mentoring monday every monday a whole bunch of us now across the industry offer to connect people with professional mentors so if you have two years or more experience with what you're doing you could be a mentor even to just tell someone what it's like to do your job so they understand if they want that job or not or recommend a book or meet with someone online and have a virtual coffee with them and talk about you know why you chose your career because that might be the information that helps them decide so if someone wants to get into this they should use that hashtag to try to meet people in the industry who will help bring them in my book at the end of each chapter there's questions like if you are going to secure a car operating system what would you do and so there's an answer key but it's very minimal and so starting march 20 i am going to do free online open discussions for three hours once a month and i'm going to invite a bunch of guests from the industry to discuss the questions and the public are invited to watch and interact with us via youtube so if you want an invite and it's all free because i'm really good at absecon not very good at business but basically go to alice and bob learn calm and then you know give me your email and i'm going to send you invites to all 11 sessions so we're going to finish around a week before christmas this year and we're going to go through all of them and discuss every single question if you read the book you'll understand better but if you don't you'll still learn a lot because basically i'm going to invite you know lots of people from class people from just different areas based on whatever chapter it is so we can talk about it and hopefully help spread the word of secure software
John Hubbard 41:54
fantastic well thank you very much so we've got a book we've got we have purple the website we've got the mentoring anything else you want to shout out before we close it up here
Tanya Janca 42:04
um my company's giving away a free mini course with app sec so there's always like one or two or three app sec people and then 1000s of devs and so you have to scale your team so i made like this little mini course that we're giving away for free about how to scale your security program without extra security dollars so if you go to newsletter.we hack purple.com like you join our mailing list and we're sending out invites and then next month we're going to run the free instant response course as well and it's how to respond to incidents that have to do with software because it's different and like how to prepare prevent incidents and yeah marketing teams with we just gave a lot away for free and like i thought we were already doing that
John Hubbard 42:51
awesome well very very cool thank you so much for all the insight on app sec and def sec ops hopefully we've inspired some folks to chase this path as a career sounds like it's going to be something that's going to be needed far into the future so thanks very much again for appearing on the podcast and we will see you around
43:10
thank you so much john
John Hubbard 43:11
alright catch you later hey blue teamers hope you enjoyed today's episode of blueprint if you've got a second and wants to help support the podcast please subscribe and leave us a review on apple podcasts it would be really really meaningful to us and if you have any ideas or suggestions i would love to hear them your views are going to be one of the best ways to help others find this podcast so anything you could do would be a big help as always thank you for listening you can connect to me on social at sec hub sec hq bb on twitter or on linkedin so until next time thank you for listening to the blueprint podcast
