ကူးယူချက် 🔗
David Burns: Testing is not just a phase in development process, it's an ongoing mindset. I wouldn't try to go ahead and automate everything, which is like boiling the ocean. I wouldn't want to go do that. It could be as simple as just set up my test, open the browser on the website and just click one button. If you're using Selenium versus Webdriver IO versus Playwrights, Cypress, Puppeteer, for the most part, Under the hood, they're all kind of doing the same thing and don't let anyone suggest otherwise. Just get started. I understand it's scary, but there a lot of resources on the internet. Happy testing everybody. Hi, everyone. Welcome to Browser Stack Talks. My name is David. I head up Developer Advocacy and Open Source Program Office here at Browser Stack. Today I'm joined by one of my other hosts, Pooja, who has fantastic colleagues, worked with me for years. But I'll let Pooja introduce herself. Hello, everyone. Thank you, for having me here today. My name is Pooja Jagani. I'm also developer advocate and a lead open source engineer and David's colleague. I contribute mainly to Selenium repo. And I think one of the beliefs that I have are like a strong, I'm a strong believer of the fact that testing is not just a phase in the development process. It's an ongoing mindset. And I think we're about to explore more on that here today. Is talking about test automation and where to get started. This is something that comes up to me regularly at conferences, people come up and it's like, David, where do I start? Like, I want to do this. I've seen your talks. I've seen other people's talks. Where do I get started? ⁓ And it just feels overwhelming. So hopefully by the end of this, you will have ⁓ some kind of tools and tricks to get it through. We're going to try to keep the specific frameworks out of the discussion and give you kind of the based knowledge that you can then go to any tool that kind of makes sense and start from there. Because it's going to be much easier. If I only teach you one thing, ⁓ it's that all frameworks are going to be equal and you'll kind of be able to do most things with most frameworks. So let's jump straight in. ⁓ Pooja, you've worked at large companies, you've worked at BrowserStack for a long time now. ⁓ and you've worked on this linear project, like all of these are sizable code bases. ⁓ If you are coming into these places, into these code bases, and you looked at all of this, what is your kind of first thoughts on where do I start with testing? Right, you know you need to do it, you need to get your team doing it, you've been a team lead for a while, how do you get people started? That's a good question, David. So when it comes to getting started, I think, the foremost question to answer would be what do I want to test first? So I need to identify what is a piece of code, what is a piece of module, what is a component that I want to first make sure I want to test. I also want to pretty much at that point, I want to
Load full transcript
make sure I know that component in an out, right? I know the inputs that will go in, what are the expected outputs. And then probably a good first step would be to begin with unit tests. Quickly, you can assemble it quickly, write one quickly. verify the smallest component and just slowly start building it out there. So I think that is my suggestion advice. Yeah, definitely. We need to test automation as a journey. a journey starts with its first step. It doesn't matter how long it's going to take, but you need to take that first step and not be put off by taking that first step. So you start with your first test and you start building out. ⁓ What would a first test kind of look like? You're a developer. I come from QA into engineering. You've been ⁓ a developer your whole career. Like where do you get people started? Like what is that first test generally for you? I think first test, like I said, would be a unit test and probably a very, very simple component, right? I wouldn't try to go ahead and like, ⁓ automate everything, you're like boiling the ocean. I wouldn't want to go do that. like, because simplest flow, for example, say you're used to manually testing your application, you open the application, you're clicking something, filling a form, doing the submit button. So pretty much I would suggest trying to mimic the same action, but using code and scripts instead. But again, I wouldn't overshoot and like, you know, try to have a moonshot of doing the whole flow in one go. It could be as simple as just set up my test, open the browser on the website and just click one button. That's it. So like probably build small and gain confidence. I think it's all about feeling confident in the code that is written and the fact that it's working and probably build it from there. highly recommend that you don't boil the ocean. Whenever you look at problems, just think of the problem, get it going and then move on. And as you do it, you get a game of confidence and you get lots of warm water. rather than trying to do it all at once and never succeeding. The other thing though that you kind of alluded to ⁓ is that we need to start at unit tests and obviously people have heard of unit tests, they've heard of integration tests and end-to-end tests. These are the key parts of the testing pyramid. ⁓ Could you elaborate on how much of what you tend to do when you look at each of those types of tests, because you've kind of said you like you start a unit test and you work through to getting a workflow. How many tests of each do you think is kind of the right answer, if there is a right answer? I don't think there's a right number. I don't think there's an arbitrary number or a magic number that does the trick for everyone. It really depends on the project situation, testing. test plan and testing strategy, et cetera. But like you rightly said, it's a pyramid, right? So at the base, the widest base is unit tests. Then comes integration and at the pinpoint narrowly part of the pyramid comes end-to-end tests. And that's also directly pretty much proportional to the frequency. So the broader the base, more number of unit tests. That means test your small competence a lot. Test them thoroughly, test them with confidence, test them pretty much early on. And that becomes like a solid foundation. and then take it to integration tests where there are two or multiple components talking to each other and kind of solidify that communication that's going on there. And then probably take it to end-to-end tests because as we kind of move up the pyramid, there are external factors that come in that sometimes are beyond our control, right? And that's what tends to make it each step. ⁓ It might get a little flaky or brittle in nature, like end-to-end tests might have network. You could have a queuing system, could have a DB you're talking to, there'll be a lot more of setup, there'll be a lot more of communication, there'll be a lot more of effort, also lot of time consumption to run the test, right? So I would suggest to go into the hierarchy the pyramid suggests. Yeah, definitely. And I think you're 100 % correct in that the flakiness does increase as you kind of head up the pyramid. ⁓ And it's because as you keep adding components, things could go wrong. I always joke that networks are more down than they are up. And kind of if you've done computer science 101, you're taught that like computers need to, when they're doing networking, they're constantly doing handshakes just to make sure everyone understood what was going back and forth between each other. And they're doing this lightning fast. And so if you keep adding things to do that, it suddenly just grows exponentially and flakiness does increase. You'll also have fun along the way with kind of how programming languages interact with kind of your tests. Because if you've got asynchronous code, knowing when and when that has finished is hard. ⁓ I've got talks coming out this year based on this, and there's a blog post on my personal blog. But ⁓ flakiness is an interesting space to be in. It's frustrating. And hopefully we'll have a... talk about this ⁓ like a podcast in the future, but we'll get some people in who've been working on this for years. Cool. So we've kind of looked at the basic things. We've started building up. You've kind of alluded that developers should be doing some, QA should be doing others. ⁓ If you were to kind of get the most out of a kind of the quality of a product. How do people go about trying to find the right framework that works for them? And so framework here is like something like Selenium or Playri, whatever, right? How do people go and pick and choose that without kind of falling into, this is the one everyone's talking about, right? Like picking something that's meaningful rather than something that looks shiny. And I think you rightly said that it's always... important to pick something that is meaningful and for it to be
meaningful, I think there are various ⁓ criteria depending on say your team expertise, some past experiences, might have a sister team who already knows how they've done it in the past. It can also depend on the developer skill set that the persona has, they could have some past experience, or they're probably trying to learn a new language and then learn testing and things and so on and so forth. each library or each test framework has something different to offer. Some of them might have a very intuitive language versus some might just be building blocks to a standardized protocol like W3C WebDriver protocol, which is fantastic because it's standardized, it's cross-browser, it's consistent and things like that. And then you have frameworks and libraries like Selenium WebDriver IO built on top of that and like Puppeteer is using ByDial and now it's helping with that. There are a lot of options out there, but I would suggest identify what one wants to test, really. Let's say you want to test the log and flow, and then you have a CAPTCHA at each step. That might not be supported by all libraries. You might need some add-on libraries, and then you might need to figure out what works well in the language of your choice or in your framework, what pairs well, and things like that. So foremost and first step should be identifying what to test, and then take it from a framework or a library that really, really meets and suits your needs. It's not about the buzzwords. It's not about popularity. There is, I don't think there is one tool that fits all. It's also okay to mix and match if you think that's what works the best for you. But I think it's important to pretty much not get sucked into that rabbit hole and just get started. Yeah, definitely. think it's important that people just use something that feels comfortable. The whole point of writing test automation is to have confidence in your code and confidence in your product, right? That you'd be able to ship it out and you can ship it out fast. And if you don't have that confidence, then it's not really useful. You want something that allows you to move fast as well. So that, like those are the criteria for me. Something that allows me to kind of parallelize my tests. And when you start parallelizing your tests, that's when you get into other interesting problems with testing. So ⁓ just as a FYI to everyone out there, make sure that each of your tests is as atomic as possible. So that means it does that one test and it does it well. Start from a known good place, do one action, and then kind of make sure that you can revert back to that known place. So then if any other tests are running at the same time, they don't interact with each other, they don't bleed over and stuff like that. And so hopefully we can get you doing that and then you'll have a much easier time when writing tests and because flakiness, another part of flakiness is the kind of when tests are not atomic and they start overlapping with each other. Yep, I think I've seen that a lot in terms of there'll be a test which works beautifully sequentially. So if you run them one by one or just a single test, all okay. But when you run the parallelly, there'll be some tests that break and that too intermittently. So it's pretty random. because then, like you said, they're gonna be bleeding over, they're not atomic in nature. And I think it's very important to plan it that way. Again, it saves a lot of time while debugging and things like that. Yeah, definitely. The other, like to the debugging points, I think that is... me the biggest one is that if I know there's a failure happening in this workflow, even at a unit test level, I call all of those workflows. I need to call an API and get a result. ⁓ But I want to be able to just instead of running 1,000 tests to get that one test, I just want to be able to go let me run this one test and then debug that one test and go from there. And so it The whole point is trying to be as productive as possible. ⁓ And when you're frustrated, I can guarantee you're not being productive at all. ⁓ I say this from experience, right? Like, every time I get frustrated to a point, like, actually, I need to walk away from my computer and then come back and think about it again. No, awesome. So it sounds like we've given people ⁓ high-level ideas of what they need to do. Like, we've said. Break down your tests, break down your to kind of a small thing. Break down your application to like, where is the highest risks? So if that area was going to cause more problems when it fails, that's where you should be doing more testing. And if it doesn't matter if that area fails, you don't necessarily need test automation. It's a nice to have and you can catch those later. But if you need to move fast, go for the higher risk areas. We've talked about frameworks at a high level. We're not recommending anything. They're all kind of equal to each other, despite what a lot of people will say. If you're using Selenium versus WebDriverIO versus Playwrights, Cypress, Puppeteer, for the most part, under the hood, they're all kind of doing the same thing. and don't let anyone suggest otherwise, right? Like the only one who's not using WebDriver these days is Playwright. Everyone's moving to WebDriver by-dial, which is the new standard that Pooja and I've been working on. So find the one that makes you comfortable and makes you productive. Everything else will just fall into place. And I think I have just one suggestion on top of that. I think it's always a good idea to keep in mind the frameworks that work with real browsers, because that's going to be something that's going to save a lot of time in the longer run in terms of bugs, and also just being confident in the application that the users are using. Yeah, 100 % agree on that. And I did a talk around that two years ago at Selenium Conference in Chicago, probably two years ago. I've lost track of time. It's two years ago, yes. But like a perfect example around what Pooja said, just to kind of reinforce what Pooja is saying, is that Matthias Byens, engineering manager at Google on the Google Chrome DevTools team, he's been instrumental in creating Chrome for testing because they were getting frustrated with people raising Chromium bugs but not being able to find the version of Chromium that kind of matches that bug. ⁓ going through that process, if he was finding his team were frustrated and all of that, where Chrome for testing does everything that you need. It's Chrome. It's got all the things. It doesn't auto-update. So if you need to be on a specific version, it's great. Selenium Manager, if you're a Selenium user, uses it natively. It will set up Chrome driver. It will set up Chrome for testing. And away you go. And you never have to worry about it. So yeah, to that point, like try use real browsers. There are differences between Brave, Edge, Vivaldi, ⁓ Google Chrome, Bright. They all might be using Chromium under the hood, but there are going to be differences. Same with WebKit and Safari. There's a famous index DB bug that came into Safari that was never in WebKit. So you can go look that up. It's all in my talk, but like, just be aware. ⁓ If you're okay with those risks, that's also fine, right? Like there is no right or wrong answer, but just ideally be where your users are. Because then when they see bugs, you'll see the same bugs and you'll be able to reproduce them. Awesome. So I think that's been kind of a lot of information that we've handed over to people. Hopefully we've given you a quick kind of action plan on what to do, how to write your tests. We've talked about testing pyramid, things like that. Who should be doing it? Just to reiterate, everyone should be doing it. And go from there. So hopefully you find this useful. Please. make sure that you kind of follow us so that you can watch future episodes. Yeah, Pooja, is there anything else you want to add before we wrap up? I think, yeah, I just want to leave everyone with the thought that just get started. I understand it's scary, but there a lot of resources on the internet. It's okay to pick something, learn something, and then pick something else, and that's fine. But consider this as a journey, and it's an ongoing one. It's not a... journey where you build your test suite and then fire and forget. It will evolve as the application evolves. ⁓ I think it's very important to have fun along the way while learning, even when things break, and you sort it out and get that sense of satisfaction. So ⁓ happy testing, everybody. Yes, definitely. And from me, happy testing, and hopefully see you in our next episode. you