Brian:
00:00
The mock library that comes with Python as unit test dot mock started with Michael Ford. In this episode, Michael and I talk about mocking, of course, but also testing philosophy, unit testing, and what a unit is, and TDD, and even where Michael's towel is, and what color. Michael was instrumental in the building of testing tools for Python, and he's got a lot of great advice about testing. I hope you enjoy the episode.
Brian:
00:39
Welcome to Test and Code, Michael Ford, did I get the last name right?
Michael:
00:51
You got it right.
Michael:
00:51
Yeah. Yeah. It's the English version or the British version.
Brian:
00:55
So 4 with 2 o's.
Michael:
00:57
Yeah. That's right. Yeah. Double o. I'm a double o.
Brian:
01:00
So what why do is it just Americans that have trouble with that?
Michael:
01:06
The thing is the single o is so much more common. People sort of assume it's Dutch for some reason. It's not it's just English. We we back in the days when we used to have telephone books, there was always a a few in every telephone book. So, you know, not common, not rare.
Brian:
01:19
Whenever whenever I see your last name, I don't know if it's spelled the same, but I think of, Ford Prefect. But did he spell it with 1 o or 2?
Michael:
01:27
He he he spelled it with 1 o's. Then there's also there's also the Hoopy Frood in, the work of Douglas Adams. And I I I love that. I've always felt sort of an affinity with those works, both because of Ford Prefect and because I am a hoopy frute who knows where his towel is.
Brian:
01:41
Definitely. So do you have your towel with you?
Michael:
01:44
I don't have it with me, but I know where it is.
Brian:
01:46
You you know where it is. Yeah. It's
Michael:
01:48
it's purple and it's upstairs. It's one of these microfiber ones, So you can pack it really small for traveling. It's great. Perfect.
Brian:
01:57
So one of the things okay. So we wanna talk about mocking eventually, but what are you doing now? You do you have, like, a consulting company or something, right?
Michael:
02:07
Yeah. Contracting and training.
Brian:
02:09
Training. Alright.
Michael:
02:10
I I started training Python, doing Python training about 10 years ago now with David Beasley teaching his practical Python and advanced Python master mastery courses alongside my regular work. You know, I just do it a couple of times a year, and I really love doing it and enjoying it, and it's a good trade to be in. And then, I worked for Red Hat. My recent career history is that my last job was with Red Hat on Ansible Tower, their enterprise web application for managing Ansible and for managing computer infrastructure with Ansible, and I I worked on the test automation team there, helping them build out a test system. I worked for them for a year, and and I felt I really wanted to go self employed.
Michael:
02:58
It was time to to do that. And, so I've done half contracting and half training, and I really enjoy that. I still teach David Beasley's courses. I've also taught some Flask and some some testing and, object oriented theory for the Royal Ordnance Survey. Object oriented theory with Python, they wanted they didn't just want a Python course, they wanted something a bit more fancier, because they're the Royal Ordinance Survey.
Michael:
03:24
So So I did object oriented theory with Python. That was great fun. Contracting, my most interesting project has been for the United Kingdom Atomic Energy Authority, and that was from January to June last year, and that was working on software for designing fusion reactors.
Brian:
03:41
Oh, wow. So normally,
Michael:
03:43
that it's like it's like a dream job. I would just say he was actually a PhD guy, and it was his PhD project, so he but so he'd already committed to make it open source before he sort of accepted the job with the with the UKAA, so they are gonna make it all open source. It's blueprinted, and I worked with him on turning it into an engineering project. He had a and at one point, I mean, he was a phenomenally clever guy. I mean, I was literally actually removing descriptors off meta class, meta class no.
Michael:
04:11
Removing meta classes off descriptors. Oh, cool. Which, you know, just does not need to exist, you know. But but still, you have to be really clever to make as much of a mess as that. That was great fun.
Michael:
04:25
Yeah. And it so the normal way that the fusion reactor design is done, the academic world, they're all silo. Everyone has their field of expertise, but there's a whole series of things that need to be done. You know, they want to work out the magnetic, containment, the plasma containment chamber, so they've got place. They've got overall size power requirements, positioning of the magnets.
Michael:
04:47
The really cool thing they're doing now is breeder blankets. Basically, fusion reactors, you put hydrogen in, you ignite it with a laser, enough heat and pressure to trigger fusion and then using magnets you shape and contain the plasma, usually in a torus, and then this produces a stream of neutrons which hit big metal absorber blocks around the side of the lining the, the, the reactor, turns it into heat and from there it's normal turbine technology, which is very well understood. What they've done is they've found that if they put lithium into these blankets, that the neutron stream bombarding the lithium every now and then will hit the lithium, split into 2 tritium atoms, split into tritium, which is the fuel for the reactor. So you need enough tritium, which is very expensive to produce in radioactive, pretty dangerous, an isotope of hydrogen, and once you've got ignition, it's then it's then self feeding in terms of, of the Trojan. So that's a brilliant innovation.
Michael:
05:53
Anyway, you used to design these things by sort of getting your specs, sending them to the first guy to do the first bit of design. He'd take a month, send you back a bunch of numbers, which you send to the next guy. So the whole process took months. So you can't really do an iterative process of trying a bunch of things that way. Yeah.
Michael:
06:10
And this this guy, his genius is, an interdisciplinary approach, and he essentially wrote one application that does all of these different and it's for, it's for initial design work. It it's not the detailed design, but but you can now do an initial design, you know, with fancy OpenGL drawings, which are beautiful, and everyone thinks it's the whole point, but, of course, it's not. It's all the numbers it spits out. You can do that in 40 minutes. So you can do iterative design processes.
Michael:
06:37
So it's it's it is and is going to revolutionize fusion reactor design. It was a fantastic opportunity to to work on it and so much fun.
Brian:
06:46
That's so cool. Did you say that was open source stuff?
Michael:
06:50
Well, I I emailed him recently to ask him if it was open source yet, and, he didn't reply to that bit. They were they were due to do it, before now, so I think the answer is no. Oh, okay. It's called Blueprint. There are some, there is a paper he's released on it.
Michael:
07:06
So yeah.
Brian:
07:08
Okay. No.
Michael:
07:08
I'm just gonna drop a link if it was there. Yeah. I'll be all over Twitter with it when when it comes out because it's, you know, it's a it's a fan it's fantastic to play with. Yeah. You know, you you can design fusion reactors in your living room.
Brian:
07:24
Thank you, PyCharm, for sponsoring this episode. I first tried PyCharm when they started supporting Pytest many years ago. Their support for Pytest is now amazing. I was a long time Vim user, so next I needed to test the idea Vim plugin so all of my finger muscle memory still worked while editing. Check.
Brian:
07:41
It works great. There's lots of reasons to love PyCharm. But for me, it is because they have the absolute best user interface for test automation. Then I learned many more ways PyCharm can save me time, like really great support for editing markdown, HTML, CSS, JavaScript, remote connections to database, and amazing version control support. Really, it's the best git diff tool I've ever used.
Brian:
08:05
And now version 2020.3 is out, and the shift shift, the find anything key sequence, even lets you search git commit messages. What even? That is so awesome. Tons of other cool features have been added in 2020.3. Check it out, and I hope you enjoy it at testencode.com/ pycharm.
Brian:
08:28
I know you've got a site, agileabstractions.com. Is that the That's
Michael:
08:33
yeah. That's my, professional site, but I haven't added any of the the projects I've worked on since 2019 and 2020. I don't think so. It's a little out of date, but that's my professional site.
Brian:
08:43
Yeah. But if somebody wanted to hit you up for training or something, they could go there. Right?
Michael:
08:47
Yeah. Oh, michael@python.org.
Brian:
08:49
Okay. A python.org email. Yeah.
Michael:
08:56
Instant credibility. I I got it because I I helped out a lot on the web. I was one of the webmasters on various mailing lists administration. And, and yes. And I asked for Michael, and nobody else had taken it.
Michael:
09:07
But it is a beautiful email address. I'm very proud of it.
Brian:
09:09
Yeah. So there was there's this, there's a Python feed thing. I can't remember where it is. Python do you remember what that was?
Michael:
09:17
Python? Is that what you mean?
Brian:
09:19
Say that again?
Michael:
09:20
Python feed. Do you mean planet Python?
Brian:
09:22
Yeah. Planet Python. I I just remember that because I think that's the first time I ran across your name because, planet when I started blogging about Python, I heard somebody say, well, you gotta get you gotta get your, your blog on planet Python to get listed or people won't pay attention to it. And then so I requested it and I think you replied and said, okay, it's there. So
Michael:
09:46
It was one of the things I was looking after. That was back in what I think of as the golden days of the Python community, back when Python was about to explode with the web revolution and Google adopting Python. And and, prior to that, Python had mostly been used only by enthusiasts, only by people who really loved the language, which made a beautiful community full of passionate people really eager to to teach you, you know, and then Python just exploded. And so back in back in the day, having your blog on planet Python, you know, you you could get a, you know, a 1000 views for a blog entry. The glory days.
Brian:
10:24
Yeah. Now the other then I ran across your name next when I started researching, testing stuff. And you had a bunch of testing articles. But then also, your name is attached to the, the mock library in unit test.
Michael:
10:42
That's right. That's right. Yeah. I I originally wrote what's now unit test dot mock and maintained that as a library for for quite a while. And that that came out of my first program and gig for Resolver Systems in London.
Michael:
10:55
I started with them back in 2006. And they were doing all extreme programming, so it was all pair programming, fully test driven development, customer representative doing prioritization, estimations, tracking, velocity, all of this this kind of stuff. We did that rigorously for 4 years, and that was an amazing experience. And it yeah. It came out of that time, and in particular, I got a test a passion for testing as a way of ensuring product quality in, in programming.
Michael:
11:26
Whereas before the sort of style of programming that I was used to do, knowing that things worked was a real challenge. You had to try everything. And we can automate so much of that. Yeah. So so yeah.
Michael:
11:39
That's that. I I I was passionate about Python, and I became passionate about testing in that time.
Brian:
11:45
And so since I guess since then, you, did you still incorporate testing within all your development processes then?
Michael:
11:55
Yeah. I mean, we did test driven development rigorously. We did extreme programming rigorously for 4 years. So that was the first thing we do is write a functional test which exercises end to end the the feature that we're trying to add or demonstrates the bug. And then we start writing unit tests and those are layered.
Michael:
12:13
And we had a test to code ratio of at least 3 to 1. And we, you know, we over tested in all sorts of places. We had testing couples to the implementation. So I learned a great deal about testing and the problems, if you over test, you couple your unit tests, you're testing the implementation, so you're testing your private methods. And then when you come to refactor, and you change you want to change your code, change the way you call your code, your tests tell you nothing useful and they're broken because even when you fixed your code again, the test is not going to show you the right thing, they're testing old code.
Michael:
12:48
Functional tests, your end to end tests, if you've got good reasonable end to end coverage, even just a set of smoke tests, then you can refactor and you can be reasonably confident that sort of you haven't broken functionality because with refactoring, your end to end tests don't change. So I think there's a lot more value in end to end testing. I think there are dangers in over testing. I like to say unit testing is about testing to the unit of behavior, not the unit of implementation. Test through the public API.
Michael:
13:15
If you can't test through the public API, then your abstraction isn't right, you know, typically. These sorts of things help you to avoid over testing. Scripting tasks, I won't test. Adding tests to big legacy projects is also very challenging, particularly in the face of ongoing feature work, and that's something you have to sort of work to incorporate gradually and pragmatically because businesses, you know, you only get paid as a programmer if the business keeps going.
Brian:
13:44
Yeah.
Michael:
13:45
So you have to, you you have to incorporate, building in tests and testing to give you keep sanity with, the, the the the work of maintaining and extending the legacy project.
Brian:
13:59
I really like that, I'm gonna steal that unit of behavior, not unit of implementation. I like that a lot.
Michael:
14:05
Yeah. Because that's what you wanna test. Right? You wanna test it does the right thing. You don't wanna test how it does it because in theory, at least, it would be very much nicer if you could change the how, and your tests still tell you useful things.
Michael:
14:18
And if you're only testing through the public API, so long as you've got the public API right, you know, often you need to change that, then, you know, you you're free to change the internals and your tests still tell you useful things.
Brian:
14:31
Yeah. Now, and then, of course, as you know, there's always there's always cases where, you really wanna beat up some algorithm bit in the middle. And so there is gonna be, some real tests around that.
Michael:
14:45
There there there are there are bugs where you can't replicate it without, you know, it's some timing issue or some file system quirk. And the easiest way of replicating is sticking some invalid data into some some internal state, because you know what triggers the bug. And you you know, that that's gonna happen. Yeah. You know?
Michael:
15:02
I mean, the all generalizations are wrong, including this one.
Brian:
15:08
Yeah. Yeah. Okay. I'm definitely gonna have to get this one transcripted because we got a lot of gems in here. The okay.
Brian:
15:15
So mocking, you got into mock mocking and and stuff is a is is a natural part of I guess, it depends on the year you were doing it, extreme programming and, and test driven development. But, but mocking is a little bit misunderstood by a lot of people, including probably myself.
Michael:
15:36
And probably me.
Brian:
15:40
So and and partly, it's partly your fault.
Michael:
15:45
Uh-huh. Uh-huh. Alex Gaybert blames me.
Brian:
15:49
Well, there's I mean, there's a well, if people start researching it, they get into things like, well, mocks the article by I'm gonna get it wrong, called mocks art stubs. Can't remember.
Michael:
16:01
By Martin Fowler. Yes. Martin Fowler. The static typing of that he defines various categories and types of mock, ways you can use mock, and he defines them as different objects. And it's the defining them as they're fine as categories, but they're fairly rigid definitions.
Michael:
16:15
And I think by his definition, mock is all of his types of mock except a mock.
Brian:
16:23
Okay. Yeah. I was curious about that.
Michael:
16:24
So I object to his definitions.
Brian:
16:27
Okay. So in Python, we use we use the unit test mock library or often wrappers around it. So, like, pytest mock has a is a is a wrapper around the mock object thing so that you can use it's basically context managers built on around it, which is kinda cool. Also really cool that you can use them as context managers anyway right off the bat. I didn't know that at first.
Brian:
16:53
But, yeah. Some
Michael:
16:54
That's that's an interesting point. And that was an innovation in mock that actually that came it was from a guy at Resolver Systems. It wasn't actually me, although I I get the credit for it. It was a guy called Tom. I forget his surname.
Michael:
17:07
Oh, dear me, poor lad. Good guy. Who discovered that so patch is a decorator patch, monkey patch is things, so you can inject or mock into, any, you know, all sorts of places. And, the reason Alex Gaynor blames me, and I think possibly what you're about to say, I might steal your thunder, is I mean, patch makes it possible to test code that was essentially really hard to test before. So the combination of mock and patch make it hard make it possible to test code that's really hard to test, which doesn't give you an incentive to write code that's easy to test.
Michael:
17:43
And code that's easy to test is generally better code. So patch and mock let you disguise the fact that you're writing terrible code, and that is definitely true. But the innovative thing is that patch patch which puts mocks into place, the the thing that it does that's really powerful is it undoes it. It puts things back the way they were before. And monkey patching things is easy, un monkey patching them is hard, and there's a lot of logic in in in patch that knows how to do that.
Michael:
18:12
So you you can you can use it as a context manager with patch, time dot time as mock time.
Brian:
18:20
Yeah. And
Michael:
18:21
that and then anything in for anything inside the context manager sees time.time as your mock object, which you can then configure the return values. But then outside the context manager, time.time is restored to what it was normally. So the scope of the effects of the patch is limited by the context manager, which is what context managers are great for, you know, a visible scope of effect. And you can also use it as a decorator where during the function for the inside of the function that's decorated, the patch is in place there. This was actually done in Python 2.4 originally back before we had context managers.
Michael:
18:57
So being and it was Tom who at Resolver Systems who realized that the way that we do decorators is entirely compatible with context managers. So Patch can do both, and Patch is an interesting beast. And there's actually in contextlib now, there's context decorator, where you can write context managers that also work as decorators, and that came out of, what first happened in mocking at Resolver systems.
Brian:
19:24
That's true.
Michael:
19:25
Python 2.5. Bit of Python bit of Python archaeology.
Brian:
19:32
Yeah. So if if people have kinda missed it so far, how how do you describe a mock, like, to somebody that doesn't even know what it is?
Michael:
19:42
Okay. Right. So this is so a mock object is an object that can pretend to be any other object, essentially. Now the thing that Martin Fowler, defined about mocks and what was common at the time, so he talks about mocks and stubs and I forget the the the the another one. But the the essential point about a mock is that it record you can record your expectations and then replay those.
Michael:
20:09
So you create a mock object, you configure the mock object and say, how I think this is the Martin Fowler one and the Java mocking libraries and, the Python mocking libraries at the time. You create a mock object, you say, I want you to have these methods, and I expect this method to be called. It should return this. Then I expect this method to be called. And then you do do you run your code and you hit replay, and it throws an exception if it was used in the wrong way.
Brian:
20:35
Oh, okay.
Michael:
20:35
So that's the record replay style of mocking. And for me, that puts the you you record your expectations on the mock and then you call your code and that's us about face. You know, what I want to do is I want to inject a mock into the system, run some code, and then I want to be able to make assertions that it was used in the right way. Because not all of the things that happened to it might be relevant. It might just be one particular thing.
Michael:
21:02
I want to assert that you were called with this argument. I want to assert the mock might even just be going in there just for the purposes of returning a pre canned value, stubbing out a a system function that you don't want called in a unit test. That's a very good use of mocking. Mocking external dependencies to return deterministic results for the purposes of testing to avoid external calls in your unit tests. That's the classic and a great use of mocking.
Michael:
21:25
So you might you might mock out API calls, mock out network calls, mock out file calls. And there's some some support in the mock library for particularly for files.
Brian:
21:36
Yeah. Yeah. Like, for instance, I I think of an example which probably isn't very common. But, if my test if I've got a system, I don't know, logging system or something, and if I find something critical, it's gonna email a bunch of people.
Michael:
21:49
Right.
Brian:
21:50
During the test, I don't wanna actually email everybody. But I can make sure that the the appropriate call to email the right people is called during the test.
Michael:
22:01
With the right premises. Yeah. Exactly. That's the sort of thing that mocking's for. And just to finish off the thought previously, so instead of being record replay style, mock is, AAA.
Michael:
22:12
What is it? Arrange, act, assert. You set up your code, you call act, which is arrange, then you call your code, which is act, and then you make the asserts. So, unit test dot mock, what it does is you can configure it so that methods will return values, or a method call call will have a side effect, like raising an exception or calling another function, or you can give a sequence of values for multiple calls. All sorts of ways you can configure mock objects that can pretend to be any objects, They can pretend to be a dictionary.
Michael:
22:40
You can configure any of the magic methods, all of this kind of stuff. You put it in place, and then it record all of the calls are recorded on it. There's a and and there's some convenient there's some convenient assert methods and convenient call methods. So, the reason I created it was for for for two reasons. The first thing was that, we were doing full test driven development, and we were creating all of these, little stubs objects in scattered throughout our code base with like a mock workbook, a mock worksheet, a mock cell, a mock row, a mock column, you know, and overall it added up to sort of 100, maybe thousands of lines of code.
Michael:
23:16
And initially I was like, I can replace all of this with, a Python object with a done to getattr that responds to every attribute lookup. So the initial implementation was, you know, about 30 lines of code, and it was to replace all of these mock objects in the resolver code base. But a requirement from Giles Thomas, who was the CTO at the at the time, was that we had to have a way of limiting the API. So if we had accessed an attribute that shouldn't exist, it would still raise an attribute error. So that's where all of the spec stuff in mock came from.
Michael:
23:48
So that was where it originally came from. It was motivated also by the desire of not wanting none of the existing mock frameworks, all of them in Python, they were all this record, replace style, which I didn't like. And there, the testing in Python community in Python was particularly close knit and fun, and we got the testing in Python boff at Pycons, which had a great run for a good number of years. It was a lovely community to be part of, and mock really evolved very rapidly, with users in the testing in Python mailing list and feedback from them, and competition with other people writing, libraries. I was I was full of passion and activity in those days, you know.
Michael:
24:24
If any mock library came out with with a feature I thought was good, I'd have a new version out with with that feature, you know, in a in a few days, you know. I answered every email about mocks on the testing in Python, email list, showing them how to do it with my library. You know, I blipped the competition.
Brian:
24:43
Well, it's interesting. The testing in Python
Michael:
24:45
share enthusiasm.
Brian:
24:46
So the the testing in Python mailing list still exists, has very very little traffic on it. So a lot of people think maybe it's dead, but it, we still get I mean, I still pay attention to it and replies go real you get really good replies. If you ask a question there, it's pretty good. Now I probably regret putting this in a podcast because suddenly people will start using it.
Michael:
25:09
Start using the testing in Python mailing list, folks. Right.
Brian:
25:14
But, yeah, you've got we've got some really great smart people paying attention to it. So, but the okay. So what is your relationship if the I'm thinking of this as somebody coming in and going, maybe I should use mocks in my testing. What do we tell people that haven't used this before? And, because there's there's this like you described, this test driven development model, which is okay.
Brian:
25:38
We have to just there's 2 there's 2 huge there's classical and mocas. But obviously, we're talking about mocas TDD, which means we try to test every function in isolation with everything around it.
Michael:
25:51
Right. Right. Right. The the trouble then is you're not testing the wiring between your parts. The the great advantage of, the great advantage of doing that is that your tests run nice and fast.
Michael:
26:03
But if you have you can have full coverage at the individual function and method level and still have no idea if your system works because this sort of wiring between your components is not tested at all. And the same effort expended at just at the functional test level would give you confidence that the application actually works. The advantage of test driven development and the reason that, it's called test driven development the other way of putting it is I often talk about test first. But the idea is that, that the test drive the design. And this is what I love about test driven development, and this is what I take from it even if I'm not always religious about test first these days.
Michael:
26:40
And it's that step of thinking about the design. It's that if you do test first, the first step has to be not what's the solution, let me bang out some code. It's how do I call this? How ought this to work? What's the best, the right API for me to be testing?
Michael:
26:57
And so there's that, it bakes in right at the start, the first bit of thinking is how should this look like? Because the default otherwise is you bang out some code, you think it works, I'll add a test and what you're testing is whatever you happen to come up with, not the best way of doing it. And the other aspects of TDD, the simplest thing that could possibly work building up tests incrementally, is that by evolving a design like this, as long as you pay off the technical debt of doing the refactoring, incorporate the cost of refactoring into your estimates, that you actually come up with better solutions by basing your incremental approach on actual usage? So those are lovely reasons to do test first. And as you say, if you're doing test first, if you're trying to maintain some level of isolation, you're gonna need some mocks.
Michael:
27:44
But I think that's the question then is the whole the question then becomes, what is my testing philosophy? And and the specific question we're asking is, how do I get started with mocks? And so I think we can answer that much more simply, but we we can say, look, the the two things in unit test dot mock library are the patch decorator, patch context manager, and the mock classes, the mock objects. And, actually, most of the time, probably, patch is gonna create your mocks for you. So first, you need to use patch.
Michael:
28:12
And this this often confuses people, so we can talk a little about that about it if you want. But basically, you say with patch and then the location of the object. I'm gonna patch out a method on a class. It's module name dot class dot method as mock object. And then inside the context manager, you can configure the value of the mock object.
Michael:
28:31
We probably want to say, mockmethod.returnvalue=3. And then after we've executed our code, we simply say mock object dot assert called with and assert it was called with the right parameters. So the very basics of using patch to inject a mock and making the asserts are quite straightforward. But where should we do this? Well, an obvious place is replace file access with, a mock object.
Michael:
28:58
You can if you patch by, open, the built in open, then you know it was called if you've got your results, and you'll save time in your unit test. Patch out calls to time dot time with the deterministic with something that's gonna return you something deterministic. That that, system calls, network calls, database queries, anything where you wanna return pre canned deterministic results, and you can avoid a real life network call. If you're not testing your database access, if you find if you're happy about the the the database access, you that's covered maybe the integration test level. Mock them out in the at the unit test level.
Michael:
29:34
Make your test faster and less dependent on your underlying model. This this kind of stuff is the place where mocking can give you a win.
Brian:
29:41
Yeah. And these these external parts of your system, well, it's really the the system under test. And I think, one of the things people don't talk about a lot is the the test architecture often mimics the people architecture. So we Interesting. Well, I mean, like, let's say I'm working on a I'm working with a database, but I've got a database layer that, some other team is working on.
Brian:
30:07
I think it would be and and I'm I'm not responsible for the user interface. I'm responsible for, like, this middle layer of stuff. Mhmm. Mhmm. It's completely reasonable to then, I think that there needs to be system level tests.
Brian:
30:21
But as a team, I'm gonna I'm gonna, like, probably feed my API and stub out my dependencies or mock my dependencies.
Michael:
30:31
The the there's, the principle is sometimes expressed as don't test the browser, which really only applies to web application development. But you test the code you own, not the cut the code you don't own unless you have to.
Brian:
30:43
Right. And then other the other really depend doesn't matter what style of testing you're doing, whether you're doing, a lot of test driven development or unit tiny unit tests or even functional tests. People are going to eventually need to mock out their external stuff like API calls to external services.
Michael:
31:04
Okay.
Brian:
31:06
You're not gonna like, oh, an example that, I'm blanking on his name, but, Harry Percival brought up is, the credit card processing. You're definitely not gonna hit the credit card API unless there's a debug one. But, why not just mock that out or stub it out? So the other thing that that Harry brought up, which I thought was a cool idea, is any real third party system. Not not some other team's system in within your own company, but like a third party, like a credit card processing or something, that you really wanna try to, replace their API with wrap it in in your own object or own module so that you Mhmm.
Brian:
31:53
Have a limited set of API functions or entry points into that service. And then Yep. Then that's a natural place to mock or stub out those functions is to to hit those. So
Michael:
32:05
Yep. Yep. And and if your API system is making network calls, then you can test this this abstraction layer by mocking out the net the network calls, calls to request or, you know, whatever, or the the the calls to their client. And, so you can be sure that's worked, that's doing the right thing. And then you've got, a nice abstraction layer where you can put your mocks and completely replace that bit of the system.
Michael:
32:34
So, yeah, you can test the 2 layers separately and be confident that it does work end to end. I know that the system under test calls the, the the the, the API correct the the API correctly that we've provided, the abstraction layer that we've provided, and I know that that makes the correct API calls because we've tested that as well. So yeah. Yeah. I like I like that approach.
Brian:
32:55
The other thing that I think is neat, and I wonder if this is one of the reasons why you get blamed for bad designs, is that I don't really have to understand dependency injection to use mock.
Michael:
33:11
Yeah. That was another motivation. In Python, dependency injection typically boils down to adding extra parameters to your function signatures. And I think function signatures are part of your API, and messing with those as a way of managing dependencies is not necessarily ideal. I might have softened on that.
Michael:
33:31
I I I kind of think, I I sometimes say this, and I think I think it's a a useful thing to say, is that every time you use patch, it's it's an admission of failure. You know, mocks mocks ought to be your last resort. It ought to be possible to test your system and the parts of your system. It ought to be testable. And if you have to replace a bit of it inside the live system in order to test it, then what you're saying is I couldn't design the system in such a way that I didn't need to do this.
Michael:
34:03
So working to minimize your use of mock and patch means that you're gonna get the best value out of it, I think. Only the situations where mocking is really clearly the best approach rather than making it the tool that you turn to first. Because there's, yeah, there's a definite there's a I mean, I've written code that whether it's, first it does this function call, then it does that function call, then it does another function call, then it returns the results. And I've mapped out all my dependencies. I've mapped out this function call and the other function call and that function call.
Michael:
34:32
And now I'm testing my mock objects and not my code.
Brian:
34:35
Right.
Michael:
34:35
You know, and that's crazy. You know, there I'm that's you've really tightly coupled your test to your implementation, and you're testing completely testing your implementation details. That's not a useful test, you know, really.
Brian:
34:48
So let let's, take that a bit further. So let's say I have, in order to test my credit card processing part of my system, I I I have I've been using mocks or something with that. What's the alternative? How would I
Michael:
35:03
I I I think having cleanly defined layers helps so you have a single point. I mean, the other thing I might do is do this via config files, you know, have an or have another mechanism that loads, a mock part of the machinery in place when I'm running under a dev config. You know, it's like, I want to be able to guarantee that when I'm running my tests, I can I can never send a, a credit card request? So I don't want to inject live things into the live system to to make sure it doesn't. I want the system I stand up to not be capable of doing it.
Michael:
35:42
Okay. So particularly for credit card or anything that's privacy or security critical, I would I would think about having machinery that does that for me. And it might put a mock API in place, and maybe that would use mock objects. But, you know, you'd have a shim layer. You you'd have some shim machinery.
Brian:
36:03
Well, that kinda reminds me of, like, the database stuff. So, one of the things that people often do in testing is to replace the live database or a file based database with an in memory database.
Michael:
36:16
And Right. It's ex exactly. Very yes. Yes.
Brian:
36:19
And but there but a lot of the databases like post and, others have a a memory feature. So you can just use the live database and just have it be in memory if you want, for instance. Mhmm. So
Michael:
36:33
Yeah. I'm yeah.
Brian:
36:36
Any, any warning signs to give people that I I like the idea of, like, look at patch first and and possibly look at, like, your external system, like, like, you gave a list, which is good, network, system calls, things like that. And then the easiest isolating those, like, I guess,
Michael:
36:59
I Yeah. Yeah.
Brian:
37:01
Making sure that those aren't all over your system. I wouldn't put, like, request, calls in every file of your
Michael:
37:08
your Yeah. Yeah. Have have it done in in a layer, which is much easier to step out and test and and have your calls go through that. And here your sort of testing strategy starts to influence your design, and I think in a good way. It's like putting the side effects into a separate function, so as much as possible, your functions are pure functions, which are then really easy to test and it's really easy then to stub out, mock out the bits of your code with the side effects.
Michael:
37:39
It's the same concept. If the file writing happens in its own function, then the rest of the function is much easier to test. When you're This this this kind of thing. Yeah.
Brian:
37:52
When you're teaching people, I I see on your training site that you do teach, testing training course. How much of how much of that is, around mocking or is that just a small part of what you're teaching people?
Michael:
38:05
It it it's a part. It depends a little bit on the the customer and what their priorities are. I maintained unit test in the standard library for quite a while, that was my other involvement with testing. I added test discovery, I helped break it up into a package. I think Benjamin Peterson actually did the work on that one, but it was controversial.
Michael:
38:24
So I committed it with under my name as the maintainer, and it happened. But even so, I still recommend pytest for new projects, and I use pytest for new projects. And it's, you know, I will teach people to use unit tests, and there's often a section in courses I I teach in general on Python. But generally but when I'm starting a new project, it's pytest I turn to, and it's pytest I teach.
Brian:
38:48
So do you use the with, if you're using Pytest, do you use the Pytest mock plugin or just use unit test mock directly?
Michael:
38:57
I tend to use unit test mock directly. That's possible. Usually because I teach mock as a separate section. Okay. Because, you know, just because, you know, you can spend a day or 2 days on mocking depending on how much of the API you wanna learn to use and how many different scenarios you wanna cover and, you know, and I'm quite fond of unit test dot mock, but I'm not sure I'm not sure that's necessarily best practice with pytest.
Brian:
39:23
Okay. The, so I when I when I'm reaching for it, I usually use the the the Pytest mock plugin. But it has, like, this marker object that you can pull in and stuff like that. But, anyway,
Michael:
39:37
the It's a it's a it's a fixture. Right?
Brian:
39:40
Yeah. It's a fixture that
Michael:
39:41
My my my one problem with Pytest is how easy it is to get into fixture hell. And I I even worked at one company where they they had a culture or convention of providing stuff via fixtures. So you would use fixtures instead of imports. And so you'd look in your code, and, like, your ID no longer has a clue where things come from or where they are. And they've got fixtures taking fixtures, returning fixtures, and you're trying to work out where something happens.
Michael:
40:09
And you're following this graph of fixtures, then you're up to the 8th level. And you're like, why? Why? You know, my life has descended into hell. Okay.
Michael:
40:20
So so, fixtures are fantastic, for, you know, limiting the scope of stuff, and I love the scoping, you know, but it's like you use them sparingly. You know, there's like imports are great. Use import ports, not fixtures.
Brian:
40:35
Well, okay. I'll disagree with you on that.
Michael:
40:39
But it I fixtures fixtures are great as well. I've just been in fixture hell.
Brian:
40:43
But also just a reminder that there is no framework or strategy that can prevent you from writing really crappy code.
Michael:
40:52
Oh, yeah. I know. Yeah. You're right. Right.
Michael:
40:55
Maybe maybe Scala or Haskell.
Brian:
40:58
Well, okay. So one of the things that that you hinted at, I just wanna, like, inject this in here, dependency inject this into the conversation, The, with pytest, you can put fixtures either in your test file or in a conf test file, and you can have one conf test file for every directory in your test structure.
Michael:
41:17
Oh, that was the the other thing, wasn't it? It's like this fixture I'm using it, where the hell does it come from?
Brian:
41:23
Yeah. You can have And
Michael:
41:24
then you're scattered across your code base like
Brian:
41:27
And they can and your fixtures can depend on any other fixture that is anywhere in its parent to hierarchy. But I recommend people in a project to have 1 conf test file at the top. You can
Michael:
41:39
I mean, I I love that feature? It's like, if I put a conf dot py at the root of the project, pytest knows what the root of my project is now and, you know, a whole bunch of other things just work.
Brian:
41:49
Yeah.
Michael:
41:50
So it one is one of the nice things about Pytest. Yeah.
Brian:
41:52
So that also and that also helps people if they they they know if my fixture isn't in the file I'm looking at, it's over here. Just helps your whole project.
Michael:
42:03
Yeah. Yeah. Pytest is really flexible, you know.
Brian:
42:07
But they the when I started looking at all these things, I did look at unit test and, I have used unit tests. The I actually was annoyed with a lot of the people's complaints of unit tests. Is the the thing that people say is it's too much boilerplate. And with test discovery added to unit tests, there's not you don't have to do the, like, name equals main thing in your file. There's not a lot of boilerplate, I don't think.
Brian:
42:37
But
Michael:
42:38
So yeah. I mean, it used to be if you used, unit test, you had to write test collection yourself and test running, and there's a test result, there's the test collectors, there's a whole load of machinery. And really, test discovery was the minimum to kind of bring it up to the sort of usable level for for for projects. A lot of the criticism I have come from the, it's a port of like JUnit version 5 or something and JUnit evolved a huge amount and unit test didn't. So we have the non PEP 8 style naming And pytest has function tests as functions, whereas with unit tests, you inherit from test case
Brian:
43:20
Yeah.
Michael:
43:21
And, and you write test methods, and you use the assert methods instead of the assert statement. And honestly, that's the boilerplate people are talking about generally. And it's like, yeah, I I like writing test functions in for pytest, but I also like grouping tests together in classes. You know? So I don't mind grouping tests together in classes.
Michael:
43:42
I don't mind the assert methods. I was suspicious for years of the assert, the pytestassert magic rewriting because the code if you get a you know, the code that runs the test run, they've rewritten the the bytecode magically so they can tell the, the intermediate values at all the points. They can tell you the the values of the variables in the expression that failed, and that's amazing. But it also means that, you know, you're dependent on them having got it right. But, like, yeah, after a bunch of years, I was convinced they got it right right enough to to let go of that.
Michael:
44:15
And you've used using the bare assert statement is great, but test is great. It's, yeah. But, yeah, unit test is fine. You know, works fine. It's a common style of testing that everyone's used to, and really the boilerplate boils down to, well, you need to inherit from test case, and you call assert methods, for doing your asserts.
Michael:
44:34
But, you know, not technical work.
Brian:
44:36
The the the main thing is it doesn't have Pytest fixtures.
Michael:
44:40
Right. It doesn't have picture fixtures, parameterized tests, and we have subtests in unit tests, which are a nice addition that not everyone will be familiar with. And and and so that's a context manager that allows you to have, a a a bunch of tests and a subtest that each can have a separate name and tell you which condition fails with which parameters. So, yeah, that's nice in unit tests, but there's there's a bunch of stuff. Pytest plugins are really pretty easy to to write.
Michael:
45:08
I did some contracting for a firm called Guroc, and they have a product called TestRail, and I wrote a Pytest plugin which reports for them, which reports Pytest test results and records them in test rail, you know, and that was that was way easier than I expected it to be. It was a that was great to do, you know, and and unit test is much harder to extend. And and really a consequence of using inheritance as an extension mechanism, common for frameworks to provide something to subclass, but like test results, if you want a test result that spits out XML, JUnit XML, and then you want another, somebody else has written another test result subclass that, reports, you know, orders your tests by time or whatever, you have to use 1 or the other in, unit tests. You know, it's composing you can't really compose the extensions. So the way Pytest does plugins with plugin points, I actually had a version of unit test that did this, unit test 2, and that, I didn't merge that back into unit test in the end, it became nose 2, which was popular for a while, but, I think that that was Jason Pellegrin, is that was is that his name?
Michael:
46:22
It was his project, and it was it was a one man project, and, you know, he he couldn't keep it going, unfortunately, which was a shame, but but that made unit tests behave very similar. But I think the way PyPI test does plug in is right. And it's an interesting point about extension through, inheritance as an extension mechanism from frameworks that it it, you know, it's it's, it lacks flexibility.
Brian:
46:44
It's easier to understand at first, but hook functions do make it easier in the long run.
Michael:
46:50
Hook functions with events, I think that's probably something that we're becoming more familiar with as more of us are doing async programming. And, you know, it is a different it's different machinery, a different pattern.
Brian:
47:02
Well, Michael, I need I'm just having a blast talking about testing with you. I need to wrap it up, but, any, calls to action for anybody
Michael:
47:12
they wanna Calls to action.
Brian:
47:14
Or, you know, or anything drop your we already talked about your Agile abstraction site.
Michael:
47:19
Yeah. Give to the homeless. But but the the other thing I can say which is useful, I did write an article which is up on opensource.com, and I'll drop you the URL, you can maybe put it in the show notes. And that's, I think it's something like 30 things every developer wishes they didn't have to learn the hard way, which is some some pithy wisdom about testing and developing my experience over the years. It's quite religious and dogmatic about the testing, but there's a lot of really good points, and some of them we we've talked about and a bunch of them we haven't.
Michael:
47:49
So I'll send you that URL. That's up on opensource.com, and I reckon that's an article worth reading.
Brian:
47:54
I love that article. Yeah. It's excellent. It's it's a huge brain dump on people of, like, some decent practices right off the bat. It's good.
Brian:
48:04
So thanks a lot for your time, and, we'll keep in touch.
Michael:
48:08
Thanks, Brian. I've I've really enjoyed it too.
Brian:
48:14
Thank you, Michael. Great discussion. Can't wait to have you on again. Thank you, PyCharm, for sponsoring the show. Try PyCharm yourself at test and code dot com slash pycharm.
Brian:
48:24
Thank you, Patreon supporters. Join them at testing code dot com slash support. Show notes for this episode are at testing code dot com slash 1 45. That's all for now. Now go out and test something.
00:00
The mock library that comes with Python as unit test dot mock started with Michael Ford. In this episode, Michael and I talk about mocking, of course, but also testing philosophy, unit testing, and what a unit is, and TDD, and even where Michael's towel is, and what color. Michael was instrumental in the building of testing tools for Python, and he's got a lot of great advice about testing. I hope you enjoy the episode.
Brian:
00:39
Welcome to Test and Code, Michael Ford, did I get the last name right?
Michael:
00:51
You got it right.
Michael:
00:51
Yeah. Yeah. It's the English version or the British version.
Brian:
00:55
So 4 with 2 o's.
Michael:
00:57
Yeah. That's right. Yeah. Double o. I'm a double o.
Brian:
01:00
So what why do is it just Americans that have trouble with that?
Michael:
01:06
The thing is the single o is so much more common. People sort of assume it's Dutch for some reason. It's not it's just English. We we back in the days when we used to have telephone books, there was always a a few in every telephone book. So, you know, not common, not rare.
Brian:
01:19
Whenever whenever I see your last name, I don't know if it's spelled the same, but I think of, Ford Prefect. But did he spell it with 1 o or 2?
Michael:
01:27
He he he spelled it with 1 o's. Then there's also there's also the Hoopy Frood in, the work of Douglas Adams. And I I I love that. I've always felt sort of an affinity with those works, both because of Ford Prefect and because I am a hoopy frute who knows where his towel is.
Brian:
01:41
Definitely. So do you have your towel with you?
Michael:
01:44
I don't have it with me, but I know where it is.
Brian:
01:46
You you know where it is. Yeah. It's
Michael:
01:48
it's purple and it's upstairs. It's one of these microfiber ones, So you can pack it really small for traveling. It's great. Perfect.
Brian:
01:57
So one of the things okay. So we wanna talk about mocking eventually, but what are you doing now? You do you have, like, a consulting company or something, right?
Michael:
02:07
Yeah. Contracting and training.
Brian:
02:09
Training. Alright.
Michael:
02:10
I I started training Python, doing Python training about 10 years ago now with David Beasley teaching his practical Python and advanced Python master mastery courses alongside my regular work. You know, I just do it a couple of times a year, and I really love doing it and enjoying it, and it's a good trade to be in. And then, I worked for Red Hat. My recent career history is that my last job was with Red Hat on Ansible Tower, their enterprise web application for managing Ansible and for managing computer infrastructure with Ansible, and I I worked on the test automation team there, helping them build out a test system. I worked for them for a year, and and I felt I really wanted to go self employed.
Michael:
02:58
It was time to to do that. And, so I've done half contracting and half training, and I really enjoy that. I still teach David Beasley's courses. I've also taught some Flask and some some testing and, object oriented theory for the Royal Ordnance Survey. Object oriented theory with Python, they wanted they didn't just want a Python course, they wanted something a bit more fancier, because they're the Royal Ordinance Survey.
Michael:
03:24
So So I did object oriented theory with Python. That was great fun. Contracting, my most interesting project has been for the United Kingdom Atomic Energy Authority, and that was from January to June last year, and that was working on software for designing fusion reactors.
Brian:
03:41
Oh, wow. So normally,
Michael:
03:43
that it's like it's like a dream job. I would just say he was actually a PhD guy, and it was his PhD project, so he but so he'd already committed to make it open source before he sort of accepted the job with the with the UKAA, so they are gonna make it all open source. It's blueprinted, and I worked with him on turning it into an engineering project. He had a and at one point, I mean, he was a phenomenally clever guy. I mean, I was literally actually removing descriptors off meta class, meta class no.
Michael:
04:11
Removing meta classes off descriptors. Oh, cool. Which, you know, just does not need to exist, you know. But but still, you have to be really clever to make as much of a mess as that. That was great fun.
Michael:
04:25
Yeah. And it so the normal way that the fusion reactor design is done, the academic world, they're all silo. Everyone has their field of expertise, but there's a whole series of things that need to be done. You know, they want to work out the magnetic, containment, the plasma containment chamber, so they've got place. They've got overall size power requirements, positioning of the magnets.
Michael:
04:47
The really cool thing they're doing now is breeder blankets. Basically, fusion reactors, you put hydrogen in, you ignite it with a laser, enough heat and pressure to trigger fusion and then using magnets you shape and contain the plasma, usually in a torus, and then this produces a stream of neutrons which hit big metal absorber blocks around the side of the lining the, the, the reactor, turns it into heat and from there it's normal turbine technology, which is very well understood. What they've done is they've found that if they put lithium into these blankets, that the neutron stream bombarding the lithium every now and then will hit the lithium, split into 2 tritium atoms, split into tritium, which is the fuel for the reactor. So you need enough tritium, which is very expensive to produce in radioactive, pretty dangerous, an isotope of hydrogen, and once you've got ignition, it's then it's then self feeding in terms of, of the Trojan. So that's a brilliant innovation.
Michael:
05:53
Anyway, you used to design these things by sort of getting your specs, sending them to the first guy to do the first bit of design. He'd take a month, send you back a bunch of numbers, which you send to the next guy. So the whole process took months. So you can't really do an iterative process of trying a bunch of things that way. Yeah.
Michael:
06:10
And this this guy, his genius is, an interdisciplinary approach, and he essentially wrote one application that does all of these different and it's for, it's for initial design work. It it's not the detailed design, but but you can now do an initial design, you know, with fancy OpenGL drawings, which are beautiful, and everyone thinks it's the whole point, but, of course, it's not. It's all the numbers it spits out. You can do that in 40 minutes. So you can do iterative design processes.
Michael:
06:37
So it's it's it is and is going to revolutionize fusion reactor design. It was a fantastic opportunity to to work on it and so much fun.
Brian:
06:46
That's so cool. Did you say that was open source stuff?
Michael:
06:50
Well, I I emailed him recently to ask him if it was open source yet, and, he didn't reply to that bit. They were they were due to do it, before now, so I think the answer is no. Oh, okay. It's called Blueprint. There are some, there is a paper he's released on it.
Michael:
07:06
So yeah.
Brian:
07:08
Okay. No.
Michael:
07:08
I'm just gonna drop a link if it was there. Yeah. I'll be all over Twitter with it when when it comes out because it's, you know, it's a it's a fan it's fantastic to play with. Yeah. You know, you you can design fusion reactors in your living room.
Brian:
07:24
Thank you, PyCharm, for sponsoring this episode. I first tried PyCharm when they started supporting Pytest many years ago. Their support for Pytest is now amazing. I was a long time Vim user, so next I needed to test the idea Vim plugin so all of my finger muscle memory still worked while editing. Check.
Brian:
07:41
It works great. There's lots of reasons to love PyCharm. But for me, it is because they have the absolute best user interface for test automation. Then I learned many more ways PyCharm can save me time, like really great support for editing markdown, HTML, CSS, JavaScript, remote connections to database, and amazing version control support. Really, it's the best git diff tool I've ever used.
Brian:
08:05
And now version 2020.3 is out, and the shift shift, the find anything key sequence, even lets you search git commit messages. What even? That is so awesome. Tons of other cool features have been added in 2020.3. Check it out, and I hope you enjoy it at testencode.com/ pycharm.
Brian:
08:28
I know you've got a site, agileabstractions.com. Is that the That's
Michael:
08:33
yeah. That's my, professional site, but I haven't added any of the the projects I've worked on since 2019 and 2020. I don't think so. It's a little out of date, but that's my professional site.
Brian:
08:43
Yeah. But if somebody wanted to hit you up for training or something, they could go there. Right?
Michael:
08:47
Yeah. Oh, michael@python.org.
Brian:
08:49
Okay. A python.org email. Yeah.
Michael:
08:56
Instant credibility. I I got it because I I helped out a lot on the web. I was one of the webmasters on various mailing lists administration. And, and yes. And I asked for Michael, and nobody else had taken it.
Michael:
09:07
But it is a beautiful email address. I'm very proud of it.
Brian:
09:09
Yeah. So there was there's this, there's a Python feed thing. I can't remember where it is. Python do you remember what that was?
Michael:
09:17
Python? Is that what you mean?
Brian:
09:19
Say that again?
Michael:
09:20
Python feed. Do you mean planet Python?
Brian:
09:22
Yeah. Planet Python. I I just remember that because I think that's the first time I ran across your name because, planet when I started blogging about Python, I heard somebody say, well, you gotta get you gotta get your, your blog on planet Python to get listed or people won't pay attention to it. And then so I requested it and I think you replied and said, okay, it's there. So
Michael:
09:46
It was one of the things I was looking after. That was back in what I think of as the golden days of the Python community, back when Python was about to explode with the web revolution and Google adopting Python. And and, prior to that, Python had mostly been used only by enthusiasts, only by people who really loved the language, which made a beautiful community full of passionate people really eager to to teach you, you know, and then Python just exploded. And so back in back in the day, having your blog on planet Python, you know, you you could get a, you know, a 1000 views for a blog entry. The glory days.
Brian:
10:24
Yeah. Now the other then I ran across your name next when I started researching, testing stuff. And you had a bunch of testing articles. But then also, your name is attached to the, the mock library in unit test.
Michael:
10:42
That's right. That's right. Yeah. I I originally wrote what's now unit test dot mock and maintained that as a library for for quite a while. And that that came out of my first program and gig for Resolver Systems in London.
Michael:
10:55
I started with them back in 2006. And they were doing all extreme programming, so it was all pair programming, fully test driven development, customer representative doing prioritization, estimations, tracking, velocity, all of this this kind of stuff. We did that rigorously for 4 years, and that was an amazing experience. And it yeah. It came out of that time, and in particular, I got a test a passion for testing as a way of ensuring product quality in, in programming.
Michael:
11:26
Whereas before the sort of style of programming that I was used to do, knowing that things worked was a real challenge. You had to try everything. And we can automate so much of that. Yeah. So so yeah.
Michael:
11:39
That's that. I I I was passionate about Python, and I became passionate about testing in that time.
Brian:
11:45
And so since I guess since then, you, did you still incorporate testing within all your development processes then?
Michael:
11:55
Yeah. I mean, we did test driven development rigorously. We did extreme programming rigorously for 4 years. So that was the first thing we do is write a functional test which exercises end to end the the feature that we're trying to add or demonstrates the bug. And then we start writing unit tests and those are layered.
Michael:
12:13
And we had a test to code ratio of at least 3 to 1. And we, you know, we over tested in all sorts of places. We had testing couples to the implementation. So I learned a great deal about testing and the problems, if you over test, you couple your unit tests, you're testing the implementation, so you're testing your private methods. And then when you come to refactor, and you change you want to change your code, change the way you call your code, your tests tell you nothing useful and they're broken because even when you fixed your code again, the test is not going to show you the right thing, they're testing old code.
Michael:
12:48
Functional tests, your end to end tests, if you've got good reasonable end to end coverage, even just a set of smoke tests, then you can refactor and you can be reasonably confident that sort of you haven't broken functionality because with refactoring, your end to end tests don't change. So I think there's a lot more value in end to end testing. I think there are dangers in over testing. I like to say unit testing is about testing to the unit of behavior, not the unit of implementation. Test through the public API.
Michael:
13:15
If you can't test through the public API, then your abstraction isn't right, you know, typically. These sorts of things help you to avoid over testing. Scripting tasks, I won't test. Adding tests to big legacy projects is also very challenging, particularly in the face of ongoing feature work, and that's something you have to sort of work to incorporate gradually and pragmatically because businesses, you know, you only get paid as a programmer if the business keeps going.
Brian:
13:44
Yeah.
Michael:
13:45
So you have to, you you have to incorporate, building in tests and testing to give you keep sanity with, the, the the the work of maintaining and extending the legacy project.
Brian:
13:59
I really like that, I'm gonna steal that unit of behavior, not unit of implementation. I like that a lot.
Michael:
14:05
Yeah. Because that's what you wanna test. Right? You wanna test it does the right thing. You don't wanna test how it does it because in theory, at least, it would be very much nicer if you could change the how, and your tests still tell you useful things.
Michael:
14:18
And if you're only testing through the public API, so long as you've got the public API right, you know, often you need to change that, then, you know, you you're free to change the internals and your tests still tell you useful things.
Brian:
14:31
Yeah. Now, and then, of course, as you know, there's always there's always cases where, you really wanna beat up some algorithm bit in the middle. And so there is gonna be, some real tests around that.
Michael:
14:45
There there there are there are bugs where you can't replicate it without, you know, it's some timing issue or some file system quirk. And the easiest way of replicating is sticking some invalid data into some some internal state, because you know what triggers the bug. And you you know, that that's gonna happen. Yeah. You know?
Michael:
15:02
I mean, the all generalizations are wrong, including this one.
Brian:
15:08
Yeah. Yeah. Okay. I'm definitely gonna have to get this one transcripted because we got a lot of gems in here. The okay.
Brian:
15:15
So mocking, you got into mock mocking and and stuff is a is is a natural part of I guess, it depends on the year you were doing it, extreme programming and, and test driven development. But, but mocking is a little bit misunderstood by a lot of people, including probably myself.
Michael:
15:36
And probably me.
Brian:
15:40
So and and partly, it's partly your fault.
Michael:
15:45
Uh-huh. Uh-huh. Alex Gaybert blames me.
Brian:
15:49
Well, there's I mean, there's a well, if people start researching it, they get into things like, well, mocks the article by I'm gonna get it wrong, called mocks art stubs. Can't remember.
Michael:
16:01
By Martin Fowler. Yes. Martin Fowler. The static typing of that he defines various categories and types of mock, ways you can use mock, and he defines them as different objects. And it's the defining them as they're fine as categories, but they're fairly rigid definitions.
Michael:
16:15
And I think by his definition, mock is all of his types of mock except a mock.
Brian:
16:23
Okay. Yeah. I was curious about that.
Michael:
16:24
So I object to his definitions.
Brian:
16:27
Okay. So in Python, we use we use the unit test mock library or often wrappers around it. So, like, pytest mock has a is a is a wrapper around the mock object thing so that you can use it's basically context managers built on around it, which is kinda cool. Also really cool that you can use them as context managers anyway right off the bat. I didn't know that at first.
Brian:
16:53
But, yeah. Some
Michael:
16:54
That's that's an interesting point. And that was an innovation in mock that actually that came it was from a guy at Resolver Systems. It wasn't actually me, although I I get the credit for it. It was a guy called Tom. I forget his surname.
Michael:
17:07
Oh, dear me, poor lad. Good guy. Who discovered that so patch is a decorator patch, monkey patch is things, so you can inject or mock into, any, you know, all sorts of places. And, the reason Alex Gaynor blames me, and I think possibly what you're about to say, I might steal your thunder, is I mean, patch makes it possible to test code that was essentially really hard to test before. So the combination of mock and patch make it hard make it possible to test code that's really hard to test, which doesn't give you an incentive to write code that's easy to test.
Michael:
17:43
And code that's easy to test is generally better code. So patch and mock let you disguise the fact that you're writing terrible code, and that is definitely true. But the innovative thing is that patch patch which puts mocks into place, the the thing that it does that's really powerful is it undoes it. It puts things back the way they were before. And monkey patching things is easy, un monkey patching them is hard, and there's a lot of logic in in in patch that knows how to do that.
Michael:
18:12
So you you can you can use it as a context manager with patch, time dot time as mock time.
Brian:
18:20
Yeah. And
Michael:
18:21
that and then anything in for anything inside the context manager sees time.time as your mock object, which you can then configure the return values. But then outside the context manager, time.time is restored to what it was normally. So the scope of the effects of the patch is limited by the context manager, which is what context managers are great for, you know, a visible scope of effect. And you can also use it as a decorator where during the function for the inside of the function that's decorated, the patch is in place there. This was actually done in Python 2.4 originally back before we had context managers.
Michael:
18:57
So being and it was Tom who at Resolver Systems who realized that the way that we do decorators is entirely compatible with context managers. So Patch can do both, and Patch is an interesting beast. And there's actually in contextlib now, there's context decorator, where you can write context managers that also work as decorators, and that came out of, what first happened in mocking at Resolver systems.
Brian:
19:24
That's true.
Michael:
19:25
Python 2.5. Bit of Python bit of Python archaeology.
Brian:
19:32
Yeah. So if if people have kinda missed it so far, how how do you describe a mock, like, to somebody that doesn't even know what it is?
Michael:
19:42
Okay. Right. So this is so a mock object is an object that can pretend to be any other object, essentially. Now the thing that Martin Fowler, defined about mocks and what was common at the time, so he talks about mocks and stubs and I forget the the the the another one. But the the essential point about a mock is that it record you can record your expectations and then replay those.
Michael:
20:09
So you create a mock object, you configure the mock object and say, how I think this is the Martin Fowler one and the Java mocking libraries and, the Python mocking libraries at the time. You create a mock object, you say, I want you to have these methods, and I expect this method to be called. It should return this. Then I expect this method to be called. And then you do do you run your code and you hit replay, and it throws an exception if it was used in the wrong way.
Brian:
20:35
Oh, okay.
Michael:
20:35
So that's the record replay style of mocking. And for me, that puts the you you record your expectations on the mock and then you call your code and that's us about face. You know, what I want to do is I want to inject a mock into the system, run some code, and then I want to be able to make assertions that it was used in the right way. Because not all of the things that happened to it might be relevant. It might just be one particular thing.
Michael:
21:02
I want to assert that you were called with this argument. I want to assert the mock might even just be going in there just for the purposes of returning a pre canned value, stubbing out a a system function that you don't want called in a unit test. That's a very good use of mocking. Mocking external dependencies to return deterministic results for the purposes of testing to avoid external calls in your unit tests. That's the classic and a great use of mocking.
Michael:
21:25
So you might you might mock out API calls, mock out network calls, mock out file calls. And there's some some support in the mock library for particularly for files.
Brian:
21:36
Yeah. Yeah. Like, for instance, I I think of an example which probably isn't very common. But, if my test if I've got a system, I don't know, logging system or something, and if I find something critical, it's gonna email a bunch of people.
Michael:
21:49
Right.
Brian:
21:50
During the test, I don't wanna actually email everybody. But I can make sure that the the appropriate call to email the right people is called during the test.
Michael:
22:01
With the right premises. Yeah. Exactly. That's the sort of thing that mocking's for. And just to finish off the thought previously, so instead of being record replay style, mock is, AAA.
Michael:
22:12
What is it? Arrange, act, assert. You set up your code, you call act, which is arrange, then you call your code, which is act, and then you make the asserts. So, unit test dot mock, what it does is you can configure it so that methods will return values, or a method call call will have a side effect, like raising an exception or calling another function, or you can give a sequence of values for multiple calls. All sorts of ways you can configure mock objects that can pretend to be any objects, They can pretend to be a dictionary.
Michael:
22:40
You can configure any of the magic methods, all of this kind of stuff. You put it in place, and then it record all of the calls are recorded on it. There's a and and there's some convenient there's some convenient assert methods and convenient call methods. So, the reason I created it was for for for two reasons. The first thing was that, we were doing full test driven development, and we were creating all of these, little stubs objects in scattered throughout our code base with like a mock workbook, a mock worksheet, a mock cell, a mock row, a mock column, you know, and overall it added up to sort of 100, maybe thousands of lines of code.
Michael:
23:16
And initially I was like, I can replace all of this with, a Python object with a done to getattr that responds to every attribute lookup. So the initial implementation was, you know, about 30 lines of code, and it was to replace all of these mock objects in the resolver code base. But a requirement from Giles Thomas, who was the CTO at the at the time, was that we had to have a way of limiting the API. So if we had accessed an attribute that shouldn't exist, it would still raise an attribute error. So that's where all of the spec stuff in mock came from.
Michael:
23:48
So that was where it originally came from. It was motivated also by the desire of not wanting none of the existing mock frameworks, all of them in Python, they were all this record, replace style, which I didn't like. And there, the testing in Python community in Python was particularly close knit and fun, and we got the testing in Python boff at Pycons, which had a great run for a good number of years. It was a lovely community to be part of, and mock really evolved very rapidly, with users in the testing in Python mailing list and feedback from them, and competition with other people writing, libraries. I was I was full of passion and activity in those days, you know.
Michael:
24:24
If any mock library came out with with a feature I thought was good, I'd have a new version out with with that feature, you know, in a in a few days, you know. I answered every email about mocks on the testing in Python, email list, showing them how to do it with my library. You know, I blipped the competition.
Brian:
24:43
Well, it's interesting. The testing in Python
Michael:
24:45
share enthusiasm.
Brian:
24:46
So the the testing in Python mailing list still exists, has very very little traffic on it. So a lot of people think maybe it's dead, but it, we still get I mean, I still pay attention to it and replies go real you get really good replies. If you ask a question there, it's pretty good. Now I probably regret putting this in a podcast because suddenly people will start using it.
Michael:
25:09
Start using the testing in Python mailing list, folks. Right.
Brian:
25:14
But, yeah, you've got we've got some really great smart people paying attention to it. So, but the okay. So what is your relationship if the I'm thinking of this as somebody coming in and going, maybe I should use mocks in my testing. What do we tell people that haven't used this before? And, because there's there's this like you described, this test driven development model, which is okay.
Brian:
25:38
We have to just there's 2 there's 2 huge there's classical and mocas. But obviously, we're talking about mocas TDD, which means we try to test every function in isolation with everything around it.
Michael:
25:51
Right. Right. Right. The the trouble then is you're not testing the wiring between your parts. The the great advantage of, the great advantage of doing that is that your tests run nice and fast.
Michael:
26:03
But if you have you can have full coverage at the individual function and method level and still have no idea if your system works because this sort of wiring between your components is not tested at all. And the same effort expended at just at the functional test level would give you confidence that the application actually works. The advantage of test driven development and the reason that, it's called test driven development the other way of putting it is I often talk about test first. But the idea is that, that the test drive the design. And this is what I love about test driven development, and this is what I take from it even if I'm not always religious about test first these days.
Michael:
26:40
And it's that step of thinking about the design. It's that if you do test first, the first step has to be not what's the solution, let me bang out some code. It's how do I call this? How ought this to work? What's the best, the right API for me to be testing?
Michael:
26:57
And so there's that, it bakes in right at the start, the first bit of thinking is how should this look like? Because the default otherwise is you bang out some code, you think it works, I'll add a test and what you're testing is whatever you happen to come up with, not the best way of doing it. And the other aspects of TDD, the simplest thing that could possibly work building up tests incrementally, is that by evolving a design like this, as long as you pay off the technical debt of doing the refactoring, incorporate the cost of refactoring into your estimates, that you actually come up with better solutions by basing your incremental approach on actual usage? So those are lovely reasons to do test first. And as you say, if you're doing test first, if you're trying to maintain some level of isolation, you're gonna need some mocks.
Michael:
27:44
But I think that's the question then is the whole the question then becomes, what is my testing philosophy? And and the specific question we're asking is, how do I get started with mocks? And so I think we can answer that much more simply, but we we can say, look, the the two things in unit test dot mock library are the patch decorator, patch context manager, and the mock classes, the mock objects. And, actually, most of the time, probably, patch is gonna create your mocks for you. So first, you need to use patch.
Michael:
28:12
And this this often confuses people, so we can talk a little about that about it if you want. But basically, you say with patch and then the location of the object. I'm gonna patch out a method on a class. It's module name dot class dot method as mock object. And then inside the context manager, you can configure the value of the mock object.
Michael:
28:31
We probably want to say, mockmethod.returnvalue=3. And then after we've executed our code, we simply say mock object dot assert called with and assert it was called with the right parameters. So the very basics of using patch to inject a mock and making the asserts are quite straightforward. But where should we do this? Well, an obvious place is replace file access with, a mock object.
Michael:
28:58
You can if you patch by, open, the built in open, then you know it was called if you've got your results, and you'll save time in your unit test. Patch out calls to time dot time with the deterministic with something that's gonna return you something deterministic. That that, system calls, network calls, database queries, anything where you wanna return pre canned deterministic results, and you can avoid a real life network call. If you're not testing your database access, if you find if you're happy about the the the database access, you that's covered maybe the integration test level. Mock them out in the at the unit test level.
Michael:
29:34
Make your test faster and less dependent on your underlying model. This this kind of stuff is the place where mocking can give you a win.
Brian:
29:41
Yeah. And these these external parts of your system, well, it's really the the system under test. And I think, one of the things people don't talk about a lot is the the test architecture often mimics the people architecture. So we Interesting. Well, I mean, like, let's say I'm working on a I'm working with a database, but I've got a database layer that, some other team is working on.
Brian:
30:07
I think it would be and and I'm I'm not responsible for the user interface. I'm responsible for, like, this middle layer of stuff. Mhmm. Mhmm. It's completely reasonable to then, I think that there needs to be system level tests.
Brian:
30:21
But as a team, I'm gonna I'm gonna, like, probably feed my API and stub out my dependencies or mock my dependencies.
Michael:
30:31
The the there's, the principle is sometimes expressed as don't test the browser, which really only applies to web application development. But you test the code you own, not the cut the code you don't own unless you have to.
Brian:
30:43
Right. And then other the other really depend doesn't matter what style of testing you're doing, whether you're doing, a lot of test driven development or unit tiny unit tests or even functional tests. People are going to eventually need to mock out their external stuff like API calls to external services.
Michael:
31:04
Okay.
Brian:
31:06
You're not gonna like, oh, an example that, I'm blanking on his name, but, Harry Percival brought up is, the credit card processing. You're definitely not gonna hit the credit card API unless there's a debug one. But, why not just mock that out or stub it out? So the other thing that that Harry brought up, which I thought was a cool idea, is any real third party system. Not not some other team's system in within your own company, but like a third party, like a credit card processing or something, that you really wanna try to, replace their API with wrap it in in your own object or own module so that you Mhmm.
Brian:
31:53
Have a limited set of API functions or entry points into that service. And then Yep. Then that's a natural place to mock or stub out those functions is to to hit those. So
Michael:
32:05
Yep. Yep. And and if your API system is making network calls, then you can test this this abstraction layer by mocking out the net the network calls, calls to request or, you know, whatever, or the the the calls to their client. And, so you can be sure that's worked, that's doing the right thing. And then you've got, a nice abstraction layer where you can put your mocks and completely replace that bit of the system.
Michael:
32:34
So, yeah, you can test the 2 layers separately and be confident that it does work end to end. I know that the system under test calls the, the the the, the API correct the the API correctly that we've provided, the abstraction layer that we've provided, and I know that that makes the correct API calls because we've tested that as well. So yeah. Yeah. I like I like that approach.
Brian:
32:55
The other thing that I think is neat, and I wonder if this is one of the reasons why you get blamed for bad designs, is that I don't really have to understand dependency injection to use mock.
Michael:
33:11
Yeah. That was another motivation. In Python, dependency injection typically boils down to adding extra parameters to your function signatures. And I think function signatures are part of your API, and messing with those as a way of managing dependencies is not necessarily ideal. I might have softened on that.
Michael:
33:31
I I I kind of think, I I sometimes say this, and I think I think it's a a useful thing to say, is that every time you use patch, it's it's an admission of failure. You know, mocks mocks ought to be your last resort. It ought to be possible to test your system and the parts of your system. It ought to be testable. And if you have to replace a bit of it inside the live system in order to test it, then what you're saying is I couldn't design the system in such a way that I didn't need to do this.
Michael:
34:03
So working to minimize your use of mock and patch means that you're gonna get the best value out of it, I think. Only the situations where mocking is really clearly the best approach rather than making it the tool that you turn to first. Because there's, yeah, there's a definite there's a I mean, I've written code that whether it's, first it does this function call, then it does that function call, then it does another function call, then it returns the results. And I've mapped out all my dependencies. I've mapped out this function call and the other function call and that function call.
Michael:
34:32
And now I'm testing my mock objects and not my code.
Brian:
34:35
Right.
Michael:
34:35
You know, and that's crazy. You know, there I'm that's you've really tightly coupled your test to your implementation, and you're testing completely testing your implementation details. That's not a useful test, you know, really.
Brian:
34:48
So let let's, take that a bit further. So let's say I have, in order to test my credit card processing part of my system, I I I have I've been using mocks or something with that. What's the alternative? How would I
Michael:
35:03
I I I think having cleanly defined layers helps so you have a single point. I mean, the other thing I might do is do this via config files, you know, have an or have another mechanism that loads, a mock part of the machinery in place when I'm running under a dev config. You know, it's like, I want to be able to guarantee that when I'm running my tests, I can I can never send a, a credit card request? So I don't want to inject live things into the live system to to make sure it doesn't. I want the system I stand up to not be capable of doing it.
Michael:
35:42
Okay. So particularly for credit card or anything that's privacy or security critical, I would I would think about having machinery that does that for me. And it might put a mock API in place, and maybe that would use mock objects. But, you know, you'd have a shim layer. You you'd have some shim machinery.
Brian:
36:03
Well, that kinda reminds me of, like, the database stuff. So, one of the things that people often do in testing is to replace the live database or a file based database with an in memory database.
Michael:
36:16
And Right. It's ex exactly. Very yes. Yes.
Brian:
36:19
And but there but a lot of the databases like post and, others have a a memory feature. So you can just use the live database and just have it be in memory if you want, for instance. Mhmm. So
Michael:
36:33
Yeah. I'm yeah.
Brian:
36:36
Any, any warning signs to give people that I I like the idea of, like, look at patch first and and possibly look at, like, your external system, like, like, you gave a list, which is good, network, system calls, things like that. And then the easiest isolating those, like, I guess,
Michael:
36:59
I Yeah. Yeah.
Brian:
37:01
Making sure that those aren't all over your system. I wouldn't put, like, request, calls in every file of your
Michael:
37:08
your Yeah. Yeah. Have have it done in in a layer, which is much easier to step out and test and and have your calls go through that. And here your sort of testing strategy starts to influence your design, and I think in a good way. It's like putting the side effects into a separate function, so as much as possible, your functions are pure functions, which are then really easy to test and it's really easy then to stub out, mock out the bits of your code with the side effects.
Michael:
37:39
It's the same concept. If the file writing happens in its own function, then the rest of the function is much easier to test. When you're This this this kind of thing. Yeah.
Brian:
37:52
When you're teaching people, I I see on your training site that you do teach, testing training course. How much of how much of that is, around mocking or is that just a small part of what you're teaching people?
Michael:
38:05
It it it's a part. It depends a little bit on the the customer and what their priorities are. I maintained unit test in the standard library for quite a while, that was my other involvement with testing. I added test discovery, I helped break it up into a package. I think Benjamin Peterson actually did the work on that one, but it was controversial.
Michael:
38:24
So I committed it with under my name as the maintainer, and it happened. But even so, I still recommend pytest for new projects, and I use pytest for new projects. And it's, you know, I will teach people to use unit tests, and there's often a section in courses I I teach in general on Python. But generally but when I'm starting a new project, it's pytest I turn to, and it's pytest I teach.
Brian:
38:48
So do you use the with, if you're using Pytest, do you use the Pytest mock plugin or just use unit test mock directly?
Michael:
38:57
I tend to use unit test mock directly. That's possible. Usually because I teach mock as a separate section. Okay. Because, you know, just because, you know, you can spend a day or 2 days on mocking depending on how much of the API you wanna learn to use and how many different scenarios you wanna cover and, you know, and I'm quite fond of unit test dot mock, but I'm not sure I'm not sure that's necessarily best practice with pytest.
Brian:
39:23
Okay. The, so I when I when I'm reaching for it, I usually use the the the Pytest mock plugin. But it has, like, this marker object that you can pull in and stuff like that. But, anyway,
Michael:
39:37
the It's a it's a it's a fixture. Right?
Brian:
39:40
Yeah. It's a fixture that
Michael:
39:41
My my my one problem with Pytest is how easy it is to get into fixture hell. And I I even worked at one company where they they had a culture or convention of providing stuff via fixtures. So you would use fixtures instead of imports. And so you'd look in your code, and, like, your ID no longer has a clue where things come from or where they are. And they've got fixtures taking fixtures, returning fixtures, and you're trying to work out where something happens.
Michael:
40:09
And you're following this graph of fixtures, then you're up to the 8th level. And you're like, why? Why? You know, my life has descended into hell. Okay.
Michael:
40:20
So so, fixtures are fantastic, for, you know, limiting the scope of stuff, and I love the scoping, you know, but it's like you use them sparingly. You know, there's like imports are great. Use import ports, not fixtures.
Brian:
40:35
Well, okay. I'll disagree with you on that.
Michael:
40:39
But it I fixtures fixtures are great as well. I've just been in fixture hell.
Brian:
40:43
But also just a reminder that there is no framework or strategy that can prevent you from writing really crappy code.
Michael:
40:52
Oh, yeah. I know. Yeah. You're right. Right.
Michael:
40:55
Maybe maybe Scala or Haskell.
Brian:
40:58
Well, okay. So one of the things that that you hinted at, I just wanna, like, inject this in here, dependency inject this into the conversation, The, with pytest, you can put fixtures either in your test file or in a conf test file, and you can have one conf test file for every directory in your test structure.
Michael:
41:17
Oh, that was the the other thing, wasn't it? It's like this fixture I'm using it, where the hell does it come from?
Brian:
41:23
Yeah. You can have And
Michael:
41:24
then you're scattered across your code base like
Brian:
41:27
And they can and your fixtures can depend on any other fixture that is anywhere in its parent to hierarchy. But I recommend people in a project to have 1 conf test file at the top. You can
Michael:
41:39
I mean, I I love that feature? It's like, if I put a conf dot py at the root of the project, pytest knows what the root of my project is now and, you know, a whole bunch of other things just work.
Brian:
41:49
Yeah.
Michael:
41:50
So it one is one of the nice things about Pytest. Yeah.
Brian:
41:52
So that also and that also helps people if they they they know if my fixture isn't in the file I'm looking at, it's over here. Just helps your whole project.
Michael:
42:03
Yeah. Yeah. Pytest is really flexible, you know.
Brian:
42:07
But they the when I started looking at all these things, I did look at unit test and, I have used unit tests. The I actually was annoyed with a lot of the people's complaints of unit tests. Is the the thing that people say is it's too much boilerplate. And with test discovery added to unit tests, there's not you don't have to do the, like, name equals main thing in your file. There's not a lot of boilerplate, I don't think.
Brian:
42:37
But
Michael:
42:38
So yeah. I mean, it used to be if you used, unit test, you had to write test collection yourself and test running, and there's a test result, there's the test collectors, there's a whole load of machinery. And really, test discovery was the minimum to kind of bring it up to the sort of usable level for for for projects. A lot of the criticism I have come from the, it's a port of like JUnit version 5 or something and JUnit evolved a huge amount and unit test didn't. So we have the non PEP 8 style naming And pytest has function tests as functions, whereas with unit tests, you inherit from test case
Brian:
43:20
Yeah.
Michael:
43:21
And, and you write test methods, and you use the assert methods instead of the assert statement. And honestly, that's the boilerplate people are talking about generally. And it's like, yeah, I I like writing test functions in for pytest, but I also like grouping tests together in classes. You know? So I don't mind grouping tests together in classes.
Michael:
43:42
I don't mind the assert methods. I was suspicious for years of the assert, the pytestassert magic rewriting because the code if you get a you know, the code that runs the test run, they've rewritten the the bytecode magically so they can tell the, the intermediate values at all the points. They can tell you the the values of the variables in the expression that failed, and that's amazing. But it also means that, you know, you're dependent on them having got it right. But, like, yeah, after a bunch of years, I was convinced they got it right right enough to to let go of that.
Michael:
44:15
And you've used using the bare assert statement is great, but test is great. It's, yeah. But, yeah, unit test is fine. You know, works fine. It's a common style of testing that everyone's used to, and really the boilerplate boils down to, well, you need to inherit from test case, and you call assert methods, for doing your asserts.
Michael:
44:34
But, you know, not technical work.
Brian:
44:36
The the the main thing is it doesn't have Pytest fixtures.
Michael:
44:40
Right. It doesn't have picture fixtures, parameterized tests, and we have subtests in unit tests, which are a nice addition that not everyone will be familiar with. And and and so that's a context manager that allows you to have, a a a bunch of tests and a subtest that each can have a separate name and tell you which condition fails with which parameters. So, yeah, that's nice in unit tests, but there's there's a bunch of stuff. Pytest plugins are really pretty easy to to write.
Michael:
45:08
I did some contracting for a firm called Guroc, and they have a product called TestRail, and I wrote a Pytest plugin which reports for them, which reports Pytest test results and records them in test rail, you know, and that was that was way easier than I expected it to be. It was a that was great to do, you know, and and unit test is much harder to extend. And and really a consequence of using inheritance as an extension mechanism, common for frameworks to provide something to subclass, but like test results, if you want a test result that spits out XML, JUnit XML, and then you want another, somebody else has written another test result subclass that, reports, you know, orders your tests by time or whatever, you have to use 1 or the other in, unit tests. You know, it's composing you can't really compose the extensions. So the way Pytest does plugins with plugin points, I actually had a version of unit test that did this, unit test 2, and that, I didn't merge that back into unit test in the end, it became nose 2, which was popular for a while, but, I think that that was Jason Pellegrin, is that was is that his name?
Michael:
46:22
It was his project, and it was it was a one man project, and, you know, he he couldn't keep it going, unfortunately, which was a shame, but but that made unit tests behave very similar. But I think the way PyPI test does plug in is right. And it's an interesting point about extension through, inheritance as an extension mechanism from frameworks that it it, you know, it's it's, it lacks flexibility.
Brian:
46:44
It's easier to understand at first, but hook functions do make it easier in the long run.
Michael:
46:50
Hook functions with events, I think that's probably something that we're becoming more familiar with as more of us are doing async programming. And, you know, it is a different it's different machinery, a different pattern.
Brian:
47:02
Well, Michael, I need I'm just having a blast talking about testing with you. I need to wrap it up, but, any, calls to action for anybody
Michael:
47:12
they wanna Calls to action.
Brian:
47:14
Or, you know, or anything drop your we already talked about your Agile abstraction site.
Michael:
47:19
Yeah. Give to the homeless. But but the the other thing I can say which is useful, I did write an article which is up on opensource.com, and I'll drop you the URL, you can maybe put it in the show notes. And that's, I think it's something like 30 things every developer wishes they didn't have to learn the hard way, which is some some pithy wisdom about testing and developing my experience over the years. It's quite religious and dogmatic about the testing, but there's a lot of really good points, and some of them we we've talked about and a bunch of them we haven't.
Michael:
47:49
So I'll send you that URL. That's up on opensource.com, and I reckon that's an article worth reading.
Brian:
47:54
I love that article. Yeah. It's excellent. It's it's a huge brain dump on people of, like, some decent practices right off the bat. It's good.
Brian:
48:04
So thanks a lot for your time, and, we'll keep in touch.
Michael:
48:08
Thanks, Brian. I've I've really enjoyed it too.
Brian:
48:14
Thank you, Michael. Great discussion. Can't wait to have you on again. Thank you, PyCharm, for sponsoring the show. Try PyCharm yourself at test and code dot com slash pycharm.
Brian:
48:24
Thank you, Patreon supporters. Join them at testing code dot com slash support. Show notes for this episode are at testing code dot com slash 1 45. That's all for now. Now go out and test something.
