Modern JavaScript Testing: Integration, Contract Testing & AI Tools - JSJ 692

WEBVTT
1
00:00:05.240 --> 00:00:09.119
Hello, everybody, Welcome to another thrilling episode of JavaScript Jabber.
2
00:00:09.160 --> 00:00:11.439
I am Steve Edwards, the host with the face for
3
00:00:11.560 --> 00:00:13.800
radio and the voice for being a mine. But I'm
4
00:00:13.800 --> 00:00:17.519
still your host with me on the panel today, all
5
00:00:17.519 --> 00:00:19.800
the way from Tel Aviv, Israel's mister Dan Shapier.
6
00:00:19.839 --> 00:00:20.559
How are you doing, Dan?
7
00:00:21.399 --> 00:00:23.079
I'm doing well. How are you, Steve?
8
00:00:23.839 --> 00:00:24.960
Doing good? Doing good?
9
00:00:25.039 --> 00:00:28.679
It's Wednesday morning, first day back at school for kids here, so.
10
00:00:29.320 --> 00:00:30.079
That's always good.
11
00:00:31.280 --> 00:00:35.200
I'm still hot and toasty there in Tel Aviv as always.
12
00:00:35.039 --> 00:00:37.960
Yeah, when is it not? Yeah?
13
00:00:38.240 --> 00:00:40.240
Yeah, it's a little cooler here, starting to cool down
14
00:00:40.280 --> 00:00:41.920
a little bit, which is always a bummer, but.
15
00:00:43.479 --> 00:00:44.240
It is what it is.
16
00:00:45.119 --> 00:00:48.640
And our guest today coming all the way also from Israel,
17
00:00:48.719 --> 00:00:49.719
mister Joannie Goldberg.
18
00:00:49.759 --> 00:00:50.479
How are you doing, Yanni?
19
00:00:51.560 --> 00:00:54.320
Hey, hey, great, great Tom. I'm really happy to be
20
00:00:54.359 --> 00:00:58.039
here today. It's one degree cooler here comparing with Al Aviv.
21
00:00:59.039 --> 00:01:01.759
Very hot, but the good news that after two months
22
00:01:01.759 --> 00:01:04.359
as of being at home, kids are finally back at
23
00:01:04.400 --> 00:01:09.239
school yesterday or before, so.
24
00:01:08.280 --> 00:01:10.319
Yeah, back to work a little yes.
25
00:01:10.599 --> 00:01:13.920
And for those of you that might see Yohannis drink,
26
00:01:14.000 --> 00:01:16.840
he is not drinking and podcasting that is tea, not beer.
27
00:01:17.000 --> 00:01:21.439
I clarified that before the episode, so so not that
28
00:01:21.480 --> 00:01:24.959
it's not okay to drink and podcasts, but just clarifying.
29
00:01:24.719 --> 00:01:29.000
I literally drank beer on stage while giving certain talks.
30
00:01:29.840 --> 00:01:32.959
My contention is that it only made the talks better.
31
00:01:34.680 --> 00:01:37.239
Started listening up a little bit. Is that it Yeah,
32
00:01:37.640 --> 00:01:38.560
if you probably.
33
00:01:38.319 --> 00:01:40.439
Know, is it also legit in the States? I know
34
00:01:40.480 --> 00:01:42.719
it's you in Europe, it's okay. But if I drink
35
00:01:43.040 --> 00:01:46.280
at like it's the States during the talk will be
36
00:01:46.359 --> 00:01:47.079
considered PC.
37
00:01:49.400 --> 00:01:51.840
I've never heard anything against it. I guess it depends
38
00:01:51.879 --> 00:01:55.400
on how well you handle the the alcohol and well
39
00:01:55.599 --> 00:01:57.640
you stay on topic and have somewhat of a sense
40
00:01:57.640 --> 00:01:58.040
of humor.
41
00:01:58.120 --> 00:02:00.439
Then you're saying that the first two okay.
42
00:02:01.040 --> 00:02:05.319
In Germany, it's perfectly legit to have been at lunch
43
00:02:05.439 --> 00:02:08.400
during work days and legita required.
44
00:02:10.400 --> 00:02:14.400
Yes, there is something to that, and it's really good beer.
45
00:02:15.159 --> 00:02:16.000
Uh.
46
00:02:16.080 --> 00:02:18.759
When I worked at Wix, let's just say. Let's just
47
00:02:18.800 --> 00:02:23.039
say that we worked hard and played hard and sometimes
48
00:02:23.120 --> 00:02:27.439
played hard during working hours.
49
00:02:28.479 --> 00:02:33.680
Alrighty, So with that we will transition into the topic
50
00:02:33.759 --> 00:02:38.199
juju early topic of the day, which is JavaScript testing
51
00:02:38.240 --> 00:02:42.159
and what's going on in testing in the JavaScript world,
52
00:02:42.360 --> 00:02:44.520
so you on need to take it away.
53
00:02:44.879 --> 00:02:45.879
Where do we want to start?
54
00:02:47.639 --> 00:02:49.400
I'm sure you wanted to start by sharing a few
55
00:02:49.400 --> 00:02:50.840
words about myself and.
56
00:02:50.680 --> 00:02:53.479
My Now we don't care about that. We just want
57
00:02:53.520 --> 00:02:55.120
to know what you have to say.
58
00:02:56.159 --> 00:02:56.360
Now.
59
00:02:56.439 --> 00:02:59.280
I just yeah, seriously, sorry, I'm a little off track.
60
00:02:59.319 --> 00:03:01.840
Tell us who you are wire famous and where people
61
00:03:01.919 --> 00:03:04.319
can give you money or will say that for the end,
62
00:03:04.360 --> 00:03:08.319
just tell us who are true things? Surprise, surprise.
63
00:03:08.360 --> 00:03:12.840
I'm a developer, freelancer and a consultant doing full stack
64
00:03:13.240 --> 00:03:17.159
a lot of JavaScript, but more than anything else, really
65
00:03:17.199 --> 00:03:19.879
my love, my passion. The things that I'm trying to
66
00:03:19.960 --> 00:03:24.639
specialize in is that all developer workflow and specifically testing.
67
00:03:25.319 --> 00:03:29.360
I think that I've worked with more than forty organizations.
68
00:03:29.719 --> 00:03:32.199
Some of them are big ney you're probably recognize on
69
00:03:32.280 --> 00:03:35.879
improving their testing. So I literally said, over the shoulders
70
00:03:35.919 --> 00:03:39.280
of like hundreds of developers, as we try to craft
71
00:03:39.360 --> 00:03:42.800
tested are both efficient and yet simple. Done a lot
72
00:03:42.800 --> 00:03:46.280
of mistakes, learned, a lot, tried a lot of new frameworks,
73
00:03:46.800 --> 00:03:49.520
and I hope to share some of these lessons here today.
74
00:03:50.479 --> 00:03:54.840
Isn't the process to your basically cursor right tests?
75
00:03:54.840 --> 00:03:55.199
For me.
76
00:03:57.439 --> 00:04:00.840
Exactly exactly this actually we're about to cover. I would
77
00:04:00.960 --> 00:04:03.919
I would also cover some kind of testing AI trends,
78
00:04:05.280 --> 00:04:08.960
mostly the useful parts of AI. So yeah, okay, just
79
00:04:09.000 --> 00:04:09.639
to just to.
80
00:04:11.520 --> 00:04:15.639
Finish that particular point. The scary thing is afterwards, when
81
00:04:15.680 --> 00:04:18.720
you make changes and then the tests break, you tell it,
82
00:04:19.319 --> 00:04:22.199
Cursor fix the test for me, and you're not sure
83
00:04:22.240 --> 00:04:25.920
whether it's fixing your test to hide the bugs or
84
00:04:25.959 --> 00:04:29.120
fixing your code to fix the tests, or doing both
85
00:04:29.360 --> 00:04:33.199
or whatever it's It can be really interesting.
86
00:04:33.279 --> 00:04:36.959
That's this way, definitely. I mean, I think that one
87
00:04:36.959 --> 00:04:39.160
of the first things one should tune in Cursor or
88
00:04:39.199 --> 00:04:41.759
cloud code or whatever you're working is that you can
89
00:04:41.920 --> 00:04:44.040
let it work in yellow mode. But I think there
90
00:04:44.079 --> 00:04:48.759
is one exception any a human A human confirmation should
91
00:04:48.759 --> 00:04:51.759
be made anytime a test is changed. You may change
92
00:04:51.759 --> 00:04:54.399
my production code, I may inspect each and every nine
93
00:04:54.480 --> 00:04:57.000
or not, I don't know, but a test changed should
94
00:04:57.040 --> 00:04:58.839
supervise that that's.
95
00:04:58.720 --> 00:05:01.040
My That's a good take. I think.
96
00:05:01.839 --> 00:05:03.279
I can't think of any time I'm just going to
97
00:05:03.319 --> 00:05:05.720
say here writing some code and then committed or do
98
00:05:05.839 --> 00:05:08.839
whatever without looking over it and making sure it works.
99
00:05:09.279 --> 00:05:09.560
Well.
100
00:05:10.000 --> 00:05:14.600
Yeah, that's easy to say, and everybody starts with good intentions,
101
00:05:14.639 --> 00:05:18.720
but at a certain point in time, you know, especially
102
00:05:18.720 --> 00:05:21.079
when you're under the gun and certain things and things
103
00:05:22.680 --> 00:05:25.240
seem to work and you build a certain level of trust.
104
00:05:26.160 --> 00:05:30.480
I've known people to almost even engineers, almost get to
105
00:05:30.519 --> 00:05:33.759
the point of almost vibe coding their way through projects.
106
00:05:34.360 --> 00:05:40.240
And I think you Only's statement that if you've got
107
00:05:40.319 --> 00:05:46.199
good test coverage, then ensuring that the tests are properly
108
00:05:46.439 --> 00:05:51.360
maintained and reviewed is a good way to add a
109
00:05:51.399 --> 00:05:55.639
significant layer of trust and security around your vibe coding.
110
00:05:59.720 --> 00:06:03.079
Yeah, I agree with both. I think that people have
111
00:06:03.360 --> 00:06:07.680
various degrees of how much they supervise AI generated God,
112
00:06:09.199 --> 00:06:12.600
I'm just saying that the least conservative one can be
113
00:06:12.800 --> 00:06:15.000
is at least guard you're testing. You might do this
114
00:06:15.079 --> 00:06:19.319
also for the production cod but I think that letting
115
00:06:19.360 --> 00:06:22.040
AI also change tests has to change God. This is
116
00:06:22.040 --> 00:06:26.240
the highest amount of frizk one may take.
117
00:06:27.839 --> 00:06:32.199
Yes, Okay, then, so what are the latest and greatest
118
00:06:32.240 --> 00:06:35.839
trends when it comes to tests, aside from having AI
119
00:06:36.120 --> 00:06:38.000
do all the coding for us?
120
00:06:39.279 --> 00:06:44.000
Yeah, so I'll kind of mention some kind of super
121
00:06:44.120 --> 00:06:48.040
trends and are very impactful changes, and then we can
122
00:06:48.879 --> 00:06:52.040
dig dive deeper into each one of them, at least
123
00:06:52.079 --> 00:06:54.920
the ones that we found interesting. So I think there
124
00:06:54.959 --> 00:06:58.920
is always in testing one kind of everlasting progress in
125
00:06:58.959 --> 00:07:02.800
which we always we try to get our tests to
126
00:07:02.839 --> 00:07:07.360
be more production like without paying the price of So
127
00:07:08.759 --> 00:07:12.519
in the past two years we keep seeing this progress
128
00:07:13.120 --> 00:07:15.600
and in this in front and the front and world.
129
00:07:15.920 --> 00:07:18.079
For the first time, I think like we ad React
130
00:07:18.120 --> 00:07:20.720
testing library ruling this world for a lot of time
131
00:07:21.120 --> 00:07:23.360
tests that runs in a jazz things that's for the
132
00:07:23.399 --> 00:07:27.639
first time it's safe to say that tests don't run
133
00:07:27.680 --> 00:07:32.839
in the browser, that are not visual are almost out
134
00:07:32.879 --> 00:07:37.279
of scope. There are many new ways and frameworks to
135
00:07:37.399 --> 00:07:39.759
run your tests in the browser and get the full
136
00:07:39.920 --> 00:07:44.839
visual experience and the confidence, so it's more like production.
137
00:07:44.920 --> 00:07:47.600
But some of them run really fast, so you're not
138
00:07:47.720 --> 00:07:51.199
paying the price of like full full blown end to
139
00:07:51.360 --> 00:07:54.600
end framework on the back inside it means that you
140
00:07:54.600 --> 00:07:56.720
having a micro service, you can more easily now run
141
00:07:56.759 --> 00:08:00.000
it on your computer with all the layers and almost
142
00:08:00.079 --> 00:08:04.560
light production infrastructure, and again without paying a tremendous price.
143
00:08:05.360 --> 00:08:09.920
I think these are bold and the changes that we're seeing,
144
00:08:09.920 --> 00:08:12.279
and we I guess we will dig deeper into the
145
00:08:12.319 --> 00:08:13.600
specific tooling.
146
00:08:13.959 --> 00:08:19.720
And just to clarify, there are obviously layers to tests,
147
00:08:19.800 --> 00:08:23.720
so there I don't know if you should refer to
148
00:08:23.759 --> 00:08:27.480
them as higher or lower layers. It's just different positions
149
00:08:27.480 --> 00:08:30.759
in the testing cycle. So you've got obviously stuff like
150
00:08:30.920 --> 00:08:36.320
unit tests, which should be very relatively small, relatively straightforward,
151
00:08:36.360 --> 00:08:40.440
and certainly very very fast, and they usually don't need
152
00:08:40.919 --> 00:08:45.679
special wiring, although you know a certain degree or a
153
00:08:45.679 --> 00:08:48.960
certain amount of mocking is often involved. Then you've got
154
00:08:49.039 --> 00:08:52.720
integration tests, and you've got into end tests. It seemed
155
00:08:52.720 --> 00:08:55.480
to me that when you were talking about the improvements
156
00:08:55.519 --> 00:09:02.399
in our ability to do effective, efficient and performance testing,
157
00:09:02.480 --> 00:09:05.919
you were mostly talking about integration tests and end to
158
00:09:06.120 --> 00:09:08.759
end tests and less about unit tests.
159
00:09:08.799 --> 00:09:14.200
Correct yes and a little no. I mean, I think
160
00:09:14.200 --> 00:09:17.000
at first there is a trend. I'm not advocating, by
161
00:09:17.000 --> 00:09:19.759
the way, on anything. I'm just trying to cover what
162
00:09:20.039 --> 00:09:25.320
like trends that I observed. And I think another trend
163
00:09:25.440 --> 00:09:28.360
is seeing less unit tests and more kind of component
164
00:09:28.480 --> 00:09:32.039
page integration tests. But so called the testing diamond. It's
165
00:09:32.039 --> 00:09:34.840
a kind of an emerging concept where you should write
166
00:09:34.919 --> 00:09:39.320
more integration test but with the new tooling, even if
167
00:09:39.360 --> 00:09:42.039
you write unit tests like take a framework that we
168
00:09:42.080 --> 00:09:44.679
will cover with that. It's called VITAS browser mode. Even
169
00:09:44.759 --> 00:09:48.200
your unit tests run in the browser. Now think about it.
170
00:09:48.320 --> 00:09:50.000
Not all the unit tests in the world are pure.
171
00:09:50.240 --> 00:09:53.039
You might test the models that actually doing some stuff
172
00:09:53.080 --> 00:09:56.360
with the web APIs. So instead of running it in
173
00:09:56.559 --> 00:10:00.000
no JS with polyphils and trust and trust the similarity,
174
00:10:00.679 --> 00:10:03.080
it runs your test in a real browser and Chromium
175
00:10:03.159 --> 00:10:06.200
for example. So even your unit test gains more kind
176
00:10:06.240 --> 00:10:09.919
of production like experience. Does this make sense?
177
00:10:10.840 --> 00:10:14.240
Yes, it does, although it's I need to.
178
00:10:16.240 --> 00:10:20.440
Think about what I think about it. It's kind of
179
00:10:20.480 --> 00:10:27.000
a recursive statement because really, when I'm thinking about unit tests,
180
00:10:27.679 --> 00:10:31.039
I'm really trying in most cases to be as stateless
181
00:10:31.039 --> 00:10:37.120
as possible and consequently not be dependent on having access
182
00:10:37.320 --> 00:10:41.399
to a browser dome, which is kind of like persistent
183
00:10:41.559 --> 00:10:43.159
by the definition.
184
00:10:45.200 --> 00:10:47.840
And so I'm.
185
00:10:47.639 --> 00:10:52.000
Not sure that this is where I want my unit
186
00:10:52.039 --> 00:10:58.080
tests to go necessarily, but I certainly appreciate the fact
187
00:10:58.159 --> 00:11:03.960
that we don't need to mock browser a p I
188
00:11:04.200 --> 00:11:09.240
S domin p I S, which is always let's say,
189
00:11:09.600 --> 00:11:14.320
less than ideal. But why do you think that is
190
00:11:14.720 --> 00:11:18.440
what has changed that has made this What has changed
191
00:11:18.879 --> 00:11:20.559
that has made this possible?
192
00:11:20.679 --> 00:11:22.799
Now? That was not available before.
193
00:11:24.600 --> 00:11:27.799
Yeah, first forced margining that I fully agree. The real
194
00:11:27.879 --> 00:11:31.399
value is in is in integration tests and about page
195
00:11:31.399 --> 00:11:33.840
testing or component tests, also in end to end. In
196
00:11:34.399 --> 00:11:42.120
unit test, it's indeed less significant. So it's mostly if
197
00:11:42.279 --> 00:11:46.240
if we if we double double click into the front
198
00:11:46.320 --> 00:11:48.919
end word for a start. I think now there is
199
00:11:48.960 --> 00:11:52.639
a key up from decisions the team should make. There
200
00:11:52.679 --> 00:11:58.000
is kind of a triangle decision between three different frameworks
201
00:11:58.000 --> 00:12:03.360
for testing components and pages, and one of them is
202
00:12:03.840 --> 00:12:07.440
relatively One of them is is new. It's for the
203
00:12:07.480 --> 00:12:11.279
past two years, and it makes running visual test in
204
00:12:11.320 --> 00:12:14.600
the browser almost as fast as unit test. So there
205
00:12:14.639 --> 00:12:16.879
was a lot of people trying to avoid that kind
206
00:12:16.879 --> 00:12:20.440
of visual experience because of the hassle involved. But with
207
00:12:20.559 --> 00:12:23.360
some of the new frameworks like vitest browser mode and
208
00:12:23.639 --> 00:12:29.120
the new storybook model, it becomes easier faster, so you
209
00:12:29.159 --> 00:12:32.559
have less reasons not to just hey, let's test it
210
00:12:32.639 --> 00:12:36.720
like where the user lives. And maybe it's worth at
211
00:12:36.720 --> 00:12:40.399
some point explaining what is vitest browser mode, how does
212
00:12:40.440 --> 00:12:43.840
it stack with playwright go for it?
213
00:12:45.399 --> 00:12:45.559
Now?
214
00:12:45.639 --> 00:12:48.720
Is that the same as v test UI this browser mode?
215
00:12:50.360 --> 00:12:54.840
Ah, not exactly, because you can you can also have
216
00:12:54.879 --> 00:12:58.759
vites I with unit tests and browser mode is a
217
00:12:58.799 --> 00:13:03.360
new sub frameworks sort VI test and to zoom out
218
00:13:03.360 --> 00:13:05.559
for a second, I think that we have now three
219
00:13:05.960 --> 00:13:09.600
measure and good options in the front and word that
220
00:13:09.679 --> 00:13:12.240
one must make a decision up front. The first one
221
00:13:12.320 --> 00:13:18.840
is Playwright, the good all known Playwright, very reputable, very
222
00:13:18.879 --> 00:13:23.200
reputable frameworks that is being very popular, which allows you
223
00:13:23.240 --> 00:13:25.440
to test your system as is like you can spin
224
00:13:25.559 --> 00:13:28.120
up a whole page and you get a real page
225
00:13:28.759 --> 00:13:34.039
in a browser, but you get the all the you
226
00:13:34.120 --> 00:13:37.840
get the entire you get d their application with the
227
00:13:37.919 --> 00:13:40.759
router and the state and a lot of things beyond
228
00:13:40.840 --> 00:13:43.200
the page level or the component what you want to test.
229
00:13:43.600 --> 00:13:47.759
So the performance is like a little of concerning Playwright
230
00:13:47.840 --> 00:13:50.000
even when you mock the back.
231
00:13:50.840 --> 00:13:56.759
Correct me if I'm wrong. Playwright is essentially just a
232
00:13:56.840 --> 00:14:04.159
headless Chrome with api is to automate various tasks. That's
233
00:14:04.240 --> 00:14:05.519
more less it right.
234
00:14:07.200 --> 00:14:09.480
Yes, and maybe a little more because it can run
235
00:14:09.519 --> 00:14:11.960
your test in other browsers as well, like run the
236
00:14:12.000 --> 00:14:14.480
test once and get it running in Chrome or Safari
237
00:14:14.639 --> 00:14:15.440
or anything.
238
00:14:15.639 --> 00:14:18.919
Yeah, because they basically later on added support for the
239
00:14:18.960 --> 00:14:20.799
same APIs as I.
240
00:14:20.720 --> 00:14:24.519
Recall exactly they imported kind of a CDP protocol for
241
00:14:24.600 --> 00:14:28.320
other browsers exactly. But on top of this, they also
242
00:14:28.360 --> 00:14:33.679
have a very nice and stable API to make the
243
00:14:33.759 --> 00:14:38.519
experience of interacting with pages less flaky. So for example,
244
00:14:38.559 --> 00:14:40.879
if you're trying to assert on something, they will take
245
00:14:40.879 --> 00:14:45.320
care to retry in wait and until that condition is
246
00:14:45.360 --> 00:14:47.919
met on the page. So the overall testing experience is
247
00:14:48.759 --> 00:14:49.519
really great, and.
248
00:14:50.879 --> 00:14:55.960
It replaced all the frameworks like Cyprus and Selenium.
249
00:14:55.440 --> 00:14:59.039
Right exactly exactly. It is.
250
00:14:59.600 --> 00:15:03.320
Okay, So how is v test browser mode different from Playwright.
251
00:15:05.279 --> 00:15:08.000
Yeah, so first worldt mentioning mentioning is that the viittest
252
00:15:08.000 --> 00:15:12.279
browser mode is using Playwright under the hood, so you
253
00:15:12.440 --> 00:15:17.240
get a battle tested engine that you can trust. But
254
00:15:17.320 --> 00:15:19.320
what they try to do Playwright is a little bit
255
00:15:19.360 --> 00:15:23.799
disruptive to front and testing. So if you take traditional
256
00:15:23.840 --> 00:15:28.200
front and testing with people used to work with things
257
00:15:28.240 --> 00:15:31.519
like React Testing Library, the syntax of React Testing library,
258
00:15:31.879 --> 00:15:34.960
the concept of React Testing library supporting from there to
259
00:15:35.039 --> 00:15:41.159
Playwright was a little disruptive. Vitus Browser mode allows you
260
00:15:41.200 --> 00:15:44.720
to take your existing test or learn nothing new, use
261
00:15:44.759 --> 00:15:48.240
the same syntax, same concept, only instead of running it
262
00:15:48.279 --> 00:15:51.320
in nogs like React Testing Library, you run it in
263
00:15:51.480 --> 00:15:57.240
real browser, which obviously allows you to see. You probably
264
00:15:57.279 --> 00:16:00.559
know that that really nasty experience when you with React
265
00:16:00.559 --> 00:16:03.360
testing library and the test failed. What you see on
266
00:16:03.399 --> 00:16:07.120
the screen is three hundred lines of XML of HTML
267
00:16:07.440 --> 00:16:09.919
in a CLI in which you try to understand in
268
00:16:10.000 --> 00:16:12.519
this my component, how does it look like? What's wrong here?
269
00:16:12.559 --> 00:16:16.799
I mean nobody wants to travel shoot test in HTML
270
00:16:16.879 --> 00:16:20.000
over CLI. With this browser mode, you can see them
271
00:16:20.000 --> 00:16:22.559
in a browser and benefits from all of the Chrome
272
00:16:22.600 --> 00:16:29.039
Developer tool bar perks. So that's that's the premise. That's
273
00:16:29.080 --> 00:16:33.799
the obvious upside. They also make it like you know,
274
00:16:33.879 --> 00:16:37.600
vitt stands for fast in French, with the word wit
275
00:16:37.759 --> 00:16:39.720
is fast in French, and they they stand up to
276
00:16:39.759 --> 00:16:43.240
their reputation like I can I already use it in production.
277
00:16:43.639 --> 00:16:47.519
It's amazing. I can like test then entire page, some interactions,
278
00:16:47.519 --> 00:16:51.120
sometimes specific scenario in two hundred milliseconds. It's like blazing fast.
279
00:16:51.120 --> 00:16:54.000
You click on a button and you don't have enough
280
00:16:54.159 --> 00:16:55.960
chance to see what happened on the screen. So the
281
00:16:56.000 --> 00:17:02.679
performance is really awesome. With that, there are some cons
282
00:17:02.720 --> 00:17:07.359
that should be mentioned as well. It's very new, it's
283
00:17:07.400 --> 00:17:12.039
still tagged as experimental for like two years or so,
284
00:17:14.839 --> 00:17:18.559
and so's it's it didn't yet stand the battle of
285
00:17:18.640 --> 00:17:22.599
time like Playwright, And there are many other other decisions
286
00:17:22.640 --> 00:17:25.119
that were made that are that might appeal to some
287
00:17:25.200 --> 00:17:28.559
teams and not to others. But it's definitely a bold
288
00:17:28.680 --> 00:17:33.039
opportunity I think now in twenty twenty five for testing
289
00:17:33.079 --> 00:17:35.960
and I could not finish that. I'm sorry. This is
290
00:17:36.039 --> 00:17:39.720
kind of a mini mini TED talk. But there is
291
00:17:39.759 --> 00:17:44.480
also Storybook, and Storybook now change their model and now
292
00:17:44.519 --> 00:17:48.519
they're using Vita's browser mode under the hood, which use Playwright.
293
00:17:49.160 --> 00:17:51.359
So you have three options. You know that Russian doll
294
00:17:51.440 --> 00:17:56.920
which is called I believe Matroyshka Babushka or matroys Career.
295
00:17:56.960 --> 00:17:59.680
It's kind of a nested doll. So what we have
296
00:17:59.799 --> 00:18:03.119
now the in the testing space, it's a troche car,
297
00:18:03.200 --> 00:18:05.359
so you have to play right or VI test brother
298
00:18:05.440 --> 00:18:07.559
mos that you just play right, or storybooks that use
299
00:18:08.720 --> 00:18:12.200
you got idea and you have to make you have
300
00:18:12.279 --> 00:18:13.279
to make the choice.
301
00:18:15.039 --> 00:18:20.599
So if you're starting a new front end project today,
302
00:18:20.680 --> 00:18:24.400
and let's say that you're using one of the two
303
00:18:24.480 --> 00:18:27.079
leading options, I don't know how familiar you are with them.
304
00:18:27.519 --> 00:18:31.720
Either React, which is approximately fifty percent of the market
305
00:18:32.480 --> 00:18:37.440
or view which is approximately twenty to thirty percent of
306
00:18:37.480 --> 00:18:41.720
the market, and you need to do and you want
307
00:18:41.759 --> 00:18:44.920
to do testing because we started with testing for the
308
00:18:44.960 --> 00:18:45.400
front end.
309
00:18:46.160 --> 00:18:47.759
What would you use?
310
00:18:51.240 --> 00:18:55.319
Yeah, that's that's first. Very I think that the choice
311
00:18:55.359 --> 00:18:57.400
is really hard today. I had to choose for a
312
00:18:57.440 --> 00:19:00.519
customer like two weeks ago, and the decision is really hard.
313
00:19:01.039 --> 00:19:04.880
So from one hand, Eileen Tower being conservative and say, hey,
314
00:19:04.920 --> 00:19:09.039
let's use play right. It works. It's like the most simple,
315
00:19:10.119 --> 00:19:13.240
it's like the last layer. It's simple, it's proven, let's
316
00:19:13.279 --> 00:19:14.920
just use it again.
317
00:19:15.119 --> 00:19:18.519
Playwright in this case is for mostly for the end
318
00:19:18.519 --> 00:19:19.599
to end tests.
319
00:19:19.359 --> 00:19:23.160
Right, or you can use it also for components, for
320
00:19:23.160 --> 00:19:26.359
for component or page testing. It it has great tooling
321
00:19:26.440 --> 00:19:28.599
to allow you to mock your bacond and I think
322
00:19:29.039 --> 00:19:32.400
I think that for the majority of the testing front
323
00:19:32.480 --> 00:19:36.599
and team want to mock the beacon. But yeah, you
324
00:19:36.640 --> 00:19:37.359
can also run in.
325
00:19:37.559 --> 00:19:40.960
Well, let's put it this way, if you're not mocking
326
00:19:41.039 --> 00:19:46.960
your back end, then you're by definition dependent. It's almost
327
00:19:47.000 --> 00:19:49.640
it kind of almost means that by definition your tests
328
00:19:49.640 --> 00:19:51.599
are going to be flaky at least some of the time.
329
00:19:52.920 --> 00:19:55.440
If you do involve if you do involve the back
330
00:19:57.640 --> 00:19:59.799
so so yes, if you want to go for a
331
00:20:00.000 --> 00:20:05.960
operations of concerns, you want to mock external services reduce
332
00:20:06.000 --> 00:20:09.039
the amount of dependencies that you have on external services
333
00:20:09.079 --> 00:20:10.599
and that includes your own back end.
334
00:20:12.920 --> 00:20:17.960
Totally agree, it's slower, it's more flaky. But my highest
335
00:20:17.960 --> 00:20:21.240
concern with end to end is actually that it's really
336
00:20:21.279 --> 00:20:25.279
hard to simulate things beyond heavy passes. So think about it.
337
00:20:25.359 --> 00:20:27.720
You want to test now that the user is disabled,
338
00:20:27.759 --> 00:20:31.200
it's a premium user. You get twenty records, all of
339
00:20:31.240 --> 00:20:34.559
the things that are beyond the simplest response. With real
340
00:20:34.599 --> 00:20:36.960
back end, it might be very hard to tune the
341
00:20:37.000 --> 00:20:39.839
backet into each and every scenarios that you want. However,
342
00:20:39.880 --> 00:20:42.279
when you mock it, you can simply just say, hey,
343
00:20:42.319 --> 00:20:44.200
I want this response, I want that response.
344
00:20:44.759 --> 00:20:45.000
Yeah.
345
00:20:45.079 --> 00:20:47.640
On the other hand, when you're mocking it, you're basically
346
00:20:47.680 --> 00:20:52.799
inventing the responses, so it's not necessarily how the system behaves.
347
00:20:52.839 --> 00:20:55.759
It's not really end to end if it's mocking.
348
00:20:56.400 --> 00:20:59.279
I mean when you normally like spin up like a
349
00:20:59.559 --> 00:21:02.720
maybe this is what you're talking about with mocking, but
350
00:21:02.839 --> 00:21:07.559
basically create a test database or a copy of your database,
351
00:21:07.599 --> 00:21:10.400
spin one upset it. You know, you got your data
352
00:21:10.440 --> 00:21:12.880
in it and then run your tests. Are actually using
353
00:21:12.920 --> 00:21:15.119
a database. It's not your Pride database obviously, but at
354
00:21:15.200 --> 00:21:17.160
least you have a clone and a real database that
355
00:21:17.200 --> 00:21:20.240
you're interacting with, because I know, speaking personal experience, a
356
00:21:20.279 --> 00:21:23.160
lot of times that's that's valuable testing.
357
00:21:23.960 --> 00:21:28.960
Yeah, yes, but I think that first before answering that
358
00:21:29.039 --> 00:21:31.319
I'd like to repeat to a dance. And I think
359
00:21:31.359 --> 00:21:35.400
it's like a key point here. If you're doing naive mocking, Okay,
360
00:21:35.440 --> 00:21:38.559
I'm just walking some Jason. Yeah, you are now at
361
00:21:38.599 --> 00:21:41.559
the risk of doing things that are not exactly the
362
00:21:41.559 --> 00:21:44.119
same in production, and this is why you also need
363
00:21:44.200 --> 00:21:47.039
contract testing. And I think this is definitely a topic
364
00:21:47.039 --> 00:21:49.279
we want to double click into later because there are
365
00:21:49.279 --> 00:21:53.119
a new interesting tooling in the JavaScript space related exactly
366
00:21:53.160 --> 00:21:57.400
with this problem. How do you create smart and aligned mogs?
367
00:21:58.000 --> 00:22:02.200
Now to this one. Yeah, in a simple case, it's
368
00:22:02.279 --> 00:22:04.559
just crawd applications, So you set up something in the
369
00:22:04.640 --> 00:22:08.640
database and you display them. But in many cases, creating
370
00:22:08.720 --> 00:22:11.480
the server response is much harder than this. Let's give
371
00:22:11.519 --> 00:22:15.119
you an example. One of my customers, is there a
372
00:22:15.599 --> 00:22:18.599
user entity and it might get disabled? How does it
373
00:22:18.640 --> 00:22:21.799
get disabled, there is some chron job running every minute
374
00:22:21.839 --> 00:22:24.799
and concluding that the user is disabled. So how would
375
00:22:24.920 --> 00:22:28.519
you create a disabled user from a test and make
376
00:22:28.559 --> 00:22:32.400
the back end run a chron job and tag it.
377
00:22:32.000 --> 00:22:36.839
It becomes much so much harder than just returning one line. Hey,
378
00:22:37.720 --> 00:22:41.319
just give me a disabled user, Jason, am I being
379
00:22:41.680 --> 00:22:42.480
clear on this point.
380
00:22:42.640 --> 00:22:47.799
No, it's It's obviously true like getting an application to
381
00:22:47.960 --> 00:22:53.519
do what you exactly what you want and can be challenging,
382
00:22:54.759 --> 00:23:00.960
and configuring a certain backend application to replicate certain scenarios
383
00:23:01.200 --> 00:23:05.319
is a problem. Just think about, you know, the scenario
384
00:23:05.359 --> 00:23:09.160
where a customer tells you that they're experiencing a certain problem,
385
00:23:09.240 --> 00:23:12.759
and maybe you even have traces that kind of show
386
00:23:12.799 --> 00:23:17.119
the problem the way that the customer is experiencing it,
387
00:23:17.240 --> 00:23:19.799
and now you're trying to replicate that scenario in the
388
00:23:19.880 --> 00:23:22.400
lab in order to debug it. We all know how
389
00:23:22.519 --> 00:23:28.079
challenging that can be. So I totally agree that replicating
390
00:23:29.599 --> 00:23:34.960
weird behaviors or edge certain edge behaviors that are legitimate
391
00:23:35.039 --> 00:23:40.079
and you want to test for in an actual environment
392
00:23:40.160 --> 00:23:44.119
that simulates production can be challenging, and it's often much
393
00:23:44.160 --> 00:23:50.480
easier to just play recordings of back end responses and
394
00:23:51.400 --> 00:23:54.839
you know, effectively mocking the back end it can be
395
00:23:55.039 --> 00:23:59.079
much much easier. Likewise, also mocking the front end.
396
00:23:59.160 --> 00:24:01.079
You know you want to.
397
00:24:02.599 --> 00:24:05.440
Test the back end, says system, then you might use
398
00:24:05.480 --> 00:24:07.880
some sort of I don't know, just post things out
399
00:24:07.920 --> 00:24:12.920
of Postman or whatever or playwright rather than actually having
400
00:24:12.960 --> 00:24:18.680
an actual front end that you know you create a
401
00:24:18.799 --> 00:24:21.799
process that clicks through. It can be much more challenging
402
00:24:21.880 --> 00:24:25.839
and more complicated than length fear.
403
00:24:28.240 --> 00:24:30.839
So is there a point, you know, Joanni, with that
404
00:24:30.920 --> 00:24:35.960
example you gave with the Crown job, is there a
405
00:24:35.960 --> 00:24:39.680
point where you have to say, Okay, we can't test
406
00:24:39.920 --> 00:24:42.519
everything in an automated test and maybe we have to
407
00:24:42.559 --> 00:24:45.880
do something manually or do you always work under the
408
00:24:45.880 --> 00:24:49.680
assumption that everything in Proude can be handled with an
409
00:24:49.680 --> 00:24:50.480
automated test.
410
00:24:52.799 --> 00:24:56.480
Yeah, so I'd say, based on my experience, as long
411
00:24:56.480 --> 00:25:00.160
as you stick to gray box testing. When I say
412
00:25:00.200 --> 00:25:03.119
gray box testing, it's a test that you don't test
413
00:25:03.119 --> 00:25:06.960
the entire system. You test one component. You test like
414
00:25:07.039 --> 00:25:09.640
the user like you just put things in and expect
415
00:25:09.640 --> 00:25:12.400
some outcomes. But you also run on the same process
416
00:25:12.759 --> 00:25:17.400
like that application. You can simulate anything I didn't encounter
417
00:25:17.480 --> 00:25:20.960
anything that is not simulated that it's hard to simulate
418
00:25:21.319 --> 00:25:24.680
as long as you are your test and the col
419
00:25:24.720 --> 00:25:29.559
under testing within the same process. In terms of ROI yeah,
420
00:25:29.599 --> 00:25:34.759
I would definitely prioritize and focus on first on the
421
00:25:34.759 --> 00:25:36.640
things that the user does and the things that are
422
00:25:36.799 --> 00:25:41.359
likely to happen and where the bugs are more severe.
423
00:25:42.160 --> 00:25:47.039
But in terms of the technical capability of simulating stuff,
424
00:25:47.160 --> 00:25:49.640
I think that today's in the jobs critical system, I
425
00:25:49.839 --> 00:25:53.440
rarely encounter the situation that something was not reproducible.
426
00:25:53.720 --> 00:25:57.759
And I'll add to that that manual testing is kind
427
00:25:57.799 --> 00:26:02.480
of like by definition anecdotal, and I hate to depend
428
00:26:02.559 --> 00:26:09.920
on anecdotal evidence of correctness. And so even if if
429
00:26:10.119 --> 00:26:13.880
if you do a certain amount of manual testing, then
430
00:26:13.960 --> 00:26:17.480
record that testing and then turn it into an automated script,
431
00:26:18.200 --> 00:26:24.519
it doesn't need to stay manual forever. So a certain
432
00:26:24.559 --> 00:26:28.799
amount of manual testing, especially for I don't know complex systems,
433
00:26:28.960 --> 00:26:33.880
might be required, but again try to automate it going
434
00:26:33.920 --> 00:26:34.680
down the line.
435
00:26:36.200 --> 00:26:39.279
Definitely agree. I also have if someone is interested in articles,
436
00:26:39.279 --> 00:26:41.519
that's called testing the dark scenario of your back end,
437
00:26:41.559 --> 00:26:44.200
in which I've shown in an OGS application how each
438
00:26:44.599 --> 00:26:47.440
in seven lines tests no test there was more than
439
00:26:47.480 --> 00:26:52.640
seven lines. You can test really critical scenarios. It usually
440
00:26:52.720 --> 00:26:56.799
teams don't test, like, for example, test that when your bootstrap,
441
00:26:56.880 --> 00:27:02.440
your your beacon bootstop phase fails you then you get
442
00:27:02.440 --> 00:27:05.839
the right monitoring or test that when your live liveliness
443
00:27:06.400 --> 00:27:10.400
route is not behaving correctly, you get the right thing.
444
00:27:10.480 --> 00:27:12.960
Or when you know, very common in JavaScript word, when
445
00:27:12.960 --> 00:27:15.480
there is a process uncut exception or one hundred that
446
00:27:15.599 --> 00:27:19.119
rejection is happening in your application, do you do the
447
00:27:19.200 --> 00:27:21.680
right thing? This is a critical event. I mean your
448
00:27:21.720 --> 00:27:24.640
process might just crush. Do you test this? I showed
449
00:27:24.640 --> 00:27:26.680
out like in five lines of code you can also
450
00:27:26.759 --> 00:27:30.480
cover these advanced or endling passes. So yeah, I think that.
451
00:27:32.240 --> 00:27:35.160
So again to summarize what we've talked about so far,
452
00:27:35.240 --> 00:27:37.960
So basically what you're saying is that you're kind of
453
00:27:38.000 --> 00:27:46.559
promoting more sophisticated tests, usually component level tests or integration tests,
454
00:27:47.279 --> 00:27:53.039
less unit tests, and that using modern tooling that you
455
00:27:53.200 --> 00:27:58.279
mentioned like Playwright, like v test, browser mode, and like
456
00:28:00.680 --> 00:28:04.599
the storybook testing, it's much easier to do than it
457
00:28:04.720 --> 00:28:07.079
used to be, is basically what you're saying. If I
458
00:28:07.119 --> 00:28:08.519
understand correctly.
459
00:28:09.519 --> 00:28:12.720
Yeah, much easier than before. And i'd say, I mean
460
00:28:12.720 --> 00:28:15.279
I'm not against unit tests, but I think that for
461
00:28:15.400 --> 00:28:18.359
a start, your first step should be what the user
462
00:28:18.440 --> 00:28:21.839
is doing. And the user is not stretching functions or models, right,
463
00:28:21.839 --> 00:28:24.200
The user is visiting a page. So I think that
464
00:28:24.240 --> 00:28:27.599
the first things that you should test is your page,
465
00:28:28.119 --> 00:28:30.599
how the user interact, and then, as you call it,
466
00:28:30.680 --> 00:28:35.680
if you realize some complex, highly complex logics that you
467
00:28:35.720 --> 00:28:40.680
want to isolate, testing isolation for better whatever performance, then yeah,
468
00:28:40.799 --> 00:28:42.799
go for unit tests. But I think this is kind
469
00:28:42.799 --> 00:28:46.240
of contextual and optional.
470
00:28:46.720 --> 00:28:50.119
Well, to be fair, unit tests actually were kind of
471
00:28:50.160 --> 00:28:56.799
born out of TDD, So initially the concept of unit tests,
472
00:28:57.759 --> 00:29:02.759
at least for people who take that approach, is basically,
473
00:29:02.880 --> 00:29:06.119
we first write the tests, then we write the code.
474
00:29:07.039 --> 00:29:12.359
Now AI writes the tests and the code, so that's
475
00:29:12.960 --> 00:29:17.079
less that approach to be Also, to be fair, I've
476
00:29:17.200 --> 00:29:20.920
never been able to adopt the TDD model. Maybe I'm
477
00:29:20.920 --> 00:29:23.400
too much set in my ways. It just doesn't seem
478
00:29:23.440 --> 00:29:25.880
to be the way that I think. But I have
479
00:29:26.759 --> 00:29:32.039
gotten value from unit tests. The main advantage of unit
480
00:29:32.039 --> 00:29:36.640
tests is they're very lightweight, so they're almost effectively free
481
00:29:36.680 --> 00:29:40.000
to run, and they run, and I have them run
482
00:29:40.079 --> 00:29:46.200
constantly because you know, I don't care and if something break,
483
00:29:46.680 --> 00:29:51.079
it's there. They test very specific things so you get
484
00:29:51.119 --> 00:29:55.799
an instantaneous indication of where that problem is, so you
485
00:29:55.880 --> 00:29:59.119
don't need to think about Like with and let's say,
486
00:29:59.440 --> 00:30:03.559
especially the end to end tests. You know the test fails, Okay,
487
00:30:03.960 --> 00:30:08.400
why did it fail? It often requires pretty significant amount
488
00:30:08.480 --> 00:30:11.880
of investigation in order to understand why in end to
489
00:30:12.079 --> 00:30:16.119
end tests failed. And like we talked about before, it
490
00:30:16.160 --> 00:30:19.799
could also be related to certain things that to you know,
491
00:30:19.920 --> 00:30:24.279
flaky tests because you're dependent on external services. With unit tests,
492
00:30:24.319 --> 00:30:27.119
you're not dependent on everything and anything. You're just testing
493
00:30:27.160 --> 00:30:30.200
your own code. It runs very quickly. You're testing a
494
00:30:30.279 --> 00:30:32.839
very specific function. That's say, and.
495
00:30:32.839 --> 00:30:35.640
You know I've introduced I made a change that function.
496
00:30:35.720 --> 00:30:36.759
That's why it breaks.
497
00:30:37.240 --> 00:30:41.319
Couldn't agree more. You know, I observed once something very
498
00:30:41.319 --> 00:30:44.119
interesting that can it back. For those of you are
499
00:30:44.119 --> 00:30:47.079
not familiar with, Can is kind of the father of DDD,
500
00:30:47.559 --> 00:30:50.480
A person who should really appreciate unit test and he
501
00:30:50.599 --> 00:30:53.680
said in one of his blog posts, unit tests give
502
00:30:53.759 --> 00:30:57.400
up predicting suitability to production and inspiring confidence. So it's
503
00:30:57.440 --> 00:31:00.559
not about product confidence at all. It's about being writeable,
504
00:31:00.720 --> 00:31:03.640
fast and specific. Now I think that most of the
505
00:31:03.640 --> 00:31:09.240
developers are looking in testing for confidence, and yes, it
506
00:31:09.279 --> 00:31:10.680
turns out it's not only about this.
507
00:31:11.880 --> 00:31:15.519
Yeah, there's that famous picture about the difference between unit
508
00:31:15.599 --> 00:31:19.440
tests and integration tests, about you know, building urinal in
509
00:31:19.519 --> 00:31:23.319
such a position that when you open the bathroom stall door.
510
00:31:23.200 --> 00:31:24.759
You can't actually get at it.
511
00:31:24.960 --> 00:31:28.519
So everything is fine at the unit level, at the
512
00:31:28.640 --> 00:31:31.359
unit level, but when you put it all together, it
513
00:31:31.400 --> 00:31:34.519
doesn't work. The only we've seem to have lost your picture.
514
00:31:34.599 --> 00:31:35.279
You're still with.
515
00:31:35.279 --> 00:31:38.279
Us, Yeah, I'm working on fixing the camera. The good
516
00:31:38.279 --> 00:31:41.759
news is that I'm still online. Yeah, okay, And I
517
00:31:41.759 --> 00:31:45.279
think it worth just revisiting for one last minute that
518
00:31:45.400 --> 00:31:51.759
that crucial decision between the three front end things. So
519
00:31:51.920 --> 00:31:57.240
we've mentioned playwright. I think that storybook and Viness browser
520
00:31:57.240 --> 00:32:00.640
mode are also great options. And we didn't story books
521
00:32:00.680 --> 00:32:04.119
or storybook if you have a design you asked me,
522
00:32:04.240 --> 00:32:07.039
what would I pick? So, if we're having a design
523
00:32:07.119 --> 00:32:10.000
system and rely a lot of small components that we
524
00:32:10.480 --> 00:32:14.599
tend to communicate and testing isolation, storybooks gives a very
525
00:32:14.599 --> 00:32:19.599
compelling package in which we get first a great workflow
526
00:32:20.039 --> 00:32:23.200
of visualizing our component and documenting them, but also a
527
00:32:23.240 --> 00:32:26.079
great new testing experience. So you write a test once
528
00:32:26.640 --> 00:32:29.759
and you get both accessibility testing. If you have some
529
00:32:29.799 --> 00:32:35.160
accessibility issue, you are covered visual regression testing without writing
530
00:32:35.240 --> 00:32:38.960
any other test out of the box, if you're if
531
00:32:39.000 --> 00:32:41.319
you are out of the box, and if you sharing
532
00:32:41.359 --> 00:32:47.759
your credit card, visual testing and also also functional testing
533
00:32:47.920 --> 00:32:50.799
using playwright under the hood. So you get where.
534
00:32:51.000 --> 00:32:55.799
Say visual testing just to verify that you understand your terminology.
535
00:32:56.279 --> 00:33:00.440
You basically mean that if I accidentally, I don't know,
536
00:33:00.680 --> 00:33:03.640
change the font or something, it will break my test
537
00:33:03.799 --> 00:33:08.240
because the content, the actual pixels on the screen are
538
00:33:08.279 --> 00:33:10.200
not the same as they're supposed to be.
539
00:33:11.200 --> 00:33:14.400
Exactly. It should be smarter than this, and it should
540
00:33:14.480 --> 00:33:21.440
realize and detect significant changes from like minor improvement, but
541
00:33:21.559 --> 00:33:24.279
exactly this. Since that functional tests can tell you about.
542
00:33:24.359 --> 00:33:30.599
So your side menu is crewed, something some I don't know,
543
00:33:31.359 --> 00:33:35.000
something is totally broken on the screen, a big change
544
00:33:35.039 --> 00:33:35.640
in the layout.
545
00:33:35.839 --> 00:33:39.039
You put a red background or something for testing purposes
546
00:33:39.079 --> 00:33:41.720
and then forgot to remove it.
547
00:33:42.559 --> 00:33:47.799
That's that's that's a nice example. Yeah, so storybook gives
548
00:33:47.799 --> 00:33:51.119
you all this together interesting option. On the other end,
549
00:33:51.519 --> 00:33:57.720
it's the more sophisticated option. It's it's all the storybook
550
00:33:58.000 --> 00:34:02.119
narratives and its browser modan hand play, right, So you
551
00:34:02.200 --> 00:34:04.799
have a lot of moving pieces to handle.
552
00:34:06.039 --> 00:34:08.599
Is it open source? Is it? Is it a paid
553
00:34:09.079 --> 00:34:10.760
product or service?
554
00:34:10.960 --> 00:34:12.239
What? What is it?
555
00:34:11.760 --> 00:34:15.719
It's open source, it's a free product, except I think
556
00:34:15.760 --> 00:34:18.639
this is how they make the revenues. Accept the visual
557
00:34:18.679 --> 00:34:23.559
regression test. What you just mentioned this one is is
558
00:34:23.559 --> 00:34:24.280
a paid service.
559
00:34:24.920 --> 00:34:25.639
Okay, cool?
560
00:34:26.159 --> 00:34:28.760
Hey, just to clarify terms. I know we've talked about
561
00:34:28.760 --> 00:34:32.880
different types of testing. We talked about unit testing, integration testing,
562
00:34:33.280 --> 00:34:35.400
and then testing visual testing.
563
00:34:36.239 --> 00:34:36.639
Uh.
564
00:34:37.159 --> 00:34:39.039
The one I always get hazy on is a difference
565
00:34:39.119 --> 00:34:42.760
between I guess what integration testing is how you find
566
00:34:42.760 --> 00:34:46.719
that versus unit tests versus and the s test? What
567
00:34:47.119 --> 00:34:50.199
excuse me, end to end testing? So what's what's your
568
00:34:50.239 --> 00:34:54.519
definition of integration testing? Yea, because without understanding that, it's
569
00:34:54.519 --> 00:34:59.000
hard to get Dan's toilet joke about uh integration testing.
570
00:34:59.280 --> 00:35:05.239
So yeah, testing terminology is totally broken. I think that
571
00:35:05.280 --> 00:35:08.599
if when you say end to end testing, or or
572
00:35:08.639 --> 00:35:12.599
even unit testing and integrationion testing, different people will understand
573
00:35:13.079 --> 00:35:17.239
different things. And for this reason, maya suggestion in every
574
00:35:17.280 --> 00:35:20.440
company in your company, name a very specific name for
575
00:35:20.480 --> 00:35:24.559
your test which is not integration and to take for example, Uh,
576
00:35:24.840 --> 00:35:26.719
just one example. In one of my customers, we used
577
00:35:26.719 --> 00:35:29.320
to test pages. We take a front and page we
578
00:35:29.440 --> 00:35:32.800
isolated and the test cope is every time one page.
579
00:35:32.920 --> 00:35:36.440
We call it page testing, and every everyone understands what
580
00:35:36.480 --> 00:35:39.159
page testing means. It's a test of a page. Integration
581
00:35:39.280 --> 00:35:42.880
tests just mean might mean three different things two different people.
582
00:35:44.039 --> 00:35:47.400
If we try somehow to clarify the integration test is.
583
00:35:48.920 --> 00:35:52.159
I think that the most common definition of it that
584
00:35:52.519 --> 00:35:55.880
we test the user journey in a thing. We test
585
00:35:55.880 --> 00:35:59.320
the wholes user journey in a single tier of the system,
586
00:35:59.400 --> 00:36:02.960
not and then tire system. So it might be the
587
00:36:03.039 --> 00:36:06.559
user visiting some front end but without stretching the entire system.
588
00:36:06.840 --> 00:36:09.519
It might be a user visit a call, coding some
589
00:36:09.719 --> 00:36:13.960
backend API and expecting your response. This is our examples
590
00:36:14.000 --> 00:36:15.719
for integration tests.
591
00:36:15.440 --> 00:36:18.559
So correct me if I'm wrong. It also includes, for example,
592
00:36:18.639 --> 00:36:24.000
testing components in isolation. So let's say I implemented a
593
00:36:24.159 --> 00:36:28.920
date picker component that the date range picking component, because
594
00:36:29.000 --> 00:36:31.239
you know you don't get that out of the box
595
00:36:31.280 --> 00:36:34.679
in the browser, and I want to verify that it's
596
00:36:34.719 --> 00:36:41.760
working correctly, and I can host it within like independently
597
00:36:41.960 --> 00:36:46.639
and basically simulate interactions with it. It's not end to
598
00:36:46.800 --> 00:36:49.079
end because it's not done in the context of the
599
00:36:49.199 --> 00:36:52.360
entire application. So it's not for example, if you think
600
00:36:52.400 --> 00:36:55.599
about I don't know, let's say some sort of hotel
601
00:36:55.679 --> 00:36:59.719
booking application, it's not done in the context of actually
602
00:37:00.039 --> 00:37:05.760
looking stay at a hotel. It's done in isolation, but
603
00:37:05.920 --> 00:37:11.360
it's still done at the level of actual user interface
604
00:37:11.360 --> 00:37:17.079
and interactions, not just invoking functions and checking return values.
605
00:37:18.800 --> 00:37:22.039
Exactly. You're in other words, you are testing the user
606
00:37:22.119 --> 00:37:26.039
experience with this in its almost real environment. There is
607
00:37:26.079 --> 00:37:29.440
a browser here, There is your code. There is like
608
00:37:29.480 --> 00:37:32.960
you have multiple layers here orchestrating together. You have that
609
00:37:34.400 --> 00:37:38.280
UI kid usually the UI kids that you're using, and
610
00:37:38.320 --> 00:37:41.079
then you have your code that trapping it, and you
611
00:37:41.159 --> 00:37:44.639
have us sometimes your state manager, and there is the browser.
612
00:37:45.039 --> 00:37:49.159
All the moving pieces that exist in production are playing
613
00:37:49.159 --> 00:37:52.039
together in this test. In this regard, it's integration.
614
00:37:52.400 --> 00:37:55.480
And if you're talking about non front and stuff, for
615
00:37:55.519 --> 00:37:59.639
example a node system, it might be testing a particular
616
00:38:00.039 --> 00:38:03.800
specific API call. Again not in the context of an
617
00:38:03.840 --> 00:38:08.199
actual complete user flow, and potentially even with certain other
618
00:38:08.320 --> 00:38:11.800
parts of the system being mocked like we said before,
619
00:38:11.880 --> 00:38:14.880
and not actually working with let's say you're working with
620
00:38:15.000 --> 00:38:19.480
various external services. You're not actually working with these services,
621
00:38:19.519 --> 00:38:24.480
you're mocking them, and you're just testing effectively your particular
622
00:38:25.039 --> 00:38:28.880
let's say micro service or particular aspect of a microservice
623
00:38:28.920 --> 00:38:29.639
in isolation.
624
00:38:30.920 --> 00:38:34.159
Yes, exactly what I observe is it works great for
625
00:38:34.280 --> 00:38:38.239
backing teams is testing your entire micro services is put
626
00:38:38.599 --> 00:38:41.920
the infrastructure on local doctor composed, test all the layer
627
00:38:41.960 --> 00:38:45.039
of the micro service. If you deploy it, you test it,
628
00:38:45.119 --> 00:38:49.239
your data access there, your business logic, your API, but
629
00:38:49.639 --> 00:38:54.840
only intercept calls to other services. Otherwise it becomes end
630
00:38:54.840 --> 00:38:57.599
to end with all the downsides that we have mentioned.
631
00:38:58.320 --> 00:39:02.480
Yeah, I'm currently actually doing exactly that, and the portrait
632
00:39:02.559 --> 00:39:07.000
that I'm working on. Being able to use a doctor
633
00:39:07.079 --> 00:39:14.559
composed to effectively recreate the entire system on your own
634
00:39:14.679 --> 00:39:20.719
particular on your local developer machine and then be able
635
00:39:20.840 --> 00:39:25.559
to dynamically deploy changes to the specific micro service that
636
00:39:25.599 --> 00:39:30.400
you're working on is effectively a godsend. It makes life
637
00:39:30.559 --> 00:39:34.920
so much easier when developing in such complex environments.
638
00:39:34.960 --> 00:39:38.199
Definitely, I mean's surprising how fast this test is. Like
639
00:39:38.559 --> 00:39:42.320
I think usually like a fifty proximately fifty test for
640
00:39:42.400 --> 00:39:46.920
micro service on a developer machine with real database last
641
00:39:47.000 --> 00:39:53.480
lesson twenty seconds. So yeah, it's a really efficient technique
642
00:39:53.480 --> 00:39:56.800
and maybe it's worth mentioning if we already discuss back end.
643
00:39:57.000 --> 00:40:00.400
There is now a build in test runners in o JS,
644
00:40:00.840 --> 00:40:04.119
mostly for back end, I think, and it's really gaining
645
00:40:04.119 --> 00:40:05.039
a lot of momentum.
646
00:40:06.079 --> 00:40:13.639
Yeah, it's it's interesting that it seems that between node
647
00:40:14.119 --> 00:40:18.079
implementing its own testing built in testing capabilities on the
648
00:40:18.119 --> 00:40:21.159
one hand, and v test on the other end, they're
649
00:40:21.239 --> 00:40:25.400
kind of cornering the testing market in a sense. And
650
00:40:25.559 --> 00:40:29.519
I recall, for example, seeing this post on x I
651
00:40:29.559 --> 00:40:34.800
think from KENC. Dodds talking about how happy is that
652
00:40:35.000 --> 00:40:38.800
the React testing library is no longer needed because effectively
653
00:40:38.840 --> 00:40:41.960
all this functionality is now built into v test.
654
00:40:42.039 --> 00:40:46.440
I think yes, And I think it's really interesting to
655
00:40:46.480 --> 00:40:54.719
see how people are contributing libraries and ideas together. So
656
00:40:54.960 --> 00:40:58.400
it reactesting Library was a library, but it was also
657
00:40:58.400 --> 00:41:01.480
a philosophy of how you test. Now the library is
658
00:41:02.039 --> 00:41:05.559
probably fading away or at least losing some momentum, but
659
00:41:05.920 --> 00:41:10.400
the concept that the concepts there, the techniques are being
660
00:41:10.920 --> 00:41:13.719
totally used in Vitas browser mode and in Playwright, and
661
00:41:13.800 --> 00:41:16.760
so many new testing framework are taking inspiration and building
662
00:41:16.760 --> 00:41:21.559
on this that It's very very interesting to see this
663
00:41:23.679 --> 00:41:25.119
evolving process.
664
00:41:27.960 --> 00:41:31.039
So in general, what would you say, is it still
665
00:41:31.320 --> 00:41:34.239
much more difficult to test the front end than the
666
00:41:34.320 --> 00:41:37.960
back end or are we achieving some sort of parity?
667
00:41:38.599 --> 00:41:43.079
M m yeah, I tell that I'd easily said that
668
00:41:43.239 --> 00:41:47.480
testing a back end is much much easier than testing
669
00:41:47.480 --> 00:41:55.199
a front end. Yeah. I mean, first, you can also
670
00:41:55.280 --> 00:41:58.119
judge it by the tooling. I means there are except
671
00:41:58.159 --> 00:42:01.280
that new test runners of NOJ that is actually bring
672
00:42:01.440 --> 00:42:04.519
less features. It's funny because the new test runners that
673
00:42:04.679 --> 00:42:07.320
a lot of back end teams are now using, it's
674
00:42:08.920 --> 00:42:11.960
us it brings much less testing features and what used
675
00:42:11.960 --> 00:42:15.320
to be like ingested people don't need even all the features,
676
00:42:15.360 --> 00:42:19.440
and there are very few like new tooling in the
677
00:42:19.480 --> 00:42:23.320
back and accept maybe contract testing tooling, which we will mention. However,
678
00:42:23.360 --> 00:42:25.320
you see that in the front and words there is
679
00:42:25.320 --> 00:42:29.559
a bloom and always constant change and struggling to build
680
00:42:29.679 --> 00:42:33.440
new tooling that are can catch up with the front
681
00:42:33.480 --> 00:42:35.199
and complexity so well.
682
00:42:35.400 --> 00:42:40.199
The fact that we actually need visual testing is an
683
00:42:40.199 --> 00:42:44.320
indication of how much more challenging front test and testing is.
684
00:42:44.360 --> 00:42:49.159
It's it's like almost like saying, let's let's test the
685
00:42:49.599 --> 00:42:54.719
bits flowing over some network connection and and basically try
686
00:42:54.760 --> 00:42:59.760
to analyze the system from that. If I'm using like
687
00:43:00.280 --> 00:43:05.039
silly analogy. So you were talking about you mentioned contract testing,
688
00:43:05.400 --> 00:43:06.719
Can you elaborate on that?
689
00:43:08.199 --> 00:43:10.039
Oh, definitely, I think that this is now one of
690
00:43:10.039 --> 00:43:15.760
the most interesting fields in testing in overall. I would
691
00:43:15.760 --> 00:43:18.320
even say that if in twenty twenty five you're not
692
00:43:18.400 --> 00:43:22.119
doing your your coding against API without any explicit contract,
693
00:43:22.920 --> 00:43:25.360
it shouldn't be an option on the table. First, let
694
00:43:25.440 --> 00:43:28.400
me explain there is. So we are all consuming API,
695
00:43:28.599 --> 00:43:30.639
whether front and to back and back and to backan
696
00:43:31.280 --> 00:43:38.519
and every system is distributed. Now, if as a consumer
697
00:43:38.559 --> 00:43:41.760
of an API, the payloads that I'm sending and receiving
698
00:43:41.840 --> 00:43:44.880
for me are just kind of unknown jason, some kind
699
00:43:44.880 --> 00:43:49.119
of Jason that I can control and predict, then imate
700
00:43:49.280 --> 00:43:53.920
risk of receiving a lot of breaking changes. So say
701
00:43:53.960 --> 00:43:57.440
for example, things work, but then my the APIs and
702
00:43:57.519 --> 00:44:02.639
un consuming change something. My test pass, but production will fail.
703
00:44:03.519 --> 00:44:06.320
And we are at this point we are really good
704
00:44:06.320 --> 00:44:12.840
at testing our components. But but the ocean, that ocean
705
00:44:12.880 --> 00:44:17.880
between components, we have much less testing ways and techniques
706
00:44:17.920 --> 00:44:21.320
to ensure the two different parties are aligned on the
707
00:44:21.400 --> 00:44:24.039
on the API between the two and now there is
708
00:44:24.079 --> 00:44:27.599
a there is a bloom of new techniques and tooling
709
00:44:27.760 --> 00:44:31.760
to cover exactly this as API consumer, how do I know?
710
00:44:31.880 --> 00:44:35.920
How do I get a guarantee that the payloads are
711
00:44:36.039 --> 00:44:38.400
changing and I should be prepared for that.
712
00:44:40.400 --> 00:44:43.320
So what are the solutions in this space?
713
00:44:44.559 --> 00:44:48.639
Yeah, so there is a spectrum of tooling and options. Uh,
714
00:44:48.840 --> 00:44:52.519
the simplest one just in a moder repot, just creates
715
00:44:52.599 --> 00:44:55.360
OD schemas, share them between the back end and the
716
00:44:55.360 --> 00:44:59.639
front end, and you get a solid protection, right, I mean,
717
00:45:00.239 --> 00:45:03.920
whenever the back end changes the schemas, your front and
718
00:45:04.039 --> 00:45:07.360
tests or it can be also your back end will
719
00:45:07.360 --> 00:45:12.239
fail because you have now you can generate typing from this.
720
00:45:13.360 --> 00:45:16.639
From these schemas, you can also do runtime validation and
721
00:45:16.679 --> 00:45:19.199
you're protected. But it doesn't cover a lot of frisk.
722
00:45:19.280 --> 00:45:22.880
On the other sound side of the spectrum, the more
723
00:45:23.039 --> 00:45:27.519
fancy tooling are mocking servers that will let you know
724
00:45:27.599 --> 00:45:31.800
that we'll provide you with a very sophisticated mechanism to
725
00:45:32.400 --> 00:45:36.039
synchronize the two parties. But there is the middle. I
726
00:45:36.079 --> 00:45:40.599
think that most of the solutions that are somewhere located
727
00:45:40.599 --> 00:45:43.480
in the middle between that are not too complex and
728
00:45:43.760 --> 00:45:47.320
not too simple and are not simplistic. Use the following.
729
00:45:47.880 --> 00:45:53.400
The API provider generates open API, usually it comes for free,
730
00:45:53.559 --> 00:45:56.519
and it somehow shared that open API with the consumer.
731
00:45:57.239 --> 00:46:00.920
Now the consumer all the consumers have tooling to generate
732
00:46:01.039 --> 00:46:05.280
both types, but also run time validation out of open API.
733
00:46:06.519 --> 00:46:10.440
So now as a fronten for example, whenever my back
734
00:46:10.559 --> 00:46:14.039
end changes the open API, if my types I say
735
00:46:14.119 --> 00:46:17.840
that's a new field was added, my code will generate
736
00:46:17.920 --> 00:46:21.159
types types based on this open API. And if I'm
737
00:46:21.199 --> 00:46:24.280
trying to access some fields it doesn't exist anymore or
738
00:46:24.400 --> 00:46:28.239
is the wrong type, I will get compile time or
739
00:46:28.320 --> 00:46:31.440
testing failure right away. So it's kind of me and
740
00:46:31.719 --> 00:46:35.840
back end are always aligned without spinning up a complex
741
00:46:35.880 --> 00:46:37.519
and fancy edit end environment.
742
00:46:38.440 --> 00:46:41.920
Yeah, but like you said, it almost kind of necessitates
743
00:46:41.960 --> 00:46:47.119
in many cases working in a mono repo. Otherwise, First
744
00:46:47.119 --> 00:46:49.519
of all, I hesitate to call it tests. I mean,
745
00:46:49.760 --> 00:46:54.639
it's it's basically you know, one of the benefits of
746
00:46:54.800 --> 00:47:01.400
using static typing is that you get certain bugs become
747
00:47:02.159 --> 00:47:05.840
impossible because of the type system. But I don't really
748
00:47:05.920 --> 00:47:10.440
consider it to be tests. It's basically that's it might
749
00:47:11.519 --> 00:47:14.480
You could call it another step in your build that
750
00:47:14.639 --> 00:47:18.119
might fail, but I hesitate to call it tests.
751
00:47:18.960 --> 00:47:22.239
Maybe it is. I don't know. That's an interesting philosophical question.
752
00:47:22.719 --> 00:47:25.800
Some of the fancy solutions are going beyond static typing.
753
00:47:25.880 --> 00:47:28.199
So what they do is that if you have an
754
00:47:28.199 --> 00:47:32.039
open API, it's not all about typing. Some kind of
755
00:47:32.079 --> 00:47:36.400
the assertions there are can be asserted only on run time. Say,
756
00:47:36.440 --> 00:47:39.159
for example, you have a rejects some of the fields
757
00:47:39.199 --> 00:47:43.360
type is a rejics. You can't assert this in typing Typescript. Also,
758
00:47:43.480 --> 00:47:46.320
Typescript is now adding some kind of tragic support. So
759
00:47:46.360 --> 00:47:50.119
what these tooling are doing during testing? Whenever you are
760
00:47:50.159 --> 00:47:53.079
sending to your mock some on payload, they are very
761
00:47:53.119 --> 00:47:58.199
fine whether the received testing payload is indeed conforming to
762
00:47:58.239 --> 00:48:02.800
the open API definition. So in that regard they can
763
00:48:03.119 --> 00:48:06.639
make the test fail if what you're using, the pedal
764
00:48:06.719 --> 00:48:12.719
you're using is not what the Beckend API defined. Of
765
00:48:12.800 --> 00:48:15.320
course for this you need like real testing. And maybe
766
00:48:15.360 --> 00:48:18.280
it's worth mentioning one work very cool features. I guess
767
00:48:18.320 --> 00:48:22.079
you both know ms W. It's a very popular JavaScript
768
00:48:22.159 --> 00:48:25.239
library for mocking intercepting HTP requests.
769
00:48:26.119 --> 00:48:27.079
No, sorry, don't.
770
00:48:27.519 --> 00:48:31.079
Oh okay, So let me introduce it. Ms w H
771
00:48:32.039 --> 00:48:36.079
is a library that allows by the way both also
772
00:48:36.079 --> 00:48:38.880
know jes also back in. It allows you to kind
773
00:48:38.880 --> 00:48:43.079
of define in your in your environment, kind of in
774
00:48:43.280 --> 00:48:46.440
kind of intercept network requests and say, hey, whenever I'm
775
00:48:46.480 --> 00:48:51.760
approaching another API, say the beckend, intercept it in browser.
776
00:48:51.800 --> 00:48:54.400
It's using by the way service workers, so you will
777
00:48:54.440 --> 00:48:57.400
see the request in the network tub and Chrome. But
778
00:48:57.559 --> 00:49:01.239
instead of going out outside your browser, MSW will return
779
00:49:01.320 --> 00:49:06.159
the response on behalf of your API, so you get
780
00:49:06.159 --> 00:49:11.599
a much more a much faster, and more predictable working environment. Now,
781
00:49:11.719 --> 00:49:15.400
ms W added a new feature this year, and we're
782
00:49:15.440 --> 00:49:20.800
discussing recent changes that instead of you defining what will
783
00:49:20.840 --> 00:49:23.400
be the API responses and you might be wrong there
784
00:49:23.480 --> 00:49:25.719
right what you think that there might be threefills in
785
00:49:25.719 --> 00:49:29.039
this payload, the beck end in fact, in production will
786
00:49:29.079 --> 00:49:33.400
return four or two. Ms W generates the responses from
787
00:49:33.440 --> 00:49:36.920
the beckend open API. So you provided, hey, this is
788
00:49:36.920 --> 00:49:39.760
what the API provided us the real world, the real
789
00:49:40.079 --> 00:49:43.639
scheme as it was generated, and it takes and generates
790
00:49:43.679 --> 00:49:47.079
responses based on this. So it's kind of you are
791
00:49:47.239 --> 00:49:51.920
you and the API that you consume are aligned, are
792
00:49:51.960 --> 00:49:54.360
always aligned? Does it make sense?
793
00:49:54.639 --> 00:49:55.599
Yes, as it does.
794
00:49:56.679 --> 00:49:59.519
Yeah, MSW stands from Mark Service Worker in case that
795
00:49:59.559 --> 00:50:02.800
wasn't clear already. So before we wrap up, is there
796
00:50:02.840 --> 00:50:05.039
anything else you wanted to talk about real quick that
797
00:50:05.039 --> 00:50:06.760
we have not yet covered in this world?
798
00:50:09.400 --> 00:50:11.960
Let me see, I've prepared a table of content here
799
00:50:12.559 --> 00:50:17.639
on the side. We can discuss a little bit more
800
00:50:17.679 --> 00:50:20.719
about testing with it. I also have we have covered
801
00:50:20.719 --> 00:50:28.039
this nothing significant syd with We have touched the measure
802
00:50:28.159 --> 00:50:33.119
points and advancement in testing, all right.
803
00:50:34.199 --> 00:50:39.440
So if before we I do have a final question though,
804
00:50:39.480 --> 00:50:43.559
in this context, So when you usually are when you're
805
00:50:43.599 --> 00:50:49.119
brought into a new project and and you want to
806
00:50:49.639 --> 00:50:52.559
before making any changes, I assume you want to make
807
00:50:52.599 --> 00:50:55.800
sure that you have good test coverage, because who wants
808
00:50:55.800 --> 00:50:58.559
to start making changes in a project before we have
809
00:50:58.639 --> 00:51:03.280
good testing coverage. What's the main thing that you're looking
810
00:51:03.400 --> 00:51:07.000
for in order to say this project has good test
811
00:51:07.079 --> 00:51:10.320
coverage versus another project that you might say this project
812
00:51:10.400 --> 00:51:12.239
does not have good test coverage.
813
00:51:15.079 --> 00:51:21.840
Yeah, that's that's the million dollar question. I'm even struggling
814
00:51:21.880 --> 00:51:30.960
to enter it. It's so it's so deep. But I
815
00:51:31.000 --> 00:51:34.440
think that I'm using two I'm using two aroistics for this.
816
00:51:37.000 --> 00:51:41.880
The first one is unfortunately not not in period. It's
817
00:51:41.880 --> 00:51:45.280
not measured. Uh, if people are moving, if people are
818
00:51:45.320 --> 00:51:50.719
really are having confidence to deploy fast, deploying deploying fast
819
00:51:50.760 --> 00:51:53.519
and just feeling confident to do it, you see it,
820
00:51:53.639 --> 00:51:56.960
I mean you you feel it, they trust are testing.
821
00:51:57.559 --> 00:52:03.320
That's I think the ultimate eventually the ultimate test. On
822
00:52:03.400 --> 00:52:04.679
top of this, I'm.
823
00:52:04.760 --> 00:52:07.760
It's literally just to just to pause you for a second,
824
00:52:08.239 --> 00:52:13.280
it's literally that subjective feeling that you test. You can
825
00:52:13.360 --> 00:52:18.199
ask people working on the project, how well do you
826
00:52:18.440 --> 00:52:23.239
trust your tests? If you if you make change and
827
00:52:23.280 --> 00:52:26.079
it makes its way all the way to to to
828
00:52:26.199 --> 00:52:31.039
being deployed to production, how secure do you feel about
829
00:52:31.119 --> 00:52:35.119
it working properly and not breaking any of the things
830
00:52:35.239 --> 00:52:39.679
or introducing any form of regression. It's it's essentially that
831
00:52:39.679 --> 00:52:42.239
that level of subjective feeling in the team.
832
00:52:42.719 --> 00:52:45.519
Yeah, that's the bottom line. I'm also using test coverage.
833
00:52:46.039 --> 00:52:48.440
I'm using branch coverage. Yeah, I know that it's not
834
00:52:48.480 --> 00:52:52.039
a perfect metric, but it's very simple metric and for
835
00:52:52.119 --> 00:52:54.519
the majority of teams, they are not like they're not
836
00:52:54.679 --> 00:52:58.360
fooling themselves. If if if system has a very high
837
00:52:58.440 --> 00:53:03.400
branch coverage. It's means that the testing are indeed testing
838
00:53:03.559 --> 00:53:06.000
most of that ninety coverage, not all of them, but
839
00:53:06.119 --> 00:53:09.599
a high percentage of this at least it tells me
840
00:53:10.199 --> 00:53:14.159
that most of the system areas are are trying to
841
00:53:14.199 --> 00:53:16.320
be tested. On top of this, I usually add some
842
00:53:16.400 --> 00:53:22.159
kind of mutation testing. So mutation testing means that it's
843
00:53:22.199 --> 00:53:25.719
a framework that plant bugs in your code, Like it
844
00:53:25.760 --> 00:53:28.480
can plant a bug in each and every line and
845
00:53:28.519 --> 00:53:30.639
then it runs your test and then true that they fail.
846
00:53:31.840 --> 00:53:36.079
And if yeah, now, because I mean it's it's an
847
00:53:36.519 --> 00:53:42.719
it's amazingly efficient technique to realize testing efficiency. But on
848
00:53:42.760 --> 00:53:45.280
the other end, it's really slow and cumbersome. So what
849
00:53:45.400 --> 00:53:48.840
I usually do I run it like I run it
850
00:53:48.880 --> 00:53:53.119
occasionally just to get a sense of what is the
851
00:53:53.239 --> 00:53:57.280
ratio between the COD coverage and the mutations. So in
852
00:53:57.320 --> 00:54:00.400
other words, if I see that I have COD coverage
853
00:54:00.559 --> 00:54:03.320
of ninety percent in a file, say I find file
854
00:54:03.360 --> 00:54:07.760
as ninety percent coverage and the mutation score is very low,
855
00:54:08.400 --> 00:54:10.639
it tells me that this is a nonsense coverage. You
856
00:54:10.679 --> 00:54:12.760
know that that's just visits amer coat off it. But
857
00:54:12.800 --> 00:54:15.800
it's not asserting one anything On the other hand, if
858
00:54:15.840 --> 00:54:19.079
I seek correlation between coverage and mutations in file, I
859
00:54:19.159 --> 00:54:24.480
know that this coverage is authentic. It's like trustworthy. So
860
00:54:24.599 --> 00:54:29.000
this is the empiric side of measuring that the test.
861
00:54:29.559 --> 00:54:32.000
Cool, Okay, thank you for that.
862
00:54:34.280 --> 00:54:39.480
Elucidation is that how you put elucidations.
863
00:54:39.639 --> 00:54:40.079
That's the word.
864
00:54:40.079 --> 00:54:43.880
I'll I'll google that word right after elucidate to explain
865
00:54:44.000 --> 00:54:46.119
something clearly is basically what it means.
866
00:54:46.159 --> 00:54:50.280
So that is the word of the day, elucidate already.
867
00:54:50.559 --> 00:54:53.119
So without move on the picks, picture that part of
868
00:54:53.119 --> 00:54:55.320
the show where we get to talk about anything we
869
00:54:55.360 --> 00:55:00.920
want to within reason of course, board, games, books, food, movie, tech,
870
00:55:01.000 --> 00:55:04.920
if you want to, I'll go first and my pick.
871
00:55:05.480 --> 00:55:07.039
I do have a pick before I get to the
872
00:55:07.159 --> 00:55:08.960
high point of the Dad jokes of the week, and
873
00:55:09.000 --> 00:55:11.639
it has to do with dad jokes. So at the
874
00:55:11.719 --> 00:55:14.639
end of our podcast episode that Dan and I recorded
875
00:55:14.719 --> 00:55:19.000
with Ryan Carneato and Tanner Lindsley, I found out something
876
00:55:19.039 --> 00:55:20.960
that I wasn't aware of that now Crosby, who was
877
00:55:20.960 --> 00:55:25.039
the founder of a G Grid, had died and apparently
878
00:55:25.079 --> 00:55:26.760
it is some type of helicopter accident.
879
00:55:27.039 --> 00:55:30.599
Well, it's to be fair, it's not new right.
880
00:55:30.440 --> 00:55:32.280
I know it is the first I'd found about it,
881
00:55:32.280 --> 00:55:34.679
you know. So yeah, that just tells you how out
882
00:55:34.679 --> 00:55:37.880
there in the news I am every day, and that
883
00:55:37.920 --> 00:55:41.119
brings back memories for me. So four years ago we
884
00:55:41.159 --> 00:55:45.719
did a Jobascript Jobber episode with Nile, myself and Ag
885
00:55:46.000 --> 00:55:49.960
and just a j O'Neal and Chuck and the Reason
886
00:55:50.000 --> 00:55:52.360
that I was. Episode five oh four came out in
887
00:55:52.440 --> 00:55:55.119
October of twenty twenty one, and this was one of
888
00:55:55.159 --> 00:55:57.639
my favorite episodes, as I mentioned before, because when it
889
00:55:57.679 --> 00:55:59.480
got to the end, Nile just jumped right in and
890
00:55:59.519 --> 00:56:04.880
throughout a whole string of geek dad jokes that I
891
00:56:05.000 --> 00:56:07.280
had to do with going to a no sequel bar
892
00:56:07.360 --> 00:56:09.000
and there's no table, So then he went to a
893
00:56:09.039 --> 00:56:11.599
sequel bar that had tables, but he couldn't join anybody,
894
00:56:11.639 --> 00:56:12.760
and it was awesome.
895
00:56:13.000 --> 00:56:14.320
I was giving him a standing ovation.
896
00:56:15.039 --> 00:56:18.000
So that's my pick of the for the week is
897
00:56:18.360 --> 00:56:21.719
you can find it on Top Endos episode five o
898
00:56:21.880 --> 00:56:24.039
four and if you want to look at the transcript
899
00:56:24.159 --> 00:56:26.760
or listen to it, it's it's one of my faves.
900
00:56:27.440 --> 00:56:30.480
So with that, we'll get to the dad jokes of
901
00:56:30.519 --> 00:56:35.000
the week. So question, how come nobody laugh at the
902
00:56:35.079 --> 00:56:36.800
joke about the fault DG eighteen.
903
00:56:38.360 --> 00:56:39.440
It was poor execution.
904
00:56:43.159 --> 00:56:44.400
That's good, all right.
905
00:56:44.639 --> 00:56:46.840
My wife told me the other day, I'm really getting
906
00:56:46.880 --> 00:56:49.639
sick of you over using contractions, and I said, it's
907
00:56:49.639 --> 00:56:56.760
what it is. And then, uh, someone threw a can
908
00:56:56.800 --> 00:56:58.760
of soda at me today or i'd say pop on
909
00:56:58.800 --> 00:56:59.440
the West coast.
910
00:56:59.440 --> 00:57:05.360
But I'm all right. It was a soft drink. And
911
00:57:05.400 --> 00:57:06.519
those are the dad jokes of.
912
00:57:06.480 --> 00:57:09.000
The week, and those who are definitely dad jokes.
913
00:57:08.800 --> 00:57:09.440
Yes they were.
914
00:57:09.760 --> 00:57:12.239
I'm embarrassed to admit that I understood two of the three.
915
00:57:13.039 --> 00:57:16.320
I promised to to rewatch that section and catch up.
916
00:57:16.800 --> 00:57:18.440
Okay, yeah, I can explain it. Yeah, it sort of
917
00:57:18.480 --> 00:57:20.639
kills a joke and you have to explain it.
918
00:57:21.079 --> 00:57:21.880
That is.
919
00:57:24.039 --> 00:57:25.559
Dan. What do you got for us?
920
00:57:26.440 --> 00:57:29.119
Okay, I've got a couple of things. So first of all,
921
00:57:29.199 --> 00:57:32.119
I posted it on x but I also want to
922
00:57:32.159 --> 00:57:35.800
shout it out a big thanks to THEO also known
923
00:57:35.840 --> 00:57:38.840
as THEO three GG, who puts out a lot of
924
00:57:38.880 --> 00:57:44.800
content online. I'm working on a project that involves Stripe
925
00:57:44.800 --> 00:57:49.239
integration for one of our services, and I've not worked
926
00:57:49.239 --> 00:57:54.639
with Stripe significantly, I think, ever, so I was kind
927
00:57:54.639 --> 00:57:57.199
of learning a lot about the stripe APIs and how
928
00:57:57.239 --> 00:58:03.119
it works, and they seem but Theo's video was essentially
929
00:58:04.480 --> 00:58:09.039
all the stripe gotchas that he encountered while working on
930
00:58:09.119 --> 00:58:16.119
various projects and how to overcome them, and it was very,
931
00:58:16.480 --> 00:58:19.480
very helpful. It came along at just the right time
932
00:58:20.679 --> 00:58:26.400
and undoubtedly saved me from a lot of pain down
933
00:58:26.480 --> 00:58:29.480
the line. So thank you for that, CEOs, Thank you
934
00:58:29.559 --> 00:58:32.599
for sharing this information. This is what sharing on the
935
00:58:32.599 --> 00:58:36.400
web is all about. So that's one thing I wanted
936
00:58:36.440 --> 00:58:40.920
to mention. Another is that we did this kind of
937
00:58:41.480 --> 00:58:45.360
watch along party at work. It's something that we tried
938
00:58:45.400 --> 00:58:48.280
for the first time and got a lot of good responses.
939
00:58:49.239 --> 00:58:53.719
It's great when you can have somebody at work giving
940
00:58:53.760 --> 00:58:57.639
a talk on a particular topic, but it's not always possible. Obviously,
941
00:58:58.119 --> 00:59:01.679
you may not have experts in how on everything. So
942
00:59:01.760 --> 00:59:04.119
what we actually did is we wanted to learn more
943
00:59:04.159 --> 00:59:08.239
about MCP and Jack Harrington, who we've had on the
944
00:59:08.280 --> 00:59:10.480
show a couple of times. He was also one of
945
00:59:10.480 --> 00:59:19.079
the hosts on the React podcast, and he created a
946
00:59:19.159 --> 00:59:24.199
series of videos about MCP that are just excellent. So
947
00:59:24.559 --> 00:59:27.639
we watched several of them in a row. They're actually
948
00:59:27.679 --> 00:59:30.719
fairly short each They're like ten to twenty minutes each
949
00:59:32.079 --> 00:59:35.519
and he explains things so well. So we basically watched
950
00:59:35.519 --> 00:59:38.039
a video and then had a brief discussion about it,
951
00:59:38.119 --> 00:59:42.079
and then watched the next video and so forth, and
952
00:59:42.400 --> 00:59:47.119
it was really great and I highly recommend this if
953
00:59:47.199 --> 00:59:51.960
you want to educate people at work. It's a great approach,
954
00:59:52.039 --> 00:59:55.199
I think. And the final thing I want to shout
955
00:59:55.239 --> 01:00:02.519
out is that I recently countered this video. This guy
956
01:00:02.679 --> 01:00:06.920
does videos about old hardware, and he released a video
957
01:00:07.000 --> 01:00:14.239
about a personal computer from the early eighties really called
958
01:00:14.280 --> 01:00:18.320
the Ti ninety nine four A. And the reason that
959
01:00:18.639 --> 01:00:22.639
this hit close to my heart is that that was
960
01:00:22.800 --> 01:00:26.920
my first personal computer when I was a kid. It
961
01:00:27.159 --> 01:00:32.519
was a very weird and wacky machine. And he literally
962
01:00:32.880 --> 01:00:36.320
kind of opened the screws and showed the insight of
963
01:00:36.400 --> 01:00:39.559
how that machine worked, to explain why it was so
964
01:00:39.679 --> 01:00:42.639
weird and wacky, and why it worked the way that
965
01:00:42.719 --> 01:00:44.719
it did, and what was good about it, and a
966
01:00:44.719 --> 01:00:47.440
lot of things that weren't so good about it. And
967
01:00:48.760 --> 01:00:52.119
you know, it brought back a lot of memories. So
968
01:00:53.039 --> 01:00:56.920
I'll share that link to that video. Let's just say
969
01:00:57.079 --> 01:01:01.119
that it was really interesting to probe for a system
970
01:01:01.559 --> 01:01:03.519
that had all of four.
971
01:01:03.400 --> 01:01:04.559
K of RAM.
972
01:01:04.880 --> 01:01:08.679
You know, in these days of gigabytes, you know, this
973
01:01:08.760 --> 01:01:09.599
is all we had.
974
01:01:11.920 --> 01:01:12.760
Yeah, that's it.
975
01:01:12.840 --> 01:01:16.800
Four kfram And within that four kf RAM, I implemented
976
01:01:16.840 --> 01:01:20.599
a Donkey Kong like game that had two screens. And
977
01:01:20.639 --> 01:01:24.880
I was like thirteen at the time, so yeah, it.
978
01:01:24.679 --> 01:01:26.760
Was which language did you use?
979
01:01:26.960 --> 01:01:34.760
I'm curious, basic, because that's what it had. It was.
980
01:01:36.760 --> 01:01:42.079
I joked that it simultaneously got me into programming and
981
01:01:42.400 --> 01:01:44.559
kind of also scarred me for life.
982
01:01:45.679 --> 01:01:47.119
So my first.
983
01:01:46.880 --> 01:01:48.800
Job in tech was doing tech support, and we were
984
01:01:48.840 --> 01:01:51.119
dealing with Win just three point one with our software,
985
01:01:51.159 --> 01:01:54.639
and I was constantly having to help users with memory
986
01:01:54.679 --> 01:01:57.679
issues and upper memory and lower memory and shifting things
987
01:01:57.679 --> 01:02:00.480
around so that things could run. And and then when
988
01:02:00.519 --> 01:02:02.000
I was ninety five came along and a lot of
989
01:02:02.000 --> 01:02:02.679
that went bye bye.
990
01:02:02.760 --> 01:02:05.920
I have a whole story about that, but we don't
991
01:02:05.920 --> 01:02:08.599
have the time. We could probably do an entire episode
992
01:02:08.639 --> 01:02:12.440
on reminiscing. So those would be my picture today.
993
01:02:13.119 --> 01:02:18.920
All right, Johanni, or to you, Yeah, I've recently found
994
01:02:18.920 --> 01:02:25.679
myself I'm interested in quality coffee, So I realized in
995
01:02:25.719 --> 01:02:28.159
mostly in the coffee story, So I really I always
996
01:02:28.199 --> 01:02:31.039
thought that like an expensive coffee is like expensive wine,
997
01:02:31.079 --> 01:02:34.079
you just pay for you know, for that brand or
998
01:02:34.079 --> 01:02:37.360
for that weird test. But then I when I started
999
01:02:37.400 --> 01:02:39.960
reading more and more and see videos about testing, I
1000
01:02:40.000 --> 01:02:43.880
realized that like between the farm and your cap, they
1001
01:02:43.920 --> 01:02:49.559
are like ten mistakes, like like dozens of mistakes. Most
1002
01:02:49.599 --> 01:02:54.480
of them are intentional, intentional to just to just cut
1003
01:02:54.559 --> 01:03:00.039
coast and make the coffee horrible. Take for example, in
1004
01:03:00.079 --> 01:03:04.960
any commodity farms, when they when they pick the cherry,
1005
01:03:05.480 --> 01:03:08.960
when they when they collect the cherry picks, some of
1006
01:03:09.000 --> 01:03:13.639
them are already just mature and some are unriped, some
1007
01:03:13.679 --> 01:03:16.519
are red, some are green. Uh so, but you know
1008
01:03:16.599 --> 01:03:18.880
they have to be efficient, so so just take the
1009
01:03:19.079 --> 01:03:21.960
entire tree down. You get coffee bins that are right
1010
01:03:22.119 --> 01:03:26.199
unripe together, some are really sour, some are really really sweet.
1011
01:03:26.559 --> 01:03:30.880
And and that's just one thing that the coffee coffee
1012
01:03:30.920 --> 01:03:34.559
production is doing in order to bring your coffee with
1013
01:03:34.719 --> 01:03:39.800
lower costs and more revenues for them, and really getting
1014
01:03:39.800 --> 01:03:43.360
a great cup of coffee demands a lot of a
1015
01:03:43.360 --> 01:03:49.320
lot of uh decisions and optimization all over the all
1016
01:03:49.360 --> 01:03:54.360
over that process, and it's it's there's no point here,
1017
01:03:54.559 --> 01:04:01.440
just a very fascinating journey that involves biology, nature people. Yeah.
1018
01:04:01.519 --> 01:04:04.440
Other than that, I realized that we are getting more
1019
01:04:04.440 --> 01:04:08.320
and more dependent on lms, right, I mean we're about
1020
01:04:08.320 --> 01:04:10.280
to use it for ten percent off coding or maybe
1021
01:04:10.360 --> 01:04:16.199
ninety percent. I mean it's objective, and I felt like
1022
01:04:16.280 --> 01:04:18.360
it's too much of a black box for me. I
1023
01:04:18.400 --> 01:04:21.199
can't understand the mechanic them. So say, for example, I'm
1024
01:04:21.199 --> 01:04:25.760
providing it with fifteen instructions, it's respecting thirty. Why it
1025
01:04:26.119 --> 01:04:28.400
fits inside the context within the Why did you ignore
1026
01:04:28.400 --> 01:04:31.320
twenty of them? So I bought a book which is
1027
01:04:31.360 --> 01:04:36.559
called I wanted to understand this box from inside. I
1028
01:04:36.599 --> 01:04:39.159
bought a book that's called Building an LM from Scratch.
1029
01:04:39.639 --> 01:04:44.000
It's just teach you build all the layers of an LM.
1030
01:04:44.039 --> 01:04:46.519
Of course a very simplistic one, but you get idea
1031
01:04:46.639 --> 01:04:50.239
of all the moving pieces. I'm now reading it, coding it,
1032
01:04:51.760 --> 01:04:56.280
and I think that I, after processing four chapters, I
1033
01:04:56.320 --> 01:04:57.679
can surely recommend it.
1034
01:04:59.800 --> 01:05:01.760
Right, to go back to your first pick, I would.
1035
01:05:01.840 --> 01:05:04.239
All I will say is that the term quality copy
1036
01:05:04.280 --> 01:05:09.840
is an oxymoron, so there is no such thing, and
1037
01:05:09.880 --> 01:05:12.760
I'll leave bit at that. So all right, Well, thank you,
1038
01:05:12.800 --> 01:05:15.400
Johanni for joining us today and talk to you about testing.
1039
01:05:15.480 --> 01:05:18.519
I appreciate it you coming here before we go.
1040
01:05:19.119 --> 01:05:20.840
If people want to get hold of you and give
1041
01:05:20.840 --> 01:05:24.360
you money and and do all that kind of stuff,
1042
01:05:24.400 --> 01:05:26.880
where is the best place to track your genius?
1043
01:05:29.000 --> 01:05:32.679
Thanks for mentioning this. Well, my gietime page has a
1044
01:05:32.800 --> 01:05:35.239
contacts There is also goldberg Oni dot com.
1045
01:05:36.639 --> 01:05:39.960
And you said that you work as a consultant, so
1046
01:05:40.039 --> 01:05:44.559
I assume that you give assistance training and help organizations
1047
01:05:44.599 --> 01:05:48.039
improve their testing and test coverage and quality of tests
1048
01:05:48.039 --> 01:05:48.599
and whatnot.
1049
01:05:49.599 --> 01:05:56.639
Yes, workshops ends on, ends on, and holding any any
1050
01:05:56.679 --> 01:05:59.280
any kind of assistant and training.
1051
01:06:00.760 --> 01:06:01.079
And that's it.
1052
01:06:01.199 --> 01:06:07.880
Joannie Goldberg Goldberg Goldberg, Yohnnie excuse me? Alrighty and Yonie
1053
01:06:07.880 --> 01:06:09.719
is y o n I just.
1054
01:06:10.239 --> 01:06:15.039
Yeah or gita dot com slash Goldberg Yonnie.
1055
01:06:15.519 --> 01:06:19.920
Okay, alrighty, Well that is it. Another thrilling episode is
1056
01:06:19.920 --> 01:06:21.800
in the books. Thank you for joining us and we
1057
01:06:21.880 --> 01:06:24.280
will talk to you next time on job script Jabber