#565: Tachyon, Python 3.15's Built-in Sampling Profiler

WEBVTT
00:00:00.020 --> 00:00:02.080
Do you know what's actually slow in your Python app?
00:00:02.480 --> 00:00:03.200
Or are you just guessing?
00:00:03.780 --> 00:00:04.140
Until now,
00:00:04.380 --> 00:00:07.000
profiling Python meant a tracing profiler
00:00:07.070 --> 00:00:09.740
that made your code two to three times slower,
00:00:10.460 --> 00:00:12.660
or a third-party tool that broke
00:00:12.670 --> 00:00:14.480
with every new release of the Python runtime.
00:00:15.180 --> 00:00:17.000
Python 3.15 fixes that.
00:00:17.340 --> 00:00:19.680
It ships with Tachyon, a sampling profiler
00:00:20.160 --> 00:00:21.500
built into the standard library.
00:00:22.100 --> 00:00:23.820
It attaches to live production apps
00:00:23.970 --> 00:00:25.300
with almost zero overhead.
00:00:26.040 --> 00:00:28.000
My guests are Pablo Galindo Salgado,
00:00:28.600 --> 00:00:30.980
a CPython core developer and steering council member,
00:00:31.560 --> 00:00:35.240
and Laszlo Kiskoller from Bloomberg's Python infrastructure team.
00:00:35.880 --> 00:00:38.840
Their first prototype ran at two samples a second.
00:00:39.380 --> 00:00:42.320
Now it can do over a million hertz sampling.
00:00:42.760 --> 00:00:46.120
The best news is this lands in Python 3.15 today.
00:00:47.360 --> 00:00:48.480
This is Talk Python To Me,
00:00:48.680 --> 00:00:50.580
episode 565,
00:00:51.040 --> 00:00:52.700
recorded September 16th,
00:00:53.000 --> 00:00:53.660
2026.
00:01:11.900 --> 00:01:16.220
Welcome to Talk Python To Me, the number one Python podcast for developers and data scientists.
00:01:16.770 --> 00:01:17.480
This is your host,
00:01:17.700 --> 00:01:18.120
Michael Kennedy.
00:01:18.320 --> 00:01:22.060
I'm a PSF fellow who's been coding for over 25 years.
00:01:22.720 --> 00:01:23.760
Let's connect on social media.
00:01:24.140 --> 00:01:26.400
You'll find me and Talk Python on Mastodon, Bluesky,
00:01:26.760 --> 00:01:27.280
and X.
00:01:27.560 --> 00:01:29.400
The social links are all in your show notes.
00:01:30.160 --> 00:01:33.680
You can find over 10 years of past episodes at talkpython.fm.
00:01:33.860 --> 00:01:37.060
And if you want to be part of the show, you can join our recording live streams.
00:01:37.420 --> 00:01:37.880
That's right.
00:01:38.140 --> 00:01:41.340
We live stream the raw uncut version of each episode on YouTube.
00:01:41.940 --> 00:01:46.260
Just visit talkpython.fm/youtube to see the schedule of upcoming events.
00:01:46.580 --> 00:01:50.220
Be sure to subscribe there and press the bell so you'll get notified anytime we're recording.
00:01:50.480 --> 00:01:51.980
This episode is brought to you by Sentry.
00:01:52.260 --> 00:01:56.360
Sentry's new MCP server connects your error reports straight to your coding agent.
00:01:56.880 --> 00:01:57.960
Claude Code reads the errors,
00:01:58.400 --> 00:01:58.940
finds the bugs,
00:01:59.300 --> 00:01:59.800
and fixes them.
00:02:00.340 --> 00:02:03.320
Get started at talkpython.fm/sentry.
00:02:03.600 --> 00:02:06.400
Pablo Laszlo, welcome to Talk Python Week.
00:02:06.640 --> 00:02:06.720
Pablo,
00:02:06.880 --> 00:02:07.280
welcome back.
00:02:07.380 --> 00:02:08.740
Always good to have you on the show.
00:02:08.899 --> 00:02:09.580
Laszlo, hello.
00:02:09.979 --> 00:02:10.700
Yeah, thank you very much.
00:02:11.039 --> 00:02:12.000
Yeah, it's good to see you both.
00:02:12.200 --> 00:02:13.760
I'm really excited to talk about profilers.
00:02:14.040 --> 00:02:16.540
I've had very extreme examples.
00:02:18.060 --> 00:02:18.860
Extreme examples.
00:02:19.860 --> 00:02:21.620
Extreme experiences with profiling.
00:02:21.880 --> 00:02:28.300
experiences. Because I was certain, Pablo, I was certain something was happening with my program,
00:02:28.500 --> 00:02:29.220
making it slow.
00:02:29.570 --> 00:02:34.780
And then I attached a profiler and it wasn't even closely related. It was just
00:02:35.080 --> 00:02:36.360
something completely else.
00:02:36.540 --> 00:02:40.420
I was using the wrong data structure instead of the 6,000 lines of
00:02:40.800 --> 00:02:46.120
complex mathematical stuff that I thought was slow. And it's really saved the day because I would
00:02:46.540 --> 00:02:50.100
have tried to rewrite stuff that was super complicated. And really all I needed to do
00:02:50.100 --> 00:02:53.920
with something silly like switch from a dictionary or a list to a set or a list to a dictionary,
00:02:54.220 --> 00:02:54.940
something like this, right?
00:02:55.040 --> 00:02:55.420
And it's like,
00:02:55.760 --> 00:02:55.880
wow,
00:02:56.220 --> 00:02:59.520
it's just like list find or list index sort of thing.
00:02:59.600 --> 00:03:00.100
Oh, my gosh.
00:03:00.720 --> 00:03:01.880
Yeah, many such cases.
00:03:03.140 --> 00:03:03.240
Yes,
00:03:03.400 --> 00:03:03.500
yes,
00:03:03.700 --> 00:03:03.760
yes.
00:03:05.000 --> 00:03:07.340
I think it's more common than you will think,
00:03:07.580 --> 00:03:07.660
right?
00:03:07.720 --> 00:03:07.940
Like,
00:03:08.280 --> 00:03:09.460
I know that when this happens,
00:03:09.720 --> 00:03:11.300
it's going to feel a bit bad because it's like,
00:03:11.320 --> 00:03:11.420
oh,
00:03:11.580 --> 00:03:12.100
man, I'm stupid.
00:03:12.300 --> 00:03:12.520
But like,
00:03:12.760 --> 00:03:16.039
no, it's extremely anti-intuitive in the world of modern software
00:03:16.160 --> 00:03:18.800
when there is a bazillion layers of things
00:03:18.940 --> 00:03:22.280
and now it's very much unclear what is costing things.
00:03:22.560 --> 00:03:23.820
And that happens everywhere.
00:03:24.160 --> 00:03:27.020
That happens in the smallest shops and the biggest shops.
00:03:27.780 --> 00:03:29.980
So the tools are very important.
00:03:30.380 --> 00:03:31.360
Yeah, they absolutely are.
00:03:31.400 --> 00:03:36.280
And you all are behind some really cool new profiling capabilities
00:03:36.520 --> 00:03:37.160
coming into Python.
00:03:37.680 --> 00:03:38.060
Yes, yes.
00:03:38.320 --> 00:03:40.560
Looking forward to diving into that with you.
00:03:40.860 --> 00:03:43.700
So let's just do a quick reintroduction for you,
00:03:43.900 --> 00:03:44.100
Pablo.
00:03:44.120 --> 00:03:45.640
Just tell people about yourself a little bit,
00:03:45.740 --> 00:03:46.460
what you've been up to.
00:03:46.940 --> 00:03:48.500
It's been a while since you've been on the show.
00:03:48.780 --> 00:03:49.000
Yes,
00:03:49.160 --> 00:03:49.560
but always,
00:03:49.720 --> 00:03:51.440
always been a lot of fun to be here.
00:03:51.620 --> 00:03:52.900
So like, let's do more, let's do more.
00:03:52.930 --> 00:03:55.620
But let me keep going and do more profilers
00:03:55.840 --> 00:03:58.260
so we can come here even more time.
00:03:58.440 --> 00:03:59.800
That's my goal,
00:03:59.980 --> 00:04:01.780
just doing more profilers to come here.
00:04:02.480 --> 00:04:03.060
So yes,
00:04:03.260 --> 00:04:04.340
I think last time we talked about
00:04:04.550 --> 00:04:06.200
perhaps PyStack or Membray, one of the two.
00:04:06.210 --> 00:04:07.400
I don't remember the last one.
00:04:08.020 --> 00:04:08.980
We spoke about both.
00:04:08.980 --> 00:04:09.800
I don't remember the order.
00:04:10.500 --> 00:04:11.000
Yeah, exactly.
00:04:11.470 --> 00:04:11.560
Yeah,
00:04:11.850 --> 00:04:12.740
but yeah.
00:04:12.890 --> 00:04:14.500
So yeah, since then, I think,
00:04:14.920 --> 00:04:15.900
Yeah, a bunch of things have changed.
00:04:15.950 --> 00:04:18.799
I think, you know, a bunch of Python releases since then.
00:04:19.250 --> 00:04:22.079
We are now in very close to 3.15.
00:04:22.520 --> 00:04:23.120
It's very hard.
00:04:23.170 --> 00:04:25.400
You will think, like, people know it is the last version,
00:04:25.540 --> 00:04:27.360
but in my mind, 3.15 is all news
00:04:27.660 --> 00:04:29.020
because the main branch is 3.16.
00:04:29.230 --> 00:04:30.740
So, like, it's like all stuff.
00:04:30.960 --> 00:04:32.200
It's like, oh, is that released already?
00:04:32.720 --> 00:04:34.340
Anyway, no, 3.15 on October.
00:04:34.560 --> 00:04:36.820
I'm sure you will do a very good episode about that
00:04:36.850 --> 00:04:36.940
so,
00:04:37.180 --> 00:04:38.340
you know, people should listen to it.
00:04:38.500 --> 00:04:38.680
Yeah,
00:04:38.850 --> 00:04:38.980
yeah.
00:04:39.580 --> 00:04:40.480
We've done before,
00:04:40.750 --> 00:04:40.840
like,
00:04:41.100 --> 00:04:43.340
kind of a launch event thing
00:04:43.640 --> 00:04:44.240
around Python.
00:04:44.580 --> 00:04:46.800
I think it was 3.13, 3.12.
00:04:47.090 --> 00:04:48.640
I can't remember which one, but that was a lot of fun.
00:04:48.780 --> 00:04:50.500
Are you involved in the release this year?
00:04:51.460 --> 00:04:53.580
I mean, from the sidelines, I still release my versions,
00:04:53.770 --> 00:04:54.560
3.10 and 3.11,
00:04:54.900 --> 00:04:57.460
but the release manager right now is Hugo Pankeminade,
00:04:57.960 --> 00:05:00.060
and the next is going to be Savannah,
00:05:01.700 --> 00:05:04.940
which is going to be the new release manager for 3.16 and 3.17.
00:05:05.320 --> 00:05:06.520
So I just help in.
00:05:06.990 --> 00:05:07.520
I mean, there's always,
00:05:07.680 --> 00:05:07.800
like,
00:05:07.980 --> 00:05:08.620
things to fix,
00:05:08.770 --> 00:05:12.780
and certainly since I'm the author of a bunch of stabbing 3.15,
00:05:13.300 --> 00:05:14.340
quite a lot of stuff actually,
00:05:14.620 --> 00:05:16.280
like, you know, lazy imports and the profiler.
00:05:16.540 --> 00:05:18.060
So like there's a bunch of broken stuff.
00:05:19.340 --> 00:05:21.100
So yeah, that does, that's,
00:05:21.700 --> 00:05:22.440
you know, I'll help it.
00:05:22.460 --> 00:05:22.540
Yeah.
00:05:22.960 --> 00:05:23.020
Yeah.
00:05:23.120 --> 00:05:24.000
So apart from that, I don't know,
00:05:24.160 --> 00:05:24.560
I,
00:05:24.760 --> 00:05:25.600
you know, same things.
00:05:25.840 --> 00:05:27.540
I think the last time are still true,
00:05:27.800 --> 00:05:28.420
you know, core developer.
00:05:28.800 --> 00:05:29.840
I was still in the stream console.
00:05:30.080 --> 00:05:31.440
I think I'm furniture at this point.
00:05:31.620 --> 00:05:33.400
I feel like I've been there for like six years already.
00:05:33.560 --> 00:05:34.500
So I don't know.
00:05:34.860 --> 00:05:37.160
Maybe it's time to, to, to, to reconsider.
00:05:38.500 --> 00:05:41.000
But yeah, I think the only thing is new is that I changed employers.
00:05:41.260 --> 00:05:42.360
I used to work for Bloomberg,
00:05:42.720 --> 00:05:43.860
which is with Laszlo.
00:05:43.910 --> 00:05:44.600
I used to work there.
00:05:45.030 --> 00:05:45.600
And this actually,
00:05:45.760 --> 00:05:48.240
the provider work started when I was at Bloomberg.
00:05:48.730 --> 00:05:50.380
But now I'm, I betrayed Bloomberg.
00:05:50.800 --> 00:05:52.800
And now I'm in Hudson River trading.
00:05:53.300 --> 00:05:53.940
Hudson River trading.
00:05:54.240 --> 00:05:54.440
Bam.
00:05:55.100 --> 00:05:57.440
Doing like fast trading.
00:05:57.860 --> 00:05:58.340
Lots of money.
00:05:58.560 --> 00:05:58.780
Yes, yes.
00:05:59.120 --> 00:05:59.960
And that's very exciting.
00:06:00.120 --> 00:06:00.300
Also,
00:06:00.580 --> 00:06:01.580
probably a little high stress,
00:06:01.770 --> 00:06:02.220
but that's okay.
00:06:02.540 --> 00:06:03.160
No, I'm chill.
00:06:03.600 --> 00:06:05.180
Like, you know, they let me record this podcast.
00:06:06.020 --> 00:06:06.100
Well,
00:06:06.280 --> 00:06:08.260
I'm just meaning like, if you remove a lot of money,
00:06:08.460 --> 00:06:10.300
that is a nerve wracking thing.
00:06:10.360 --> 00:06:23.920
It's more like the stress I think is more like what happens when you screw up because like a bloomer I think you screw up and you know maybe your manager says bad bad bad developer you know but like it's fine and I can talk about bloomer this way because I'm not worried anymore.
00:06:25.400 --> 00:06:26.500
But sorry guys.
00:06:26.560 --> 00:06:28.320
You're going to make this interesting for me Pablo.
00:06:28.410 --> 00:06:28.540
Yes.
00:06:28.960 --> 00:06:29.680
Sorry Chime.
00:06:30.960 --> 00:06:31.480
This guy.
00:06:31.900 --> 00:06:31.980
Anyway.
00:06:33.080 --> 00:06:38.640
But yeah here is like when you screw up there is a very scary number that comes with the build you know.
00:06:38.700 --> 00:06:41.000
and like your little deployment failure
00:06:41.580 --> 00:06:44.280
caused like sort of big numbers.
00:06:44.540 --> 00:06:45.780
So you're like, shit,
00:06:46.740 --> 00:06:46.980
damn.
00:06:47.680 --> 00:06:48.320
So when that number,
00:06:48.620 --> 00:06:49.560
you know, it's very high,
00:06:49.720 --> 00:06:49.900
then,
00:06:50.160 --> 00:06:50.780
you know, so it's scary.
00:06:51.040 --> 00:06:51.120
So,
00:06:51.280 --> 00:06:52.340
yeah, I don't know.
00:06:52.360 --> 00:06:53.060
It keeps interesting.
00:06:53.520 --> 00:06:53.940
I find it.
00:06:53.940 --> 00:06:53.980
Yeah,
00:06:54.180 --> 00:06:55.780
but after you release Python
00:06:56.200 --> 00:06:59.580
to how many tens of millions or more people
00:06:59.900 --> 00:07:02.220
and platforms and it's low state.
00:07:02.320 --> 00:07:02.740
I mean,
00:07:03.040 --> 00:07:04.360
that's also a big deal.
00:07:04.360 --> 00:07:04.880
I'm used to it.
00:07:04.980 --> 00:07:05.480
I'm used to it.
00:07:05.680 --> 00:07:08.340
This is just like high speed.
00:07:08.640 --> 00:07:09.000
high speed.
00:07:09.190 --> 00:07:09.960
Like in Python,
00:07:10.140 --> 00:07:10.820
when you screw up,
00:07:11.240 --> 00:07:13.260
you know, it's after a year that people use your thing
00:07:13.270 --> 00:07:13.940
and they say, oh,
00:07:14.210 --> 00:07:14.400
you know,
00:07:14.680 --> 00:07:16.680
lazy imports is bad. And then you say, damn.
00:07:17.080 --> 00:07:18.000
They're not going to say
00:07:18.030 --> 00:07:19.420
that. Lazy imports are going to be awesome.
00:07:19.780 --> 00:07:20.580
I'm for it.
00:07:20.920 --> 00:07:21.900
That's it. That's it.
00:07:22.280 --> 00:07:23.560
One of us. One of us.
00:07:24.080 --> 00:07:24.400
But here,
00:07:24.600 --> 00:07:24.960
you know, like,
00:07:25.180 --> 00:07:27.520
they will tell you the millisecond it fails.
00:07:27.680 --> 00:07:28.440
So you will know.
00:07:28.650 --> 00:07:28.940
But yeah,
00:07:29.280 --> 00:07:30.040
last look. And,
00:07:30.250 --> 00:07:30.440
you know,
00:07:30.980 --> 00:07:31.580
also,
00:07:32.050 --> 00:07:34.860
quick shout out to core.py. That's a fun. Oh, thank you. I forgot.
00:07:34.920 --> 00:07:35.940
I only have one job.
00:07:36.100 --> 00:07:36.880
I only have one job.
00:07:37.000 --> 00:07:37.760
I failed at the job.
00:07:38.100 --> 00:07:38.260
Sorry,
00:07:38.460 --> 00:07:38.540
Lucas.
00:07:38.700 --> 00:07:38.860
Sorry,
00:07:39.020 --> 00:07:39.100
Lucas.
00:07:40.060 --> 00:07:40.700
Yeah, Lucas,
00:07:40.980 --> 00:07:41.200
run that.
00:07:41.320 --> 00:07:42.860
That's a great, that's a fun deep dive.
00:07:43.100 --> 00:07:44.740
Like core developer internals.
00:07:45.020 --> 00:07:45.200
Yes,
00:07:45.360 --> 00:07:45.440
yes.
00:07:45.900 --> 00:07:46.500
Listen to us.
00:07:46.580 --> 00:07:47.300
We're still alive.
00:07:47.700 --> 00:07:52.960
Lucas just like joined Meta and Mark Zuckerberg himself doesn't allow him to record that often.
00:07:53.240 --> 00:07:57.200
Like that, when he joined Meta, Mark called him and was like, you cannot just, you cannot
00:07:57.320 --> 00:07:58.580
just do this podcast often.
00:07:58.760 --> 00:07:59.800
So like now we cannot do it.
00:07:59.800 --> 00:08:01.960
It's like, if you're not going to have me on the show to talk about AI,
00:08:02.120 --> 00:08:02.820
we can do it.
00:08:03.000 --> 00:08:03.260
Yeah,
00:08:03.480 --> 00:08:03.620
exactly.
00:08:03.920 --> 00:08:04.900
So, but we have integrity.
00:08:04.970 --> 00:08:07.060
We say, no, Mark, you cannot be on Corel.py
00:08:07.100 --> 00:08:08.720
because you don't talk about Python.
00:08:09.900 --> 00:08:11.460
It's not a PHP show, man.
00:08:11.550 --> 00:08:11.800
Come on.
00:08:11.980 --> 00:08:12.200
Exactly.
00:08:12.330 --> 00:08:12.420
Yeah,
00:08:12.580 --> 00:08:12.760
yeah.
00:08:13.280 --> 00:08:13.740
Those guys.
00:08:14.880 --> 00:08:15.340
So, yeah, yeah.
00:08:15.450 --> 00:08:17.280
So we're still alive,
00:08:17.700 --> 00:08:18.380
just slower.
00:08:18.960 --> 00:08:20.920
But listen to Corel.py in all platforms.
00:08:21.480 --> 00:08:21.740
Solid.
00:08:22.040 --> 00:08:23.020
Lazlo, welcome to the show.
00:08:23.280 --> 00:08:23.660
Tell us about yourself.
00:08:24.000 --> 00:08:24.360
Thanks.
00:08:24.950 --> 00:08:25.640
Happy to be here.
00:08:25.950 --> 00:08:26.040
So,
00:08:27.210 --> 00:08:28.400
yeah, so we've been working.
00:08:29.220 --> 00:08:31.240
I have worked with Pablo for quite a few years.
00:08:31.820 --> 00:08:31.860
Originally,
00:08:33.360 --> 00:08:34.280
I think since 2018,
00:08:34.840 --> 00:08:38.340
we started in the Python team together in Lundberg.
00:08:38.780 --> 00:08:42.020
And up until the end of last year, we've worked together.
00:08:43.320 --> 00:08:48.100
I actually participated in some of the projects like PyStack and Memory internally,
00:08:49.180 --> 00:08:51.860
not nearly as much to the extent as in Tachyon.
00:08:52.540 --> 00:08:55.820
But then we started collaborating with Pablo last year.
00:08:56.620 --> 00:08:57.520
And I remember this because,
00:08:57.720 --> 00:09:01.380
you know, probably just mentioned, like, it seems like 315 years old.
00:09:01.580 --> 00:09:04.540
I remember that we started working on this in the Sprint last year.
00:09:04.540 --> 00:09:05.440
Picon last year.
00:09:05.540 --> 00:09:05.780
Yeah, yeah.
00:09:06.240 --> 00:09:06.320
Crazy.
00:09:06.720 --> 00:09:07.060
April,
00:09:07.300 --> 00:09:07.580
May.
00:09:08.020 --> 00:09:10.820
And it just seems like so far away.
00:09:11.140 --> 00:09:13.040
And it's crazy that it's still not out.
00:09:14.100 --> 00:09:15.880
But yeah, so that's where everything started.
00:09:16.140 --> 00:09:18.320
But now that you have this thing fresh,
00:09:18.500 --> 00:09:19.180
like now, right?
00:09:19.240 --> 00:09:20.140
Like the last year,
00:09:20.300 --> 00:09:21.740
you know, we were young and full of hope.
00:09:21.920 --> 00:09:26.100
But like imagine the time when Python releases were every two years.
00:09:26.540 --> 00:09:26.840
Oh, yeah.
00:09:26.980 --> 00:09:27.380
Damn.
00:09:27.840 --> 00:09:27.920
Damn.
00:09:28.220 --> 00:09:29.020
Because you think, oh,
00:09:29.200 --> 00:09:29.300
man,
00:09:29.460 --> 00:09:29.720
312,
00:09:29.900 --> 00:09:30.260
313.
00:09:30.460 --> 00:09:30.580
No,
00:09:30.760 --> 00:09:30.940
man.
00:09:31.300 --> 00:09:32.740
from 3.8 to 3.9.
00:09:33.280 --> 00:09:34.260
Well, 3.8 was the first one,
00:09:34.360 --> 00:09:36.540
but 3.7 to 3.8 was two years.
00:09:36.900 --> 00:09:38.460
You forget everything about the feature.
00:09:38.920 --> 00:09:42.060
You introduced ramp by the time it's released.
00:09:42.620 --> 00:09:42.780
Exactly.
00:09:43.100 --> 00:09:44.240
Do you guys think it should go faster?
00:09:44.640 --> 00:09:44.820
Yeah.
00:09:45.700 --> 00:09:47.220
Do you think there should be a six months
00:09:47.280 --> 00:09:49.640
or every three months version of Python that comes out?
00:09:50.760 --> 00:09:51.160
Or is it not?
00:09:51.900 --> 00:09:54.560
I think from the usability thing,
00:09:55.000 --> 00:09:57.420
I think I will do rolling releases if I could,
00:09:57.660 --> 00:10:00.079
but the problem is that CPython is too big
00:10:00.100 --> 00:10:01.220
to like do that.
00:10:01.460 --> 00:10:03.920
In the sense that stability is really hard.
00:10:04.300 --> 00:10:05.880
In the sense that ideally in the ideal world,
00:10:06.000 --> 00:10:07.280
you merge things that aren't ready,
00:10:07.500 --> 00:10:07.600
right?
00:10:07.660 --> 00:10:08.360
And then you don't like,
00:10:08.700 --> 00:10:10.760
you know, merge half bike stuff.
00:10:10.880 --> 00:10:11.080
But like,
00:10:11.440 --> 00:10:11.600
you know,
00:10:12.020 --> 00:10:13.000
for a software this complex,
00:10:13.500 --> 00:10:14.960
I like that affects so much things
00:10:15.260 --> 00:10:16.600
and like so many people depend on it.
00:10:16.680 --> 00:10:17.960
Of course, like people are going to run
00:10:17.980 --> 00:10:19.240
on the latest one, right?
00:10:19.340 --> 00:10:21.360
But like, it's very unfortunate.
00:10:21.600 --> 00:10:23.380
And from this thought that is obviously wrong
00:10:23.839 --> 00:10:25.700
to like the one year, two year release,
00:10:25.760 --> 00:10:26.440
there is a continuum,
00:10:26.660 --> 00:10:26.780
right?
00:10:26.860 --> 00:10:28.040
So like, where do you put the line?
00:10:28.480 --> 00:10:35.100
But in our experience in the core team so far is that one year is already like running to the edges of,
00:10:35.880 --> 00:10:37.320
you know, how much we can ensure.
00:10:38.180 --> 00:10:41.460
Because like there's always some last time scare.
00:10:41.800 --> 00:10:42.040
You know,
00:10:42.260 --> 00:10:43.220
there's always like, oh,
00:10:43.600 --> 00:10:46.180
you know, for example, in Python 3.12,
00:10:46.440 --> 00:10:50.340
we have to revert an entire new GC implementation.
00:10:50.800 --> 00:10:50.900
Right.
00:10:51.120 --> 00:10:53.360
And now in Python 3.14,
00:10:54.080 --> 00:10:54.900
sorry, that was 3.13.
00:10:55.040 --> 00:10:55.920
In Python 3.14,
00:10:56.180 --> 00:10:57.340
we have to revert it again.
00:10:57.800 --> 00:11:02.640
So like, because it was like having a huge amount of memory impact and that was only
00:11:02.860 --> 00:11:04.260
known after people started to use it.
00:11:04.620 --> 00:11:09.640
So that gave us like some information that perhaps we are, or either way we need to look
00:11:09.640 --> 00:11:10.060
at the process,
00:11:10.340 --> 00:11:11.080
which we are of course,
00:11:11.300 --> 00:11:14.180
but like, or one year is already like on the line
00:11:14.280 --> 00:11:15.160
of what is responsible.
00:11:15.560 --> 00:11:15.720
So,
00:11:16.120 --> 00:11:19.040
you know, it would be amazing if it's like rolling release and then you can get like
00:11:19.120 --> 00:11:22.520
the features the next day, but not a possibility.
00:11:23.140 --> 00:11:23.220
Yeah.
00:11:23.500 --> 00:11:24.880
And I guess it doesn't change.
00:11:25.300 --> 00:11:26.880
One of the things that's really nice about Python,
00:11:27.360 --> 00:11:27.560
honestly,
00:11:28.040 --> 00:11:32.340
is there's almost never a reason not to run on the new lease.
00:11:32.780 --> 00:11:33.900
It's very,
00:11:34.120 --> 00:11:34.580
very stable.
00:11:34.780 --> 00:11:40.300
Only once have I had a major release of Python break any of my apps.
00:11:40.560 --> 00:11:45.020
And the reason was the async coroutine decorator was removed
00:11:45.860 --> 00:11:49.660
in lieu of async and await only for async funks, something like that.
00:11:50.080 --> 00:11:51.280
Oh, I also didn't look at it.
00:11:51.580 --> 00:11:51.840
Yeah.
00:11:52.560 --> 00:11:58.220
And one of the libraries I was using for the database way deep down still used that.
00:11:58.400 --> 00:11:59.400
And I didn't know.
00:11:59.560 --> 00:12:00.820
But when I rolled out the new version,
00:12:00.980 --> 00:12:02.700
it's like, what are you talking about, decorator
00:12:03.300 --> 00:12:03.720
async code routine?
00:12:03.780 --> 00:12:05.240
I'm like, it just doesn't run.
00:12:05.370 --> 00:12:05.780
I don't know.
00:12:05.900 --> 00:12:06.360
This isn't great.
00:12:07.020 --> 00:12:07.980
But that's not what happens.
00:12:08.050 --> 00:12:10.560
And I guess the faster you go, the more likely that is to happen, right?
00:12:10.880 --> 00:12:11.080
Yeah.
00:12:11.190 --> 00:12:12.180
I think you're also lucky.
00:12:12.270 --> 00:12:13.320
I think like for,
00:12:13.880 --> 00:12:14.300
I mean, I don't know.
00:12:14.400 --> 00:12:15.340
I don't want to assume.
00:12:15.350 --> 00:12:17.080
But I suppose my experience,
00:12:17.490 --> 00:12:17.780
for instance,
00:12:17.960 --> 00:12:20.120
working at Bloomberg and also here is not
00:12:20.120 --> 00:12:20.540
really the same.
00:12:21.160 --> 00:12:22.420
when your software is big enough,
00:12:22.770 --> 00:12:25.860
you will run into the sync decorators all the time.
00:12:26.080 --> 00:12:26.480
You know what I mean?
00:12:27.500 --> 00:12:28.760
And in the weirdest cases,
00:12:29.020 --> 00:12:30.920
normally it's not even just Python.
00:12:31.090 --> 00:12:32.000
It's like, oh,
00:12:32.340 --> 00:12:37.240
this Python 3.15 requires you to upgrade your Pandas data science stack
00:12:37.330 --> 00:12:41.700
because the NumPy only supports 3.15 in version 2.5, right?
00:12:41.810 --> 00:12:42.960
So if you're running version 2,
00:12:43.180 --> 00:12:45.540
that forces you to move 0.5,
00:12:45.860 --> 00:12:47.980
sorry, five versions of NumPy or whatever it is.
00:12:48.090 --> 00:12:50.540
And then, I don't know, you run into all sorts of things,
00:12:50.700 --> 00:12:50.780
right?
00:12:50.860 --> 00:12:51.340
Like here,
00:12:51.540 --> 00:12:56.560
the work I do right now in HRT, we have a lot of numeric work, for instance.
00:12:56.890 --> 00:13:01.640
And there is times when we upgrade the interpreter and nothing breaks in the sense that,
00:13:01.900 --> 00:13:04.340
you know, this function doesn't exist or this doesn't work.
00:13:04.440 --> 00:13:04.900
It's like, no,
00:13:05.340 --> 00:13:07.360
like the floating points are slightly off,
00:13:07.700 --> 00:13:10.700
you know, like so the simulation is slightly off.
00:13:11.020 --> 00:13:11.100
And,
00:13:11.770 --> 00:13:16.940
you know, you chase that down and it's like pandas changed slightly the algorithm and like nothing,
00:13:17.400 --> 00:13:18.240
you know.
00:13:19.240 --> 00:13:25.820
like maybe the compiler is slightly different or something or like a compiler optimization flag
00:13:26.840 --> 00:13:32.980
so people you know they still are relatively major events even if in the core team we've been working
00:13:32.980 --> 00:13:37.960
with for instance numpy and pandas to ensure that they have wheels now on release so when you you
00:13:37.960 --> 00:13:43.800
know when you start 315 uh and they won we won for can use no buy and it used to be not true right
00:13:43.940 --> 00:13:48.660
3.8 that was not a thing but now it is yeah it's certainly that uh the deploying of packages is
00:13:49.040 --> 00:13:52.440
I want to talk about a couple things as a way to kind of introduce,
00:13:52.900 --> 00:13:56.980
work our way into performance and profiling and so on.
00:13:57.380 --> 00:14:00.540
And since you brought it up, let's talk real quick about lazy imports,
00:14:00.700 --> 00:14:02.500
explicit lazy imports.
00:14:03.260 --> 00:14:03.620
Explicit.
00:14:03.960 --> 00:14:04.040
Yeah.
00:14:04.690 --> 00:14:06.600
Well, by the way, did you consider not explicit?
00:14:07.000 --> 00:14:07.540
Of course, yeah.
00:14:07.950 --> 00:14:08.840
And that was rejected.
00:14:09.520 --> 00:14:10.360
I wonder why.
00:14:11.060 --> 00:14:12.800
So what does the syntax even look like?
00:14:13.220 --> 00:14:13.640
Yeah, yeah.
00:14:13.800 --> 00:14:16.700
So you just put lazy import to your import and bam,
00:14:16.880 --> 00:14:17.620
your problem is faster.
00:14:17.920 --> 00:14:18.400
Isn't that cool?
00:14:18.680 --> 00:14:18.760
Well,
00:14:19.060 --> 00:14:19.280
asterisk,
00:14:19.440 --> 00:14:22.220
no, but that's the selling point.
00:14:23.040 --> 00:14:23.140
Well,
00:14:23.360 --> 00:14:23.460
okay.
00:14:23.720 --> 00:14:25.120
The guess is that, you know,
00:14:26.060 --> 00:14:26.400
so Python,
00:14:26.660 --> 00:14:29.120
one of the powers of Python is very unsurprising
00:14:29.220 --> 00:14:29.840
is packaging,
00:14:30.220 --> 00:14:30.320
right?
00:14:30.400 --> 00:14:32.020
Like you have a package for everything,
00:14:32.760 --> 00:14:33.900
but guess what?
00:14:34.220 --> 00:14:39.780
Those packages also use packages for everything and so on and so forth until you download is
00:14:40.060 --> 00:14:40.160
even,
00:14:40.340 --> 00:14:43.280
you know, and like just to check if a number is even, you just download a package
00:14:43.320 --> 00:14:43.660
or something.
00:14:44.020 --> 00:14:47.560
So what happens is that because this library is packaged a bunch of functionality,
00:14:48.000 --> 00:14:51.520
sometimes when you only want to reach for a tiny function in some package,
00:14:51.760 --> 00:14:54.040
you end importing the entire thing, right?
00:14:54.180 --> 00:14:58.700
And the entire thing will end importing the entire thing of other packages and so on and so forth.
00:14:59.000 --> 00:15:02.320
So it's very common that just by importing a tiny function,
00:15:02.880 --> 00:15:05.080
you end paying for the entire world.
00:15:05.240 --> 00:15:06.840
And some of those imports are really slow,
00:15:07.180 --> 00:15:07.340
unfortunately,
00:15:07.780 --> 00:15:14.480
because like they need to load big files or anyone that has dealt with Torch knows how chunky the video wheels are and whatnot.
00:15:15.020 --> 00:15:15.320
So,
00:15:15.740 --> 00:15:17.800
you know, that's bad because you should only pay for what you use.
00:15:17.950 --> 00:15:21.320
So the idea of explicit lazy imports is that, you know,
00:15:21.600 --> 00:15:23.400
if people start to mark the imports as lazy,
00:15:23.780 --> 00:15:26.400
that you will only pay for the imports that you really need.
00:15:26.470 --> 00:15:28.220
So only when you actually need that thing,
00:15:28.410 --> 00:15:29.320
then it will be imported.
00:15:29.780 --> 00:15:35.460
And that transforms your mega graph of imports of things that you don't really need into only the ones that you need.
00:15:35.510 --> 00:15:37.340
And that saves you a lot of time and memory,
00:15:37.600 --> 00:15:37.680
right?
00:15:37.780 --> 00:15:39.540
Because loading those things is not free.
00:15:40.080 --> 00:15:40.620
Not only in time,
00:15:40.780 --> 00:15:40.860
right?
00:15:40.900 --> 00:15:41.540
It's in memory as well.
00:15:41.760 --> 00:15:42.000
Yeah.
00:15:42.510 --> 00:15:42.600
Honestly,
00:15:43.410 --> 00:15:44.940
I think, I mean, there are,
00:15:45.120 --> 00:15:46.280
speaking of profilers,
00:15:46.440 --> 00:15:47.380
there are some tools.
00:15:47.780 --> 00:15:48.960
Now, this is not a very popular one.
00:15:49.020 --> 00:15:49.360
It's very old.
00:15:49.390 --> 00:15:50.300
I don't know if it even works.
00:15:50.500 --> 00:15:55.400
But there are some tools to do things like profile import time and so on.
00:15:55.760 --> 00:15:57.080
Certainly the time can matter.
00:15:57.400 --> 00:15:58.140
I guess it matters.
00:15:58.680 --> 00:15:59.720
It depends on what you're doing.
00:16:00.070 --> 00:16:01.240
If you're doing serverless,
00:16:01.440 --> 00:16:04.600
if you're doing CLIs, if you're doing subprocessing,
00:16:04.990 --> 00:16:05.100
those
00:16:05.360 --> 00:16:08.700
kinds of things really matter because it's spin up the thing, do a little bit of work
00:16:08.800 --> 00:16:09.260
and go away.
00:16:09.580 --> 00:16:10.400
If it's a web app,
00:16:10.640 --> 00:16:12.640
you know, startup and then it just chills,
00:16:12.880 --> 00:16:13.000
right?
00:16:13.060 --> 00:16:14.680
It doesn't really matter in terms of time.
00:16:14.850 --> 00:16:17.340
But I'll tell you where it does matter a lot is memory.
00:16:17.640 --> 00:16:17.880
Right,
00:16:18.080 --> 00:16:18.160
right.
00:16:18.260 --> 00:16:23.180
I was on a mission to reduce how much memory it takes to run like the Talk Python courses
00:16:23.360 --> 00:16:24.460
site and all the different things.
00:16:24.920 --> 00:16:30.140
And I was surprised a couple hundred megabytes were ports that were rarely used.
00:16:30.700 --> 00:16:34.120
So maybe if I run some back-end admin,
00:16:34.640 --> 00:16:37.940
it would pull up NumPy, Matplotlib, and other types of stuff, right?
00:16:38.300 --> 00:16:39.380
But otherwise it wouldn't.
00:16:39.600 --> 00:16:41.900
And that added at least 100 megabytes to the runtime.
00:16:42.480 --> 00:16:44.700
And because the way the imports worked, everything was eager.
00:16:45.120 --> 00:16:47.220
It basically imported everything at startup.
00:16:47.480 --> 00:16:49.760
So just by having those import statements there,
00:16:49.920 --> 00:16:53.000
the app was using 100 megabytes more than it needed.
00:16:53.200 --> 00:16:55.780
I changed them to the best I could of lazy imports,
00:16:55.960 --> 00:16:59.000
where you just import in the function instead of the top of the file.
00:16:59.040 --> 00:16:59.940
Lazy imports at home.
00:17:00.600 --> 00:17:00.700
Yeah,
00:17:01.120 --> 00:17:02.140
like homegrown,
00:17:02.320 --> 00:17:03.320
homebrew lazy imports.
00:17:04.199 --> 00:17:07.500
And it really dropped the memory by 100 megabytes.
00:17:07.620 --> 00:17:10.740
And on the server, that's one of the most expensive things is RAM,
00:17:11.000 --> 00:17:11.600
not compute.
00:17:12.339 --> 00:17:15.640
I was going to say that there's one more, I think, super cool area
00:17:15.800 --> 00:17:17.920
where lazy imports help a lot,
00:17:18.079 --> 00:17:19.060
which is CLI tools,
00:17:19.860 --> 00:17:22.180
which typically can be, and we've seen a lot of this,
00:17:22.360 --> 00:17:26.640
And I know with Pablo, we actually even looked at some stuff at work in the past where once
00:17:26.640 --> 00:17:28.820
you have a complex enough application,
00:17:29.420 --> 00:17:31.360
but you literally just want to see the help page
00:17:31.600 --> 00:17:34.940
or just spin something up and it can take seconds.
00:17:35.400 --> 00:17:39.200
And lazy imports can make a massive difference in that sort of the responsiveness of that
00:17:39.300 --> 00:17:39.680
CLI tool.
00:17:40.040 --> 00:17:40.220
Right.
00:17:40.480 --> 00:17:45.640
I was going to just add that there is also second and third order effects that people
00:17:45.800 --> 00:17:46.380
think about,
00:17:46.540 --> 00:17:49.180
but depending on the size of your thing can be quite dire.
00:17:49.480 --> 00:17:50.440
Like one story, for instance,
00:17:50.660 --> 00:17:55.240
for my current employer is that even if you don't care about memory,
00:17:55.400 --> 00:17:57.040
let's say you have all the memory in the world, right?
00:17:57.680 --> 00:17:59.620
And let's say you don't care about the startup tag,
00:17:59.870 --> 00:18:00.740
because like you said, like,
00:18:01.360 --> 00:18:02.380
you know, you have a web app,
00:18:02.640 --> 00:18:02.740
right?
00:18:02.740 --> 00:18:04.500
And like, you just start your thing and that's fine.
00:18:05.220 --> 00:18:13.900
What can happen still is that if you import a lot of stuff that you don't use, all those imports will bring into existence a bunch of like a lot of Python objects.
00:18:14.140 --> 00:18:15.300
And those objects can be classes,
00:18:15.760 --> 00:18:16.920
functions,
00:18:17.600 --> 00:18:18.280
constants,
00:18:19.480 --> 00:18:21.180
and, you know, like all sorts of things.
00:18:21.270 --> 00:18:23.360
And those things will be more and more and more and more.
00:18:23.500 --> 00:18:26.160
And so you end with a lot of Python objects in your,
00:18:26.420 --> 00:18:28.920
you know, in your program that you don't use, right?
00:18:29.340 --> 00:18:33.600
And those things have non-zero costs because every time the garbage collector runs,
00:18:33.930 --> 00:18:38.140
it will, like, look at all those things and then every run of the garbage collector will be slower.
00:18:38.760 --> 00:18:42.899
Of course, like, you know, you have a web app that is a solution like the Instagram GC Freeze
00:18:43.340 --> 00:18:44.220
that they implemented,
00:18:44.520 --> 00:18:46.360
but this will happen also, you don't have web apps.
00:18:46.470 --> 00:18:50.500
Like you have like even like the case of my employee, like scientific computing,
00:18:51.100 --> 00:18:54.440
and you are crunching out of numbers just by importing stuff that you don't use,
00:18:54.490 --> 00:18:55.820
those numbers will be crunched slower.
00:18:55.990 --> 00:19:00.560
And you cannot just DC freeze because you don't have like a fork model
00:19:00.760 --> 00:19:02.460
that then you run your workers,
00:19:02.740 --> 00:19:02.820
right?
00:19:03.450 --> 00:19:07.820
So importing less will make these objects not exist,
00:19:08.020 --> 00:19:10.080
and therefore the app will run more linear.
00:19:10.520 --> 00:19:11.140
That's the key, right?
00:19:11.340 --> 00:19:14.640
So there is like very interesting second and third order effects to this.
00:19:15.340 --> 00:19:18.040
This portion of Talk Python is brought to you by Sentry.
00:19:18.280 --> 00:19:20.860
Now, I've told you about Sentry many times on the show,
00:19:21.340 --> 00:19:24.980
but this time I have something new that should be very exciting to you.
00:19:25.540 --> 00:19:28.180
Sentry has recently released their MCP server.
00:19:28.680 --> 00:19:29.560
And for me personally,
00:19:29.800 --> 00:19:32.060
this has truly unlocked a new level of integration
00:19:32.300 --> 00:19:33.340
and workflow with Sentry.
00:19:33.820 --> 00:19:34.180
You see,
00:19:34.720 --> 00:19:36.660
Sentry is integrated into Talk Python,
00:19:37.260 --> 00:19:37.560
courses,
00:19:38.020 --> 00:19:38.220
website,
00:19:38.640 --> 00:19:39.340
the mobile app,
00:19:39.440 --> 00:19:42.280
and a whole bunch of things behind the scenes no one ever sees.
00:19:43.240 --> 00:19:44.980
When something goes wrong in any of these apps,
00:19:45.160 --> 00:19:46.560
I'm alerted immediately over email,
00:19:46.740 --> 00:19:49.480
and I can log into Sentry and see all the details of what happened,
00:19:49.780 --> 00:19:51.380
the parameters that were passed to functions,
00:19:51.960 --> 00:19:53.900
what user encountered this error, and much more.
00:19:54.700 --> 00:19:55.100
Traditionally,
00:19:55.480 --> 00:19:57.780
what I would do is I would read the stack trace,
00:19:58.340 --> 00:19:59.100
open up the source code,
00:19:59.280 --> 00:20:01.180
try to piece together the detective story
00:20:01.200 --> 00:20:03.140
that would solve what was wrong with my code.
00:20:03.540 --> 00:20:04.240
Once I figured it out,
00:20:04.400 --> 00:20:06.560
I would fix the code and push out a new version.
00:20:07.180 --> 00:20:07.980
But you know what's coming,
00:20:08.160 --> 00:20:08.240
right?
00:20:08.800 --> 00:20:09.460
It's 2026.
00:20:09.820 --> 00:20:13.440
This is the kind of work that AI is spectacularly good at.
00:20:14.340 --> 00:20:18.120
Now when there's an error, I simply open up Claude Code with the project that had the error,
00:20:18.560 --> 00:20:21.060
and I tell Claude, we have new errors in Sentry.
00:20:21.280 --> 00:20:23.260
Please investigate them, and if you can,
00:20:23.630 --> 00:20:23.960
solve them.
00:20:24.320 --> 00:20:28.500
Claude immediately knows to use Sentry's MCP server because I connected it to
00:20:28.640 --> 00:20:29.180
Claude Code.
00:20:29.400 --> 00:20:31.020
It investigates all the errors,
00:20:31.580 --> 00:20:32.920
uses the information I discussed,
00:20:33.180 --> 00:20:33.740
and much more,
00:20:33.900 --> 00:20:35.660
as well as the code base it's working in.
00:20:36.360 --> 00:20:41.120
And 99% of the time, it fixes this without another instruction from me.
00:20:41.390 --> 00:20:46.140
The magic of Sentry's MCP server is that it gives you full Sentry error reporting
00:20:46.190 --> 00:20:50.620
and management directly connected to Frontier models running inside your code base.
00:20:51.160 --> 00:20:53.260
If you use Agentic AI for your code at all,
00:20:53.370 --> 00:20:57.120
you definitely need to check out Sentry and connect Sentry's MCP server
00:20:57.340 --> 00:20:58.540
straight to your favorite coding agent.
00:20:59.060 --> 00:21:00.280
Get started today at Sentry.
00:21:00.370 --> 00:21:02.500
Be sure to use our link and discount code.
00:21:03.040 --> 00:21:05.700
It's talkpython.fm/sentry,
00:21:05.920 --> 00:21:07.980
code talkpython26.
00:21:08.880 --> 00:21:10.640
That's talkpython.fm/sentry.
00:21:10.880 --> 00:21:12.100
It gives you extra credits,
00:21:12.320 --> 00:21:12.880
but more importantly,
00:21:12.920 --> 00:21:14.480
it lets Sentry know you came from us.
00:21:14.980 --> 00:21:17.120
Thank you to Sentry for supporting Talk Python To Me.
00:21:17.940 --> 00:21:20.320
Python does a bunch to try to keep the memory together,
00:21:21.039 --> 00:21:23.380
arenas and blocks and all that kind of business.
00:21:23.700 --> 00:21:25.880
But yeah, if you allocate a whole bunch of pointers,
00:21:26.320 --> 00:21:28.400
all of a sudden the GC has to count them, right?
00:21:28.660 --> 00:21:28.740
Right.
00:21:28.880 --> 00:21:30.220
The Gen 2 collections, yeah.
00:21:30.560 --> 00:21:30.660
Okay.
00:21:31.120 --> 00:21:32.120
Well, I'm excited for it.
00:21:32.240 --> 00:21:38.340
So everyone get ready to write lazy import module instead of just import module and see where it takes you.
00:21:38.680 --> 00:21:38.800
Yes.
00:21:39.200 --> 00:21:39.480
All right.
00:21:39.880 --> 00:21:40.060
Also,
00:21:40.440 --> 00:21:43.440
let's talk faster CPython a little bit.
00:21:43.740 --> 00:21:43.960
All right.
00:21:44.170 --> 00:21:46.100
I mean, we're talking profiling.
00:21:46.350 --> 00:21:48.080
We're talking about making code faster.
00:21:48.530 --> 00:21:50.060
The Faster CPython initiative,
00:21:50.300 --> 00:21:52.999
even though it's gone through a little bit of organizational challenges.
00:21:54.100 --> 00:21:54.640
Drop my life,
00:21:54.820 --> 00:21:54.880
yeah.
00:21:55.180 --> 00:21:55.340
Yeah,
00:21:55.700 --> 00:22:00.300
like with the Microsoft pulling the support or the people off of the project or whatever.
00:22:00.960 --> 00:22:02.120
I think it's been a huge success.
00:22:02.320 --> 00:22:05.220
I think Python is so much faster in a lot of ways.
00:22:05.540 --> 00:22:06.160
And yeah,
00:22:06.340 --> 00:22:07.340
what are your thoughts?
00:22:07.680 --> 00:22:07.960
Either of you.
00:22:08.240 --> 00:22:08.760
Yeah, I think,
00:22:09.240 --> 00:22:09.460
I mean,
00:22:10.040 --> 00:22:11.140
it's pretty obvious
00:22:11.360 --> 00:22:13.140
if you look at like the performance of like,
00:22:13.360 --> 00:22:14.000
let's say 39,
00:22:14.600 --> 00:22:15.600
that was when this started,
00:22:15.780 --> 00:22:16.740
or like 310 perhaps,
00:22:17.280 --> 00:22:18.940
like on the performance of today's Python,
00:22:19.420 --> 00:22:20.840
like that has been a clear success.
00:22:21.580 --> 00:22:22.740
And this kind of continues.
00:22:22.880 --> 00:22:26.140
It's just that like we just run out of like obvious things.
00:22:26.260 --> 00:22:27.060
Obvious doesn't mean easy,
00:22:27.240 --> 00:22:27.320
right?
00:22:27.340 --> 00:22:28.780
But like obvious things to look at.
00:22:29.360 --> 00:22:29.780
And right now,
00:22:29.940 --> 00:22:32.940
the biggest efforts are into big chunks of this work,
00:22:33.080 --> 00:22:35.780
like JIT, which is like, you know, a big piece of software.
00:22:35.920 --> 00:22:37.540
I think you got Brandon in the show,
00:22:37.860 --> 00:22:38.040
perhaps,
00:22:38.960 --> 00:22:39.380
in the past,
00:22:39.680 --> 00:22:40.340
to talk about it.
00:22:40.680 --> 00:22:41.040
So now,
00:22:41.780 --> 00:22:44.660
you know, that's something that you cannot just commit in a couple of months.
00:22:45.340 --> 00:22:47.020
Pips are needed and all sorts of things.
00:22:47.180 --> 00:22:47.860
So right now,
00:22:48.240 --> 00:22:48.400
you know,
00:22:48.600 --> 00:22:50.900
the successor of that project is still happening.
00:22:51.340 --> 00:22:56.120
It's just that now those optimizations are much harder and require more careful consideration.
00:22:56.480 --> 00:23:03.540
And as you very well know, we are also very deep into the weeds of free threading and free thread Python dropping the GIL.
00:23:04.040 --> 00:23:08.360
And, you know, it turns out that those things kind of like don't play necessarily well together.
00:23:08.780 --> 00:23:12.080
So there is also like, you know, a lot of considerations there.
00:23:12.800 --> 00:23:14.460
And going to the topic at hand,
00:23:15.180 --> 00:23:18.120
profiling is really important to ensure that like none of this gets wrong.
00:23:18.220 --> 00:23:18.380
Right.
00:23:18.580 --> 00:23:22.800
And to also know that indeed your application became faster.
00:23:23.120 --> 00:23:23.480
Why?
00:23:23.780 --> 00:23:23.980
Because,
00:23:24.160 --> 00:23:24.260
like,
00:23:24.520 --> 00:23:24.780
you know,
00:23:25.000 --> 00:23:29.380
is this because Python 3.15 is going to be faster or is it because, like, NumPy is faster?
00:23:29.560 --> 00:23:30.180
Like, how do you know?
00:23:30.340 --> 00:23:31.620
Well, you need to use one of these tools,
00:23:31.820 --> 00:23:31.900
right?
00:23:32.100 --> 00:23:32.220
Yeah.
00:23:32.540 --> 00:23:32.860
Vaslo?
00:23:33.080 --> 00:23:33.300
Yeah,
00:23:33.460 --> 00:23:41.060
I mean, I watched the sort of the fastest SuperTor project from the sidewings for the past couple of years.
00:23:41.320 --> 00:23:42.400
And I think the,
00:23:42.560 --> 00:23:43.320
I actually remember,
00:23:43.480 --> 00:23:44.240
I think the biggest jump,
00:23:44.500 --> 00:23:45.960
was it 3.12?
00:23:46.020 --> 00:23:47.840
there was a very significant jump there.
00:23:48.190 --> 00:23:53.220
And it was very exciting because when we introduced that version,
00:23:53.290 --> 00:23:55.460
there was such a significant jump that whereas,
00:23:55.760 --> 00:23:56.340
you know,
00:23:57.400 --> 00:23:58.880
oftentimes you would have to convince folks,
00:23:58.970 --> 00:24:00.400
like you need to upgrade your Python versions,
00:24:00.640 --> 00:24:00.760
right?
00:24:01.000 --> 00:24:02.060
But when you have this carrot,
00:24:02.480 --> 00:24:06.560
where you have literally a 30-40% performance jump just for free,
00:24:06.730 --> 00:24:06.840
basically,
00:24:07.380 --> 00:24:08.780
that just made it a lot more appealing.
00:24:08.830 --> 00:24:11.320
And I think it opened up a lot more areas
00:24:11.430 --> 00:24:14.200
where Python became competitive and more appealing.
00:24:14.280 --> 00:24:14.940
So I think,
00:24:15.140 --> 00:24:22.600
yeah, I mean, I shared sort of what Paul mentioned about the complexity,
00:24:22.840 --> 00:24:24.720
sort of the JIT coming in,
00:24:25.460 --> 00:24:26.040
no-gil Python.
00:24:26.300 --> 00:24:29.340
I think it's quite a bit challenging now,
00:24:29.500 --> 00:24:31.940
I think, to look at these performance improvements.
00:24:32.900 --> 00:24:35.420
I mean, I have a lot of faith in the JIT compiler,
00:24:35.600 --> 00:24:38.520
you know, making significant performance improvements.
00:24:39.760 --> 00:24:43.600
I'm not sure exactly how exactly it's going to work.
00:24:44.000 --> 00:24:46.480
I have a lot of compilers as well, I think.
00:24:46.680 --> 00:24:47.460
Compilers are pretty awesome.
00:24:47.980 --> 00:24:50.420
Some stuff out of the C eval,
00:24:50.660 --> 00:24:51.940
which for statement,
00:24:52.220 --> 00:24:52.980
that would be good, you know?
00:24:53.100 --> 00:24:53.320
Yep.
00:24:53.400 --> 00:24:55.420
Yeah, and I think at the same time,
00:24:55.640 --> 00:25:03.480
it's also not forgetting that free trading Python has the potential to provide very significant speedups,
00:25:03.640 --> 00:25:03.740
right?
00:25:03.820 --> 00:25:06.600
It really depends on the application and the domain.
00:25:07.020 --> 00:25:17.900
But I think we can actually see very significant speedups and we can see like certain extensions and performance sensitive libraries can produce very significant speedups.
00:25:18.280 --> 00:25:22.620
And I think that's like sort of, I guess what we're seeing is now it's going into in multiple different directions,
00:25:22.940 --> 00:25:23.900
depending on your application.
00:25:24.530 --> 00:25:27.820
You can benefit from free threading potentially a lot more than,
00:25:28.280 --> 00:25:30.980
for example, waiting for a JIT profiler to go further.
00:25:31.320 --> 00:25:31.500
Yeah,
00:25:32.000 --> 00:25:33.460
I'm really excited about the free threading stuff.
00:25:33.580 --> 00:25:35.300
I think it has tons of potential,
00:25:35.760 --> 00:25:36.679
but I'm also concerned.
00:25:37.270 --> 00:25:43.700
I feel like a lot of the people building libraries and packages for Python have not deeply thought about race conditions.
00:25:44.300 --> 00:25:48.140
And I think just because they've assumed, well, the gil and generally single threaded,
00:25:48.420 --> 00:25:49.940
at least effectively,
00:25:50.150 --> 00:25:51.520
even if they have threads, so whatever.
00:25:52.020 --> 00:25:53.240
And then they're going to say, oh,
00:25:53.680 --> 00:25:54.080
it's free.
00:25:54.190 --> 00:25:56.560
We got these bug reports from free threading.
00:25:56.660 --> 00:25:57.480
So we're going to add locks.
00:25:57.570 --> 00:25:59.220
And then you're going to have deadlocks in your application.
00:25:59.420 --> 00:26:04.280
I just, I feel like there's going to be a couple of waves of growing pains and I don't know.
00:26:04.580 --> 00:26:04.760
Yeah.
00:26:05.520 --> 00:26:05.800
Yeah.
00:26:05.940 --> 00:26:09.360
And part of that is like intrinsic to the porn in particular,
00:26:09.820 --> 00:26:10.560
you know, there is a lot
00:26:10.560 --> 00:26:13.220
of jokes in the internet about like threads being complicated.
00:26:13.700 --> 00:26:17.500
And I think people in the Python ecosystem have not been exposed to what that really
00:26:17.660 --> 00:26:18.920
means because sure.
00:26:19.020 --> 00:26:19.080
Yeah.
00:26:19.190 --> 00:26:22.340
I mean, you, you, you have some problems with threads in Python, but like most of the real
00:26:22.440 --> 00:26:23.980
problems are not really exposed to you.
00:26:24.500 --> 00:26:25.180
While now,
00:26:25.680 --> 00:26:27.400
you know, there is two problems that you need to suffer.
00:26:27.460 --> 00:26:28.240
One of them is like,
00:26:28.640 --> 00:26:28.760
you know,
00:26:29.180 --> 00:26:32.860
you are not tall enough to do multiple threading, which was the joke,
00:26:33.020 --> 00:26:33.080
right?
00:26:33.480 --> 00:26:39.240
And then the other one is that, you know, free thread may be buggy just because there's bugs in the interpreter,
00:26:39.440 --> 00:26:39.920
hopefully not,
00:26:40.120 --> 00:26:42.200
but also maybe bugs in the packages,
00:26:42.500 --> 00:26:42.580
right?
00:26:43.139 --> 00:26:51.280
Or like the other problem that we can see a lot of people running into sometimes is that it doesn't scale linearly.
00:26:51.660 --> 00:26:52.240
Like it's not magic.
00:26:52.460 --> 00:26:52.580
Again,
00:26:52.780 --> 00:26:54.380
this is a matter of threading, it's not easy.
00:26:54.980 --> 00:27:00.440
and sometimes you know the cost of not crashing and the multiple threads is that extra checks
00:27:00.500 --> 00:27:04.940
need to be done and those extra checks make your use of multiple threading not work and
00:27:05.180 --> 00:27:11.640
people are not getting used to profile that thing as well so you know like it's very hard to
00:27:11.800 --> 00:27:15.840
understand why your algorithm that should be scaling with number of threads is not scaling
00:27:16.540 --> 00:27:20.659
because now you need different tools and not even the tool that we're going to talk about today
00:27:20.680 --> 00:27:25.860
works in that case right so you need to like people people need to be really careful with that i like
00:27:25.860 --> 00:27:30.760
your analogy of like yeah you're using threadings but you've kind of been sheltered it's like oh yeah
00:27:30.960 --> 00:27:37.300
i i traversed the desert car versus on a bicycle right it's yeah you it's not the same experience
00:27:37.700 --> 00:27:43.180
of going through the desert in a car like oh it is hot exactly exactly yeah yeah yeah and people
00:27:43.250 --> 00:27:48.499
don't know you know they don't know how fresh that the car was when they were towards in the desert
00:27:48.580 --> 00:27:58.860
right uh but the deal it made so everything so easy right with uh right is your ac ac in your uh
00:27:59.020 --> 00:28:06.480
onda you know desert on that all right exactly yeah all right so so then let's talk profiling so
00:28:06.820 --> 00:28:12.860
i think maybe a quick history of profilers in python we've got the more advanced sort of external ones
00:28:13.020 --> 00:28:17.540
like memory and so on but let's let's just focus on what i always thought was a little bit of a
00:28:17.320 --> 00:28:20.260
weird distinction that there's two profilers,
00:28:20.680 --> 00:28:23.520
the C profile and the just profile in Python.
00:28:23.800 --> 00:28:23.900
Yeah.
00:28:24.360 --> 00:28:26.200
I just pick one, but tell us,
00:28:26.540 --> 00:28:28.060
this is what we've had for quite a while.
00:28:28.220 --> 00:28:29.780
I've done a lot with these.
00:28:30.340 --> 00:28:30.460
So,
00:28:31.220 --> 00:28:35.100
yeah, this is like, it's kind of weird to say today where like,
00:28:35.260 --> 00:28:35.400
you know,
00:28:35.620 --> 00:28:36.040
so many
00:28:36.120 --> 00:28:39.480
people are using Python and Python is so big that what we're going to explain here is a
00:28:39.480 --> 00:28:39.900
bit like,
00:28:40.180 --> 00:28:40.380
you know,
00:28:41.060 --> 00:28:41.860
it sounds strange,
00:28:42.180 --> 00:28:44.540
but so it used to be a time where like,
00:28:44.820 --> 00:28:44.920
you know,
00:28:44.980 --> 00:28:47.340
the bar for the modules was much lower.
00:28:48.190 --> 00:28:50.220
So the profile module,
00:28:50.380 --> 00:28:51.900
I mean, you know, it's a different
00:28:52.140 --> 00:28:52.200
time.
00:28:52.400 --> 00:28:53.880
I'm not saying I will not do that,
00:28:54.060 --> 00:28:56.440
right? But like the quality of the tools that we're using
00:28:56.610 --> 00:28:56.720
today,
00:28:56.960 --> 00:28:58.140
not just profilers in general,
00:28:58.320 --> 00:29:00.720
it's under much more scrutiny because,
00:29:01.020 --> 00:29:01.540
you know, it serves
00:29:01.590 --> 00:29:04.240
more people and those people have like more demanding workflows.
00:29:04.390 --> 00:29:04.860
And this is true,
00:29:05.040 --> 00:29:05.140
right?
00:29:05.200 --> 00:29:09.240
The software that was written long, long ago had less complexity to deal with because,
00:29:09.520 --> 00:29:09.620
you know,
00:29:10.040 --> 00:29:14.340
computers were not that, you know, powerful and like the stack was like saluer,
00:29:14.520 --> 00:29:14.600
right?
00:29:14.980 --> 00:29:19.300
So the profile module was the first attempt that people tried to do profiling in Python.
00:29:19.860 --> 00:29:21.680
It's a very good attempt, so like, oh, it's actually good.
00:29:21.760 --> 00:29:23.140
The problem is architecturally speaking,
00:29:23.300 --> 00:29:24.940
it's not really a good profiler in general.
00:29:25.180 --> 00:29:30.180
The reason is because it's written in Python and it runs inside application that is being profiled.
00:29:30.420 --> 00:29:37.340
So you don't need to be a profiler expert to know that you're doing extra work to measure the work that is going to skew the results, right?
00:29:37.480 --> 00:29:39.600
Because now you're not measuring your application.
00:29:39.800 --> 00:29:41.920
You're measuring your application and the profiler at the same time,
00:29:42.120 --> 00:29:42.240
right?
00:29:42.640 --> 00:29:44.180
So that was bad.
00:29:44.300 --> 00:29:50.540
And then this profiler module kind of like does a bunch of tricks to try to measure how much itself is contributing to the problem.
00:29:50.690 --> 00:29:53.200
But like you can imagine how flaky that is.
00:29:54.080 --> 00:29:59.160
And then the other problem is that it makes your application much slower because it's being measured, right?
00:29:59.260 --> 00:30:01.100
And the other slowness was like huge.
00:30:01.280 --> 00:30:04.080
Like we're talking about like 20 times slower and things like that.
00:30:04.130 --> 00:30:04.500
Or like,
00:30:04.760 --> 00:30:04.920
you know,
00:30:05.180 --> 00:30:05.860
depending on what you're doing.
00:30:06.380 --> 00:30:08.280
So part of the reason as well, again,
00:30:08.460 --> 00:30:10.300
is because the profiler itself was written in Python.
00:30:10.640 --> 00:30:11.120
So perhaps,
00:30:11.820 --> 00:30:12.000
you know,
00:30:12.260 --> 00:30:16.180
that was not a great experience.
00:30:16.660 --> 00:30:17.960
So the C profile module,
00:30:18.320 --> 00:30:18.400
which,
00:30:19.120 --> 00:30:21.120
you know, is very explicit about what is a trick,
00:30:21.440 --> 00:30:21.700
but like,
00:30:21.960 --> 00:30:22.120
unfortunately,
00:30:22.900 --> 00:30:27.500
it's a leaky name because like the C in the C profile is the C language, right?
00:30:27.700 --> 00:30:28.220
And also,
00:30:29.100 --> 00:30:34.580
it has one of the biggest problems that I don't like this module is that the C is lowercase and the P is uppercase.
00:30:34.680 --> 00:30:36.520
And the name of the stupid module is like this.
00:30:36.700 --> 00:30:38.480
So writing the thing is very annoying.
00:30:39.380 --> 00:30:41.820
I always forget which one is uppercase and lowercase.
00:30:41.980 --> 00:30:42.060
Anyway,
00:30:42.320 --> 00:30:46.220
so the C profile is basically a re-implementation of the profile module in C,
00:30:46.540 --> 00:30:47.300
the illegal language.
00:30:47.620 --> 00:30:50.560
And then many people don't know why I say this.
00:30:50.620 --> 00:30:55.760
This is because the White House released this hyper thing that C was now dangerous because,
00:30:55.940 --> 00:30:57.120
you know, it's like memory unsafe.
00:30:57.480 --> 00:31:03.540
So I find very funny that the White House said that C is illegal and CPython is,
00:31:04.360 --> 00:31:05.060
we are all,
00:31:05.700 --> 00:31:08.700
you know, like bandits writing in the illegal language.
00:31:08.720 --> 00:31:08.760
Exactly.
00:31:09.380 --> 00:31:09.740
Yeah, yeah.
00:31:10.050 --> 00:31:10.140
Okay,
00:31:10.340 --> 00:31:10.860
so-
00:31:10.860 --> 00:31:11.620
Living underground,
00:31:12.220 --> 00:31:16.620
hiding from everything with your GCC and your clang.
00:31:17.100 --> 00:31:17.460
Exactly.
00:31:17.640 --> 00:31:19.460
Give food to your core developer underground.
00:31:19.860 --> 00:31:20.080
Yeah, yeah.
00:31:20.220 --> 00:31:25.060
At least Seth and team were able to get Python on the good list.
00:31:25.440 --> 00:31:26.120
Oh, yeah, yeah, yeah.
00:31:26.820 --> 00:31:28.180
We both like that also.
00:31:28.220 --> 00:31:28.600
Memory-safe.
00:31:29.140 --> 00:31:32.100
And I was like, everybody's like, yeah, yeah, of course, Python is memory-safe.
00:31:32.180 --> 00:31:32.780
And I was like thinking,
00:31:33.150 --> 00:31:33.380
is it?
00:31:33.450 --> 00:31:34.380
They're like, I don't know, man.
00:31:35.820 --> 00:31:37.640
I see a lot of bugs, but like, okay,
00:31:37.820 --> 00:31:39.040
okay, we are allowed to have bugs,
00:31:39.240 --> 00:31:41.220
you know, like Rust as well can have bugs and crash,
00:31:41.560 --> 00:31:41.800
maybe less,
00:31:42.040 --> 00:31:42.140
but,
00:31:42.460 --> 00:31:42.660
you know,
00:31:43.100 --> 00:31:46.380
in theory, the language is very safe if you assume that there's no bugs.
00:31:46.820 --> 00:31:46.900
Anyway,
00:31:47.890 --> 00:31:51.900
so C profile is this like implementation of the whole tree bank in C.
00:31:52.300 --> 00:31:54.400
So it's faster than profile,
00:31:54.700 --> 00:31:54.920
actually,
00:31:55.100 --> 00:31:56.180
it's 10 times faster or so.
00:31:56.700 --> 00:31:57.440
But it still,
00:31:57.740 --> 00:31:57.800
again,
00:31:57.980 --> 00:31:58.740
has the same problem,
00:31:59.000 --> 00:32:01.940
just less. So it makes your application still two or three times slower,
00:32:02.720 --> 00:32:04.500
something like that, which is already huge, right?
00:32:04.620 --> 00:32:06.900
Like thinking that your application takes five minutes,
00:32:07.060 --> 00:32:08.060
now it takes 15 minutes, right?
00:32:08.100 --> 00:32:08.760
So not great.
00:32:09.640 --> 00:32:13.160
And also the problem is that it also runs in the process.
00:32:13.680 --> 00:32:15.080
So I don't want to get the impression
00:32:15.180 --> 00:32:16.160
that this is just negative.
00:32:17.100 --> 00:32:18.700
So because now Cprofile is actually,
00:32:19.340 --> 00:32:20.380
it's well architecture,
00:32:21.540 --> 00:32:22.300
it's now in C,
00:32:22.460 --> 00:32:23.260
so it's all good.
00:32:23.760 --> 00:32:26.020
And now the problems in the slowdowns
00:32:26.160 --> 00:32:28.220
are an inherited property of these kind of tools,
00:32:28.440 --> 00:32:28.520
right?
00:32:28.580 --> 00:32:30.100
Because that run inside the profiler.
00:32:30.260 --> 00:32:31.840
These are called tracing profilers.
00:32:32.320 --> 00:32:33.980
The reason they are called tracing profilers
00:32:34.000 --> 00:32:36.900
because they see every single call that is happening,
00:32:37.150 --> 00:32:37.260
right?
00:32:37.440 --> 00:32:39.340
So every time you call a function, they see that
00:32:39.370 --> 00:32:40.360
and they need to record that.
00:32:40.460 --> 00:32:41.900
So it's a lot of work to do,
00:32:42.720 --> 00:32:45.060
but also because they run in the process,
00:32:45.250 --> 00:32:46.080
every time you call a function,
00:32:46.220 --> 00:32:47.100
now you do this extra work
00:32:47.350 --> 00:32:49.680
and then therefore your thing is worse.
00:32:49.980 --> 00:32:51.180
The bright side, of course,
00:32:51.620 --> 00:32:52.600
is that you collect everything.
00:32:53.060 --> 00:32:54.300
So if you want to know how many times
00:32:54.370 --> 00:32:55.660
your progress is calling,
00:32:56.220 --> 00:32:57.260
send money to employees,
00:32:57.840 --> 00:32:59.600
you will know how much money are you selling to employees.
00:32:59.880 --> 00:33:00.840
You are not missing it.
00:33:01.220 --> 00:33:03.600
If it says seven times, it was called seven times.
00:33:03.840 --> 00:33:06.140
Yeah, it wasn't just a statistical analysis,
00:33:06.380 --> 00:33:06.440
yeah.
00:33:06.600 --> 00:33:06.860
Exactly,
00:33:07.040 --> 00:33:07.140
exactly.
00:33:07.220 --> 00:33:08.980
So you can tell your employees that were not paid
00:33:09.080 --> 00:33:09.940
that that was not the reason,
00:33:10.140 --> 00:33:10.200
right?
00:33:10.340 --> 00:33:12.680
So in that case, then,
00:33:13.040 --> 00:33:13.840
it's really good for that.
00:33:13.980 --> 00:33:15.300
The same way Membray, for example,
00:33:15.520 --> 00:33:15.900
if you recall,
00:33:16.480 --> 00:33:17.820
Membray is also a tracing profiler
00:33:18.160 --> 00:33:21.680
because, you know, imagine that you lose a one gigabyte allocation.
00:33:21.980 --> 00:33:22.860
Can you imagine that, oh,
00:33:23.020 --> 00:33:24.040
well, we just lose that one.
00:33:24.220 --> 00:33:24.500
And, like,
00:33:24.860 --> 00:33:26.560
you know, I suppose your application is fine.
00:33:26.820 --> 00:33:27.640
Like, you don't want that, right?
00:33:27.700 --> 00:33:30.240
Like, you want Membray to tell you exactly everything that happened.
00:33:30.400 --> 00:33:31.340
In time, it's different, right?
00:33:31.420 --> 00:33:33.100
because like we will talk about that in a second,
00:33:33.640 --> 00:33:34.280
Laszlo will explain,
00:33:34.460 --> 00:33:35.500
but, you know,
00:33:35.720 --> 00:33:35.960
here,
00:33:36.360 --> 00:33:39.100
that's the advantage of Zprofile still even today
00:33:39.390 --> 00:33:39.500
because,
00:33:39.780 --> 00:33:39.980
you know,
00:33:40.290 --> 00:33:41.680
perhaps people will hear today's episode
00:33:41.750 --> 00:33:41.940
and say,
00:33:42.100 --> 00:33:43.760
well, Zprofile is not good anymore.
00:33:43.920 --> 00:33:44.660
We should throw it away,
00:33:44.860 --> 00:33:46.640
but it still has its own use cases.
00:33:47.120 --> 00:33:48.720
So that is, I think, a brief story
00:33:48.920 --> 00:33:50.440
of like why we have to...
00:33:50.580 --> 00:33:51.660
Laszlo, what do you want to add to that?
00:33:52.060 --> 00:33:54.100
Yeah, so I think it's definitely,
00:33:54.230 --> 00:33:57.340
I think, a problem with the ergonomics as well.
00:33:57.480 --> 00:33:58.420
And if you're a developer
00:33:58.510 --> 00:34:00.180
and you want to provide a Python application,
00:34:00.480 --> 00:34:02.280
And I think if you look at the documentation,
00:34:03.040 --> 00:34:03.140
I,
00:34:03.340 --> 00:34:04.280
when I first looked at this,
00:34:04.840 --> 00:34:05.760
found it very confusing.
00:34:06.240 --> 00:34:08.480
How do I decide whether it's Cprofile or Profile, right?
00:34:08.580 --> 00:34:11.960
And then I think it just became sort of the defacto standard.
00:34:12.050 --> 00:34:13.580
You just always use Cprofile, right?
00:34:14.320 --> 00:34:15.440
Why exactly it's unclear,
00:34:15.820 --> 00:34:15.919
right?
00:34:16.190 --> 00:34:20.440
So I think, but I also think that there haven't been,
00:34:20.820 --> 00:34:23.360
like CPI didn't have obviously something profiling
00:34:23.450 --> 00:34:25.600
and it didn't have powerful profiling tools.
00:34:25.740 --> 00:34:28.179
So you always had to look for external tools
00:34:28.200 --> 00:34:32.659
to get actually like higher quality sort of profiling results.
00:34:32.770 --> 00:34:33.260
And of course,
00:34:33.500 --> 00:34:37.100
there's an entire family of situations
00:34:37.190 --> 00:34:39.419
where you cannot use a tracing profiler
00:34:39.860 --> 00:34:41.780
because effectively for a tracing profiler,
00:34:41.790 --> 00:34:44.120
you have to reconstruct a small sample test.
00:34:44.620 --> 00:34:47.240
You cannot run that on your application,
00:34:47.580 --> 00:34:47.639
right?
00:34:47.800 --> 00:34:49.440
Because it would just be too slow.
00:34:50.300 --> 00:34:51.120
It's too time consuming.
00:34:51.230 --> 00:34:53.760
So generally you would just have a test case or something
00:34:53.950 --> 00:34:54.780
and run it on that.
00:34:55.060 --> 00:34:56.480
Yeah, the worst of that is
00:34:56.500 --> 00:34:58.020
It doesn't work great in production,
00:34:58.580 --> 00:35:02.220
but I profiled it locally and it seems okay.
00:35:02.520 --> 00:35:03.580
So what's going on, right?
00:35:03.900 --> 00:35:04.040
Exactly.
00:35:04.700 --> 00:35:07.420
You can't really reproduce the exact same situation oftentimes.
00:35:07.800 --> 00:35:12.220
And then you're basically left with no options or you have to start downloading,
00:35:12.680 --> 00:35:14.480
finding some third-party option.
00:35:14.760 --> 00:35:20.260
I can't, Pablo, I got to circle back to this a little bit because I think it's really interesting.
00:35:20.340 --> 00:35:25.180
So you talked about the tracing profilers, which C profile and profile is,
00:35:25.420 --> 00:35:26.560
and how it makes,
00:35:26.830 --> 00:35:31.860
they're very accurate, but they make a big impact on basically what is happening with your code.
00:35:32.120 --> 00:35:34.880
And it reminds me of quantum mechanics a little bit.
00:35:35.230 --> 00:35:35.360
Wow.
00:35:35.920 --> 00:35:36.580
Let's go.
00:35:36.700 --> 00:35:37.060
Let's go.
00:35:37.160 --> 00:35:39.080
It's like there might've been a function.
00:35:39.580 --> 00:35:40.320
Let's take two functions.
00:35:40.520 --> 00:35:43.420
They each take half a second in practice on their own,
00:35:43.620 --> 00:35:43.720
right?
00:35:43.940 --> 00:35:48.220
One of them has to do something with a file system a million times in a loop.
00:35:48.320 --> 00:35:50.100
Another one of them calls an API,
00:35:50.680 --> 00:35:52.000
which takes half a second to return.
00:35:52.280 --> 00:35:53.980
You run that in a tracing profiler.
00:35:54.320 --> 00:35:56.620
The one that calls the API is barely changed.
00:35:57.040 --> 00:35:58.080
It's tracing one call,
00:35:58.480 --> 00:35:58.680
sort of,
00:35:58.960 --> 00:35:59.040
right?
00:35:59.300 --> 00:36:00.180
You do the other one.
00:36:00.500 --> 00:36:05.600
It's now a million plus little intercepts of like a function call over and over and over.
00:36:05.720 --> 00:36:09.260
And it makes that one that's chatty in Python seem really,
00:36:09.420 --> 00:36:09.880
really slow.
00:36:10.220 --> 00:36:12.000
By observing it, you've changed it.
00:36:12.080 --> 00:36:12.660
You know what I mean?
00:36:12.900 --> 00:36:13.160
Yeah,
00:36:13.340 --> 00:36:13.540
yeah, yeah.
00:36:13.560 --> 00:36:14.020
In this sense.
00:36:15.300 --> 00:36:17.740
I'm very sad to report that, unfortunately,
00:36:18.060 --> 00:36:19.400
because you're talking to a physicist,
00:36:19.620 --> 00:36:20.780
I'm going to be a pain here.
00:36:21.080 --> 00:36:22.320
Do it. Let's do it. Let's get it right.
00:36:22.700 --> 00:36:25.700
I'm going to put my pedantic hat here.
00:36:27.080 --> 00:36:27.700
So, yes,
00:36:27.880 --> 00:36:28.280
you're right.
00:36:28.620 --> 00:36:29.000
Also,
00:36:31.000 --> 00:36:32.700
technically speaking, it's not really correct.
00:36:33.020 --> 00:36:33.660
The reason is because,
00:36:33.980 --> 00:36:36.320
yes, that is indeed what happens in quantum mechanics.
00:36:36.440 --> 00:36:36.880
When you observe,
00:36:37.120 --> 00:36:38.080
you modify the system.
00:36:38.260 --> 00:36:39.300
That is correct. That part is correct.
00:36:39.760 --> 00:36:44.720
But the reason the metaphor kind of like doesn't fully work is because the important part of quantum mechanics
00:36:44.840 --> 00:36:45.420
is that before,
00:36:45.780 --> 00:36:46.860
if this was quantum mechanics,
00:36:47.230 --> 00:36:49.860
before you actually measure your program,
00:36:50.320 --> 00:36:52.660
your program is both fast and slow at the same time.
00:36:52.940 --> 00:36:55.160
But that's the key,
00:36:55.320 --> 00:36:55.380
right?
00:36:55.780 --> 00:36:58.100
We got the cat in the box as the profiler now.
00:36:58.430 --> 00:36:59.000
Yeah, right, right.
00:36:59.290 --> 00:36:59.380
Here,
00:36:59.640 --> 00:37:00.540
the profiler,
00:37:01.140 --> 00:37:02.440
sorry, your program was the cat.
00:37:02.770 --> 00:37:03.780
So your cat is dead,
00:37:04.120 --> 00:37:04.220
right?
00:37:04.360 --> 00:37:06.080
And then you say, well,
00:37:06.320 --> 00:37:06.540
you know,
00:37:06.930 --> 00:37:10.980
I can move the box and then suddenly,
00:37:11.180 --> 00:37:11.340
you know,
00:37:11.420 --> 00:37:13.640
I open the box and it's dead, right?
00:37:13.760 --> 00:37:15.400
So you say, well, you modify the system.
00:37:15.940 --> 00:37:16.360
But here,
00:37:16.780 --> 00:37:16.900
you know,
00:37:17.640 --> 00:37:18.520
for being quantum mechanics,
00:37:18.800 --> 00:37:19.200
you,
00:37:19.670 --> 00:37:20.800
I mean, it would be a very good excuse
00:37:20.990 --> 00:37:23.100
to tell your boss and then say, no, no, no, you don't understand.
00:37:23.420 --> 00:37:25.440
This program is fast and slow at the same time.
00:37:25.760 --> 00:37:27.520
If we measure, we would screw it up,
00:37:27.760 --> 00:37:27.920
you know?
00:37:28.020 --> 00:37:28.240
So like,
00:37:28.610 --> 00:37:30.260
we must not measure the program.
00:37:31.240 --> 00:37:31.700
Well, here,
00:37:31.960 --> 00:37:35.240
of course, you can imagine that what happens here is that the program is either
00:37:35.460 --> 00:37:36.220
fast or slow.
00:37:36.940 --> 00:37:39.900
And what happens is that when you measure, you are making it much slower.
00:37:40.210 --> 00:37:40.300
So,
00:37:40.790 --> 00:37:42.600
but it was already fast or slow.
00:37:42.660 --> 00:37:43.940
It's just you're making it worse.
00:37:44.050 --> 00:37:46.040
So it was slow, now it's very slow.
00:37:46.340 --> 00:37:47.740
And if it's less fast, now it's slow.
00:37:48.020 --> 00:37:49.240
But it's not like, oh,
00:37:49.460 --> 00:37:51.900
wow, it was in this superposition of states.
00:37:52.360 --> 00:37:53.820
I know I'm not a fan person,
00:37:53.990 --> 00:37:56.280
so don't feel too bad.
00:37:56.380 --> 00:37:57.240
It's still a good metaphor.
00:37:58.600 --> 00:38:04.060
My warning is that the tracing profilers make it unevenly slower.
00:38:04.740 --> 00:38:05.820
Some parts get away.
00:38:06.120 --> 00:38:11.240
Like parts that do a lot of internal in Python function calls are affected a lot.
00:38:11.360 --> 00:38:16.700
stuff that works with external systems that don't require much execution of python but themselves
00:38:16.890 --> 00:38:23.100
are slow are not very affected by tracing profilers that was my warning my my quantum mechanics
00:38:23.380 --> 00:38:27.460
analogy wasn't perfect i understand so let's try this sorry sorry sorry no what about string theory
00:38:28.359 --> 00:38:34.580
no no i'm just kidding let's let's talk about let's talk about the peps let's talk about what's
00:38:34.600 --> 00:38:35.760
coming here.
00:38:36.060 --> 00:38:36.380
Oh,
00:38:36.630 --> 00:38:36.940
right, right.
00:38:37.290 --> 00:38:39.500
And we got the PEP 799,
00:38:39.900 --> 00:38:41.500
a dedicated profiling package
00:38:41.700 --> 00:38:45.140
for organizing Python profiling tools by the two of you.
00:38:45.400 --> 00:38:45.540
Awesome.
00:38:45.980 --> 00:38:46.360
Tell us about this.
00:38:46.580 --> 00:38:48.780
And then we're going to talk about 768,
00:38:48.980 --> 00:38:51.680
which is some of the actual changes.
00:38:52.130 --> 00:38:52.240
Yeah.
00:38:52.240 --> 00:38:54.240
So how did we start the back story,
00:38:54.480 --> 00:38:54.620
Pablo,
00:38:54.840 --> 00:38:56.720
that I think we actually were planning
00:38:56.790 --> 00:38:59.440
to put the something profiler in the profile package,
00:38:59.800 --> 00:38:59.960
right?
00:39:00.070 --> 00:39:02.300
I think that was the original conning plan.
00:39:02.540 --> 00:39:05.660
And then it turned out we cannot do that.
00:39:06.240 --> 00:39:07.340
And that led to this path.
00:39:08.240 --> 00:39:09.040
Because basically,
00:39:09.440 --> 00:39:10.740
let's do the right thing, right?
00:39:10.920 --> 00:39:11.740
Let's clean up this.
00:39:12.260 --> 00:39:14.400
I don't want to say mess, but let's clean things up.
00:39:14.560 --> 00:39:18.240
And that would mean that you're going to have a single tracing profiler.
00:39:19.100 --> 00:39:20.760
Eventually, we can talk about how that's going to go.
00:39:20.840 --> 00:39:22.640
And then you're going to have one sampling profiler.
00:39:22.740 --> 00:39:25.480
And this is all going to be grouped into the profiling package,
00:39:25.800 --> 00:39:28.640
which should provide much better sort of ergonomics.
00:39:28.720 --> 00:39:31.080
It should make it much easier for developers to decide
00:39:31.100 --> 00:39:32.820
what tool they should reach for.
00:39:32.820 --> 00:39:34.480
And it also means that there are not going to be
00:39:34.660 --> 00:39:36.080
two different tracing profilers,
00:39:36.180 --> 00:39:37.180
which makes it difficult to,
00:39:37.740 --> 00:39:37.880
again,
00:39:38.020 --> 00:39:39.220
like to choose the right thing.
00:39:39.360 --> 00:39:41.820
So profiling the tracing effectively becomes,
00:39:42.820 --> 00:39:44.760
that's basically C-profile.
00:39:45.160 --> 00:39:47.020
And C-profile is going to continue living on
00:39:47.200 --> 00:39:49.740
as the C-profile import as well for compatibility.
00:39:51.080 --> 00:39:51.980
But profile,
00:39:52.240 --> 00:39:54.540
so that the old Python-only version
00:39:54.900 --> 00:39:59.180
is effectively deprecated and will be removed eventually.
00:39:59.680 --> 00:40:01.640
And then profiling with sampling is the new profiler,
00:40:01.860 --> 00:40:02.080
Tachium.
00:40:02.380 --> 00:40:03.220
It makes it super obvious.
00:40:03.740 --> 00:40:07.680
You just import profiling.tracing or profiling.sampling.
00:40:08.080 --> 00:40:09.980
If you understand the nomenclature of profiling,
00:40:10.140 --> 00:40:11.580
it tells you exactly what you're getting, right?
00:40:11.840 --> 00:40:13.880
Yeah, it gets a bit long with time,
00:40:14.140 --> 00:40:14.240
right?
00:40:14.420 --> 00:40:15.600
Because like you mostly,
00:40:16.040 --> 00:40:16.640
as we will discuss,
00:40:16.940 --> 00:40:19.240
mostly all the time you want profiling.sampling.
00:40:19.560 --> 00:40:20.380
So now it's like,
00:40:20.540 --> 00:40:20.640
damn,
00:40:21.220 --> 00:40:22.940
profiling.sampling.
00:40:23.080 --> 00:40:24.660
Like, this is very, very boring.
00:40:25.440 --> 00:40:25.520
But,
00:40:25.720 --> 00:40:28.360
you know, I propose to just call it Tachium.
00:40:28.580 --> 00:40:28.740
Like,
00:40:28.990 --> 00:40:31.160
you know, maybe in 3.16,
00:40:31.500 --> 00:40:35.560
I can convince people to let me have a top level binary name.
00:40:35.960 --> 00:40:38.920
You know, the same way when you install Python in a VAMF, you get,
00:40:39.340 --> 00:40:40.660
so you can get Tachyon.
00:40:41.080 --> 00:40:42.300
Like, we'll see, we'll see.
00:40:43.440 --> 00:40:44.480
Maybe you should,
00:40:44.680 --> 00:40:44.960
sorry,
00:40:45.200 --> 00:40:48.740
maybe you should propose to have a C-Tachyon and a Tachyon module.
00:40:49.280 --> 00:40:49.800
Confuse everyone.
00:40:50.530 --> 00:40:51.580
Oh, C-Tachyon.
00:40:51.880 --> 00:40:52.940
But Tachyon is C already.
00:40:53.400 --> 00:40:55.940
And then just make a compromise to have just Tachyon,
00:40:56.260 --> 00:40:56.340
right?
00:40:56.920 --> 00:40:57.240
No, no, no.
00:40:57.360 --> 00:41:00.040
Because Tachyon is C, then it will be P-Python in Python.
00:41:00.310 --> 00:41:02.280
Like a pure Python implementation of Tachyon.
00:41:02.480 --> 00:41:02.940
P-Python.
00:41:04.020 --> 00:41:05.480
I've got a fix for you, Pablo.
00:41:06.060 --> 00:41:08.760
Just create a Python package called Tachyon.
00:41:09.060 --> 00:41:10.060
Oh, already exists,
00:41:10.220 --> 00:41:10.320
man.
00:41:10.390 --> 00:41:14.000
And then all it does is import profiling,
00:41:14.280 --> 00:41:14.780
tracing,
00:41:15.570 --> 00:41:17.780
or sampling as Tachyon.
00:41:18.120 --> 00:41:18.840
But apparently we cannot.
00:41:18.840 --> 00:41:19.980
And then you'll be exit.
00:41:20.170 --> 00:41:20.960
Yeah, you're good to go.
00:41:21.280 --> 00:41:27.060
Okay, or I can bribe Charlie Marsh that they add UB-Tachyon.
00:41:27.300 --> 00:41:28.340
You know, like uv profile.
00:41:29.090 --> 00:41:29.660
UV profile?
00:41:30.000 --> 00:41:35.160
How many megadollars, how many megadollars are needed for this?
00:41:35.360 --> 00:41:36.020
Like, we will see.
00:41:36.540 --> 00:41:36.920
Open AI.
00:41:37.360 --> 00:41:38.040
Sam Alman.
00:41:38.180 --> 00:41:38.560
Make it happen,
00:41:38.730 --> 00:41:38.820
yes.
00:41:39.010 --> 00:41:39.680
Allow this to happen.
00:41:40.060 --> 00:41:40.380
Yes, yes.
00:41:41.100 --> 00:41:43.320
You will be so much popular on the Anthropics, those guys.
00:41:44.040 --> 00:41:45.320
You just need a little prompt injection.
00:41:45.370 --> 00:41:45.960
You'll be good to go.
00:41:46.080 --> 00:41:46.960
Yes, exactly, exactly.
00:41:47.960 --> 00:41:48.300
All right, so,
00:41:48.600 --> 00:41:53.060
Oslo, you mentioned the old Python profile module
00:41:53.480 --> 00:41:55.460
is under deprecation warning.
00:41:56.500 --> 00:41:58.960
And that will be in 15 and 16.
00:41:59.300 --> 00:42:00.580
And actually two years from now,
00:42:00.920 --> 00:42:01.460
two years from now,
00:42:01.620 --> 00:42:02.140
plus a month,
00:42:02.820 --> 00:42:03.500
it will be out.
00:42:03.700 --> 00:42:05.160
It will be removed in 317.
00:42:05.560 --> 00:42:06.680
So just heads up for people.
00:42:07.380 --> 00:42:07.700
Obliterated.
00:42:08.240 --> 00:42:08.740
That's right.
00:42:08.960 --> 00:42:11.340
It will be recated into history for real.
00:42:11.440 --> 00:42:11.980
Oblivion.
00:42:12.300 --> 00:42:12.580
Oblivion.
00:42:12.940 --> 00:42:12.980
Okay.
00:42:13.100 --> 00:42:16.400
So this is really just kind of an organizational deal, right?
00:42:16.520 --> 00:42:16.900
This PEP.
00:42:17.180 --> 00:42:17.300
Okay.
00:42:17.700 --> 00:42:19.720
And then we have PEP 768,
00:42:20.180 --> 00:42:21.720
which sort of makes this possible,
00:42:21.880 --> 00:42:22.000
right?
00:42:22.080 --> 00:42:23.360
It makes Tachyon,
00:42:23.800 --> 00:42:24.720
the new profiler,
00:42:24.960 --> 00:42:25.480
accessible,
00:42:25.780 --> 00:42:25.840
right?
00:42:26.100 --> 00:42:26.360
Yes.
00:42:26.540 --> 00:42:29.060
So the original of that PEP was to,
00:42:30.120 --> 00:42:33.000
that's the previous work we did with my colleagues from Bloomberg,
00:42:33.160 --> 00:42:33.760
Ivana and Matt.
00:42:34.040 --> 00:42:37.080
And the idea here is that we were chasing this goal,
00:42:37.240 --> 00:42:40.840
like North Star, of allowing PDB, the Python debugger,
00:42:41.040 --> 00:42:44.920
and technically any other debugger, but also the one that we ship in the standard library.
00:42:45.380 --> 00:42:48.000
So we wanted that to be able to attach to their programs.
00:42:48.200 --> 00:42:51.560
So you could have a program running in production that is doing weird stuff.
00:42:51.900 --> 00:42:55.560
And instead of having to stop it and then running under the debugger,
00:42:55.860 --> 00:42:56.780
you could just say, hey,
00:42:57.160 --> 00:42:57.380
you know,
00:42:57.620 --> 00:42:58.500
touch to that program
00:42:58.680 --> 00:42:59.760
and like, tell me what's wrong,
00:42:59.920 --> 00:43:00.000
right?
00:43:00.000 --> 00:43:00.920
So we wanted that.
00:43:01.120 --> 00:43:01.720
And then we added,
00:43:01.940 --> 00:43:02.420
therefore,
00:43:02.700 --> 00:43:03.400
to achieve this,
00:43:03.780 --> 00:43:06.680
we added all these capabilities to CPython
00:43:07.040 --> 00:43:08.760
such that you could attach to a running process
00:43:09.000 --> 00:43:10.860
and then you could inspect the running process, right?
00:43:10.920 --> 00:43:12.720
Because the debugger needs to inspect the running process.
00:43:12.940 --> 00:43:14.860
So we have all this machinery there.
00:43:15.200 --> 00:43:15.320
And,
00:43:15.540 --> 00:43:18.280
you know, that was this PEP and that's shipping 3.14.
00:43:19.000 --> 00:43:22.180
And 3.14 has a bunch of utilities that use this machinery
00:43:22.820 --> 00:43:25.740
and allow you to attach with PDV and much more,
00:43:25.920 --> 00:43:25.980
right?
00:43:26.220 --> 00:43:29.240
But then we were at PyCon US 2025,
00:43:30.080 --> 00:43:31.180
and then I went to Laszlo,
00:43:31.400 --> 00:43:32.380
and then I told Laszlo,
00:43:32.540 --> 00:43:35.460
hey, Laszlo, would it be cool to call this in a loop?
00:43:35.840 --> 00:43:37.300
And then we have a profiler,
00:43:38.120 --> 00:43:40.240
and then Laszlo is like, yes, let's just do it.
00:43:40.240 --> 00:43:41.160
So in the sprint,
00:43:41.520 --> 00:43:43.980
we did this mini program
00:43:44.260 --> 00:43:45.760
that was calling this thing in a loop,
00:43:45.960 --> 00:43:46.040
right?
00:43:46.040 --> 00:43:48.820
So this thing allows you to get the stack of the application,
00:43:49.200 --> 00:43:50.760
kind of, and then at the end,
00:43:50.860 --> 00:43:55.200
the profiler is just doing it very quickly and dump the information somewhere.
00:43:55.380 --> 00:43:57.340
But that is at the point not important.
00:43:57.400 --> 00:44:01.820
The most important part is how fast can you see what the program is doing, right?
00:44:01.900 --> 00:44:03.040
That's what a sampling profiler does.
00:44:03.760 --> 00:44:05.400
And just for reference,
00:44:05.560 --> 00:44:08.080
because many people may not be aware of these tools,
00:44:09.160 --> 00:44:12.280
the minimum, minimum, minimum number that you are looking at
00:44:12.440 --> 00:44:14.580
may be something like 100 times per second,
00:44:15.000 --> 00:44:15.060
right?
00:44:15.120 --> 00:44:18.120
That would be on the realm of useful already.
00:44:18.560 --> 00:44:20.960
And it goes from there to more than that,
00:44:21.120 --> 00:44:21.200
right?
00:44:21.620 --> 00:44:24.380
So we put the loop and then we measure, okay,
00:44:24.560 --> 00:44:27.580
how fast is this loop here?
00:44:27.940 --> 00:44:29.380
And it was like two times per second,
00:44:29.980 --> 00:44:31.020
like two times per second.
00:44:31.100 --> 00:44:33.240
That was how fast it was,
00:44:34.000 --> 00:44:36.460
which as you can imagine is not fantastic, not fantastic.
00:44:36.700 --> 00:44:40.260
So we sat down together and for the course of the sprint,
00:44:40.480 --> 00:44:46.500
we basically moved from two times per second to an illegal number,
00:44:47.420 --> 00:44:49.040
like a month of frames per second,
00:44:49.240 --> 00:44:52.560
which I think even today is more than a million times per second.
00:44:53.220 --> 00:44:53.440
But yeah,
00:44:53.620 --> 00:44:55.960
how was your experience during that time, Lasslo?
00:44:56.280 --> 00:44:59.000
Well, yeah, I remember that we were chatting about this
00:44:59.520 --> 00:45:00.640
in the beginning of the sprints
00:45:00.820 --> 00:45:02.980
and Pablo laid out the plan.
00:45:02.980 --> 00:45:03.780
This is what we're going to do.
00:45:04.040 --> 00:45:06.820
I'm going to figure out how fast this thing is
00:45:07.200 --> 00:45:08.840
and then I will see if I can make it faster
00:45:08.900 --> 00:45:11.160
because it might not be that good yet.
00:45:11.540 --> 00:45:15.660
And you're going to work on building UI around this, the CLI, right?
00:45:15.780 --> 00:45:15.860
Like,
00:45:16.020 --> 00:45:17.380
let's build up all the machinery.
00:45:18.100 --> 00:45:19.900
And then we sat down and probably just said,
00:45:20.020 --> 00:45:21.720
hey, this thing is so very slow.
00:45:22.020 --> 00:45:22.540
This is, like,
00:45:22.920 --> 00:45:23.360
not useful.
00:45:23.820 --> 00:45:26.800
And then we literally spent the entire rest of the sprint working on that.
00:45:26.810 --> 00:45:29.860
I think we actually ended up with no code committed,
00:45:30.540 --> 00:45:30.780
I think,
00:45:30.830 --> 00:45:31.280
during the sprint.
00:45:31.520 --> 00:45:35.000
All of the actual commits went in later on.
00:45:35.320 --> 00:45:38.300
But the sort of going through these motions of,
00:45:38.490 --> 00:45:38.600
like,
00:45:38.960 --> 00:45:41.900
basically every single change, getting a 10x.
00:45:42.070 --> 00:45:44.540
I mean, initially, it was easy to get a 10x improvement
00:45:44.560 --> 00:45:46.500
when you have like two hertz,
00:45:47.420 --> 00:45:48.900
two samples a second, right?
00:45:49.300 --> 00:45:51.600
But then just making it faster and faster.
00:45:51.660 --> 00:45:53.680
And actually I remember that even after that,
00:45:54.840 --> 00:45:57.220
Pablo made a couple of improvements later on
00:45:57.220 --> 00:45:58.820
and maybe we can go into details later,
00:45:58.980 --> 00:46:00.240
but he made a couple of improvements
00:46:00.620 --> 00:46:02.080
which really just made it.
00:46:02.120 --> 00:46:04.140
It was already, I think around 200,000,
00:46:04.340 --> 00:46:06.060
maybe 100,000, 200,000 hertz
00:46:06.280 --> 00:46:08.140
at the end of that on the sprints
00:46:08.300 --> 00:46:10.300
and then still like a 4X,
00:46:10.460 --> 00:46:12.220
5X improvement later on.
00:46:12.260 --> 00:46:12.940
So I think That's
00:46:12.940 --> 00:46:13.200
crazy.
00:46:13.760 --> 00:46:16.740
Yeah, and I think it was a headline later on.
00:46:16.860 --> 00:46:19.820
It's the fastest sampling profiler,
00:46:20.770 --> 00:46:20.900
basically.
00:46:21.440 --> 00:46:21.620
Yeah,
00:46:21.800 --> 00:46:22.740
just to be clear also,
00:46:23.000 --> 00:46:25.480
I think it's a good moment to mention this.
00:46:25.720 --> 00:46:29.860
This is not the first sampling profiler of this kind there is by far, right?
00:46:29.980 --> 00:46:31.500
There is a couple of them.
00:46:32.280 --> 00:46:33.760
There's quite a lot of tooling in this space.
00:46:34.280 --> 00:46:36.840
Perhaps the most famous existing tooling there is
00:46:36.840 --> 00:46:39.100
is another profiler called Pyospy,
00:46:39.720 --> 00:46:42.140
which has been for a while.
00:46:42.220 --> 00:46:45.220
and it's a very good piece of software.
00:46:45.280 --> 00:46:46.040
It's written in Rust.
00:46:46.840 --> 00:46:49.140
And this is quite efficient.
00:46:49.900 --> 00:46:51.480
The thing, so you may ask,
00:46:51.560 --> 00:46:53.960
okay, why to have one in the standard library,
00:46:54.160 --> 00:46:54.260
right?
00:46:54.520 --> 00:46:57.500
So the answer to that is that, first of all,
00:46:58.860 --> 00:47:01.400
unfortunately, PySpy and any other provider of this kind,
00:47:01.520 --> 00:47:03.040
because it doesn't run inside your application,
00:47:03.500 --> 00:47:05.360
the trade-off is that it needs to learn
00:47:05.860 --> 00:47:07.380
how to analyze your application,
00:47:07.680 --> 00:47:07.800
right?
00:47:07.900 --> 00:47:10.360
And that is understanding the Python interpreter.
00:47:10.420 --> 00:47:13.100
But these parts of the Python interpreter are not public,
00:47:13.600 --> 00:47:13.680
right?
00:47:13.760 --> 00:47:14.680
Like we don't expose that.
00:47:14.730 --> 00:47:16.820
It needs to illegally access to them.
00:47:17.140 --> 00:47:22.220
And that means that there is a competing set of circumstances here
00:47:22.450 --> 00:47:23.320
because the core developers,
00:47:23.600 --> 00:47:24.060
first of all,
00:47:24.160 --> 00:47:28.600
we used to not care about this use case because it was not part of our APIs.
00:47:29.130 --> 00:47:31.880
So we keep changing things in ways that make it really, really hard
00:47:32.000 --> 00:47:35.440
for Biospy and other profiles like Austin to catch up with this.
00:47:35.650 --> 00:47:36.460
To the point that,
00:47:36.790 --> 00:47:38.179
for instance, it used to be the case
00:47:38.200 --> 00:47:41.580
that profiling 3.11 was much slower than profiling 3.10
00:47:41.860 --> 00:47:44.340
because 3.11 made changes that make this really, really hard.
00:47:45.060 --> 00:47:46.020
And sometimes,
00:47:46.460 --> 00:47:48.640
I was keeping an eye to help these tools,
00:47:48.700 --> 00:47:50.340
but sometimes people were making changes
00:47:50.920 --> 00:47:52.420
that were making it impossible to profile
00:47:52.700 --> 00:47:54.460
because it was hiding this so much
00:47:54.640 --> 00:47:55.920
that you couldn't have access to it.
00:47:56.860 --> 00:47:59.280
So one advantage to have this on the standard library,
00:47:59.380 --> 00:48:03.160
of course, is that now this is something that we offer.
00:48:03.240 --> 00:48:06.959
And this profile is also like the reference implementation
00:48:06.980 --> 00:48:07.860
of such a profiler.
00:48:08.340 --> 00:48:10.460
So if we can do it, then anyone can do it, right?
00:48:10.620 --> 00:48:12.860
So that we ensure that this is a feature
00:48:13.100 --> 00:48:15.000
that Python offers and will never break.
00:48:15.300 --> 00:48:16.100
And if you want to,
00:48:16.260 --> 00:48:16.720
you know,
00:48:16.940 --> 00:48:18.520
make your profiler externally,
00:48:19.100 --> 00:48:21.100
then you just need to reimplement what we do,
00:48:21.340 --> 00:48:21.420
right?
00:48:21.520 --> 00:48:22.660
And then you always have a way.
00:48:22.700 --> 00:48:23.820
You can look at how we do it
00:48:23.840 --> 00:48:24.860
and then you can do it as well.
00:48:25.020 --> 00:48:26.160
So that is very important.
00:48:26.460 --> 00:48:27.640
The second part is that,
00:48:28.040 --> 00:48:28.220
you know,
00:48:28.440 --> 00:48:28.760
as many,
00:48:29.040 --> 00:48:30.100
there's many examples
00:48:30.280 --> 00:48:32.740
and this is a very delicate aspect of it,
00:48:32.780 --> 00:48:34.600
is that having this tool in the standard library,
00:48:34.760 --> 00:48:39.760
we consider now that this was like very important to have it in the set of like batteries included
00:48:39.920 --> 00:48:44.780
right because you know you need to like learn for a external tool like is that external tool going to
00:48:44.840 --> 00:48:50.700
be as with the quality that we expect and is going to like have all the edge cases that can have so
00:48:50.840 --> 00:48:55.740
because we are dealing with internal soft cpython we don't know in real time all the little
00:48:55.980 --> 00:49:00.479
intricacies intricacies of like handling these problems so we want to share that the tools
00:49:00.500 --> 00:49:03.640
that we offer are of the utmost quality that we can provide,
00:49:03.900 --> 00:49:04.000
right?
00:49:04.340 --> 00:49:06.960
Which is to say that it's not that we don't trust the others,
00:49:07.220 --> 00:49:07.300
right?
00:49:07.300 --> 00:49:11.600
But we want to ensure that users at least have a very good reason
00:49:11.860 --> 00:49:15.020
to use something that is built in.
00:49:15.380 --> 00:49:16.380
And then the third reason,
00:49:16.680 --> 00:49:19.700
which is piggybacking on what Laszlo was saying,
00:49:20.140 --> 00:49:24.380
is that at some point, we managed to achieve the same speed
00:49:24.500 --> 00:49:25.800
as any other tool outside,
00:49:26.300 --> 00:49:27.220
including PySpy and Austin,
00:49:27.460 --> 00:49:27.520
right?
00:49:27.820 --> 00:49:31.040
But then because we have an extra trick that they don't have,
00:49:31.380 --> 00:49:32.780
which is that we also control the interpreter,
00:49:33.220 --> 00:49:33.300
right?
00:49:33.360 --> 00:49:34.560
Because this is now in CPython.
00:49:34.820 --> 00:49:39.980
So now we could do changes to the interpreter with the only purpose to ensure the profiling is faster,
00:49:40.420 --> 00:49:40.520
right?
00:49:40.960 --> 00:49:43.400
And also, by the way, this means that anyone can also do that,
00:49:43.580 --> 00:49:43.720
right?
00:49:43.860 --> 00:49:44.600
So now anyone,
00:49:45.240 --> 00:49:46.500
like those other profilers can also,
00:49:46.880 --> 00:49:49.500
you know, now we say that this is the faster Python profiler,
00:49:49.800 --> 00:49:49.940
but,
00:49:50.240 --> 00:49:52.100
you know, I'm pretty sure this is not going to be for long.
00:49:52.820 --> 00:49:53.520
Spoilers in a second.
00:49:53.700 --> 00:49:58.260
but like part of the reason is because anyone can use the tricks is but it's just like we did
00:49:58.580 --> 00:50:02.120
you know we can ensure that those tricks work and those tricks are there so part of the reason we
00:50:02.220 --> 00:50:06.720
could go past what was possible before is because we can cheat a bit and we can like change the
00:50:06.900 --> 00:50:11.680
interpreter such that like it exposes easier information and some of those tricks are really
00:50:11.840 --> 00:50:17.800
really cool and complicated like the way you know it can go faster is really interesting
00:50:18.300 --> 00:50:22.999
perhaps i don't know if we can explain it here but like uh they're quite advanced but like anyone
00:50:23.020 --> 00:50:25.520
can pull those tricks if they know how it works, right?
00:50:25.620 --> 00:50:29.660
It's just that they need to ensure that they implement those things.
00:50:29.960 --> 00:50:30.120
Yeah,
00:50:30.340 --> 00:50:30.700
very interesting.
00:50:31.720 --> 00:50:34.060
I don't know if we actually called out the title,
00:50:34.260 --> 00:50:38.540
but PEP768 is a safe external bugger interface for CPython.
00:50:38.740 --> 00:50:40.980
And debugger means like, hey, what are you doing?
00:50:41.220 --> 00:50:43.700
But that's also kind of a profiling thing of your only purpose
00:50:43.920 --> 00:50:44.740
just to say, what are you doing?
00:50:45.200 --> 00:50:45.640
Where are you?
00:50:45.760 --> 00:50:46.000
Where are you?
00:50:46.060 --> 00:50:46.480
Where are you?
00:50:46.640 --> 00:50:48.320
Like over and over a million times a second,
00:50:48.780 --> 00:50:50.380
whatever frequency you want to run at, right?
00:50:50.680 --> 00:50:50.820
Right.
00:50:51.140 --> 00:50:57.320
There's another interesting aspect where a built-in sampling profiler in CPython is super useful,
00:50:57.470 --> 00:51:01.620
which is that, and I ran into this myself as well, that when you're working on CPython itself,
00:51:02.010 --> 00:51:04.960
you don't have a profiler which you can use to measure performance.
00:51:05.550 --> 00:51:08.400
Because typically the third-party profilers will,
00:51:08.620 --> 00:51:12.340
when you're in an alpha phase of the changes,
00:51:12.800 --> 00:51:17.660
typically those profilers wouldn't support that version of CPython.
00:51:17.920 --> 00:51:20.740
Now you will have a something profiler which works,
00:51:21.290 --> 00:51:23.140
which is within NCPython itself.
00:51:23.960 --> 00:51:26.880
I run into this working on the profiler itself.
00:51:27.420 --> 00:51:32.560
How do you measure the performance of the profiling code,
00:51:32.660 --> 00:51:33.860
like all of those Python,
00:51:34.140 --> 00:51:36.820
fairly complex Python code?
00:51:37.340 --> 00:51:41.140
You can't really do that without having a profiler built in.
00:51:41.500 --> 00:51:43.960
So that's another nice effect of having a something
00:51:44.140 --> 00:51:46.980
profile built into the standard library itself.
00:51:47.280 --> 00:51:51.440
Yeah. And as Pablo said, you can optimize for it so that now that you have a stable interface,
00:51:51.950 --> 00:51:54.860
we can say, okay, this is the thing we're going to optimize with everyone else just randomly
00:51:55.200 --> 00:51:58.140
poking around the bytes and my code and all that kind of stuff.
00:51:58.340 --> 00:51:58.640
It's like,
00:51:58.760 --> 00:52:01.500
well, you still need to poke around the bytes on the bytecode. It's just like,
00:52:01.620 --> 00:52:02.140
at least now,
00:52:02.460 --> 00:52:03.040
you know, it's possible.
00:52:03.240 --> 00:52:04.640
You know, you can follow the trail,
00:52:04.790 --> 00:52:08.340
you can follow the soldiers of giants kind of thing, you know?
00:52:08.600 --> 00:52:08.880
Yeah,
00:52:09.180 --> 00:52:09.420
exactly.
00:52:10.300 --> 00:52:10.360
Exactly.
00:52:11.140 --> 00:52:11.880
It is interesting.
00:52:12.040 --> 00:52:14.319
You could use it to figure out how stuff runs on the
00:52:14.340 --> 00:52:14.860
newer versions,
00:52:15.340 --> 00:52:17.240
because even though maybe the shape of stuff has changed,
00:52:17.460 --> 00:52:18.660
it's still the same API.
00:52:18.760 --> 00:52:18.820
Oh,
00:52:19.000 --> 00:52:19.560
actually,
00:52:20.020 --> 00:52:20.800
yeah.
00:52:21.000 --> 00:52:23.080
And also just to clarify another thing,
00:52:23.240 --> 00:52:25.260
like right now, which is also
00:52:25.480 --> 00:52:29.000
important and it's also an advantage of the other profilers is that, of course,
00:52:29.420 --> 00:52:30.300
this is using this
00:52:30.480 --> 00:52:32.360
extra technology that we added to 3.15,
00:52:32.700 --> 00:52:34.680
which means that it only runs in 3.15.
00:52:34.880 --> 00:52:35.680
So you can only
00:52:35.840 --> 00:52:38.380
analyze 3.15 code from 3.15,
00:52:39.080 --> 00:52:40.700
while PySpy works with any Python.
00:52:41.080 --> 00:52:43.439
So right now you want to run
00:52:43.460 --> 00:52:44.780
PiSpy on 3.12,
00:52:44.960 --> 00:52:45.280
you can.
00:52:45.580 --> 00:52:46.920
Or 3.13 or 3.14.
00:52:47.340 --> 00:52:49.140
You know, with different performance
00:52:49.460 --> 00:52:51.280
characteristics for sure, but you can
00:52:51.380 --> 00:52:51.720
do that.
00:52:52.120 --> 00:52:53.440
Or Austin or whatever
00:52:53.480 --> 00:52:54.280
of the other ones, right?
00:52:55.820 --> 00:52:57.320
But right now, this one
00:52:57.520 --> 00:52:59.220
only works from 3.15 forward,
00:52:59.340 --> 00:52:59.460
right?
00:52:59.780 --> 00:53:01.140
And between the same version.
00:53:01.500 --> 00:53:03.440
So that's an important
00:53:03.660 --> 00:53:03.720
thing.
00:53:03.880 --> 00:53:05.340
It's not a limitation for the
00:53:05.580 --> 00:53:05.660
future.
00:53:05.860 --> 00:53:07.280
It's possible that we can
00:53:07.500 --> 00:53:09.000
allow 3.15 to analyze forward.
00:53:09.600 --> 00:53:09.760
Sorry,
00:53:11.120 --> 00:53:12.180
backward for 3.15,
00:53:12.440 --> 00:53:13.420
so 3.16 can
00:53:13.440 --> 00:53:15.480
3.15 and 3.16 or
00:53:15.540 --> 00:53:17.700
3.17 can do 3.15, 3.16
00:53:17.780 --> 00:53:18.440
and 3.17
00:53:19.480 --> 00:53:21.480
but still we don't know. We don't know
00:53:21.480 --> 00:53:23.280
if we want to get into that
00:53:24.060 --> 00:53:25.320
but yeah, that's another advantage
00:53:26.120 --> 00:53:27.500
but if you go to the
00:53:27.800 --> 00:53:28.480
I don't know if you,
00:53:28.800 --> 00:53:29.520
I can see that
00:53:29.600 --> 00:53:31.400
Hold on really quick before we move on
00:53:31.580 --> 00:53:33.520
what about this Python backport
00:53:33.640 --> 00:53:33.780
stuff?
00:53:34.120 --> 00:53:34.840
Are you familiar with this?
00:53:35.040 --> 00:53:35.640
Python profiling?
00:53:36.940 --> 00:53:38.540
Backporting the new profiling library
00:53:38.940 --> 00:53:39.100
to,
00:53:39.520 --> 00:53:40.620
I don't know what version goes
00:53:40.760 --> 00:53:41.420
to 3.14.
00:53:41.960 --> 00:53:45.740
Like a Clanker made version of this, like, or something like a version?
00:53:45.740 --> 00:53:46.460
Let's see.
00:53:47.100 --> 00:53:50.200
Yeah, possibly it was made over a one-week period.
00:53:50.440 --> 00:53:50.560
So,
00:53:50.760 --> 00:53:51.400
yeah, probably.
00:53:51.860 --> 00:53:52.000
Yeah.
00:53:52.540 --> 00:53:54.520
Well, I mean, it is possible because,
00:53:54.760 --> 00:53:55.240
technically speaking,
00:53:55.740 --> 00:53:57.520
you can,
00:53:58.360 --> 00:53:59.820
like, you go up in this repo.
00:53:59.820 --> 00:54:01.340
We are looking at the GitHub repo.
00:54:01.380 --> 00:54:02.760
So, you go up in this page.
00:54:03.140 --> 00:54:05.500
You can see that there's this underscore remote debugging.
00:54:05.900 --> 00:54:06.020
So,
00:54:06.180 --> 00:54:08.240
that is the C-Python folder,
00:54:08.440 --> 00:54:08.560
right?
00:54:08.760 --> 00:54:11.240
So probably what happened here is that they copy this code
00:54:11.560 --> 00:54:13.360
and they strip all the new cool,
00:54:13.540 --> 00:54:14.040
you know,
00:54:14.600 --> 00:54:15.860
bells and whistles for 3.15.
00:54:16.460 --> 00:54:18.380
And that still works very well for 3.14.
00:54:19.680 --> 00:54:21.080
I don't know if it works for less.
00:54:21.290 --> 00:54:22.060
So I will be interested.
00:54:22.200 --> 00:54:23.240
Can you check the Python version,
00:54:23.580 --> 00:54:23.700
perhaps,
00:54:23.790 --> 00:54:25.120
in the set of py or something?
00:54:25.500 --> 00:54:25.640
Yeah,
00:54:25.800 --> 00:54:25.920
sure.
00:54:26.100 --> 00:54:27.120
Py project.
00:54:27.380 --> 00:54:28.400
Oh, they will set py.
00:54:28.540 --> 00:54:29.140
Wow, how about that?
00:54:29.270 --> 00:54:29.380
Okay.
00:54:30.820 --> 00:54:31.240
Yeah, okay.
00:54:31.500 --> 00:54:31.600
Well,
00:54:32.140 --> 00:54:32.360
interesting.
00:54:32.570 --> 00:54:33.500
Yeah, 3.14, you see?
00:54:34.279 --> 00:54:34.640
So,
00:54:35.050 --> 00:54:37.060
yeah, it supports 3.14 only because,
00:54:37.500 --> 00:54:37.620
you know,
00:54:37.700 --> 00:54:39.640
It's the 3.0 version minus the cool things.
00:54:39.900 --> 00:54:42.440
So it's still using the debugging interface.
00:54:43.580 --> 00:54:46.300
You can see the bag obsessed validation.h,
00:54:46.660 --> 00:54:48.280
which is clearly the bag stuff.
00:54:49.660 --> 00:54:51.040
So that is possible.
00:54:51.200 --> 00:54:51.520
Unfortunately,
00:54:51.980 --> 00:54:52.940
as you really well know,
00:54:52.940 --> 00:54:54.980
we cannot backport features in CPython.
00:54:55.120 --> 00:54:56.520
So this is good.
00:54:56.570 --> 00:54:57.020
This is good.
00:54:57.140 --> 00:54:59.000
I mean, I'm happy people are doing this.
00:54:59.000 --> 00:55:01.660
So maybe it gives you a little bit of a lever of today and tomorrow.
00:55:02.600 --> 00:55:06.100
Yeah, and this makes me happy,
00:55:06.300 --> 00:55:06.400
right?
00:55:06.600 --> 00:55:12.100
Like, look, man, if people are willing to like pull tricks to use our stuff in all versions,
00:55:12.390 --> 00:55:12.860
like amazing.
00:55:13.050 --> 00:55:13.580
You know what I mean?
00:55:13.800 --> 00:55:14.840
Like that's success to me.
00:55:15.860 --> 00:55:17.880
What's worse is they don't care and they don't pay any attention.
00:55:17.990 --> 00:55:18.720
They just leave it, right?
00:55:19.000 --> 00:55:19.400
Yeah, yeah.
00:55:19.770 --> 00:55:22.160
Well, you know, who the bugs are going to go to?
00:55:22.340 --> 00:55:23.380
Like that's an interesting question.
00:55:23.860 --> 00:55:24.160
But like,
00:55:24.640 --> 00:55:24.920
you know,
00:55:25.300 --> 00:55:27.320
if you report a 3.14 bug in Cpython,
00:55:27.540 --> 00:55:28.660
I may tell you, hey, man, what
00:55:28.660 --> 00:55:29.060
are you doing?
00:55:29.820 --> 00:55:30.200
But like,
00:55:30.860 --> 00:55:31.040
you know,
00:55:31.560 --> 00:55:34.000
if they are willing to bug for the bugs as well,
00:55:34.220 --> 00:55:34.840
then let's go.
00:55:34.910 --> 00:55:36.260
You know, like that's good to me.
00:55:36.560 --> 00:55:38.340
So I actually did this.
00:55:38.480 --> 00:55:39.580
So I tried doing this.
00:55:39.740 --> 00:55:40.560
This is not my project,
00:55:41.180 --> 00:55:45.200
of course, but I tried this because this seemed like a good idea.
00:55:45.360 --> 00:55:47.180
Hey, how can we make this work in 3.14?
00:55:47.780 --> 00:55:48.680
And that was the idea.
00:55:48.780 --> 00:55:53.420
You can just literally take the profiling package and just copy that over.
00:55:53.640 --> 00:55:54.800
But it's not that simple.
00:55:54.980 --> 00:56:01.320
So there are some changes in internal CPython structures in CLO.
00:56:03.360 --> 00:56:06.400
It's not a lot of changes, but you do get some changes there.
00:56:06.770 --> 00:56:10.200
You may be able to get away without those, but the patch is enormous.
00:56:10.580 --> 00:56:11.840
It's absolutely gigantic.
00:56:11.970 --> 00:56:15.720
I did not realize how much code there is in Tachyon until I was doing this.
00:56:16.100 --> 00:56:18.620
And that just means that every single patch version, every single change,
00:56:18.780 --> 00:56:20.440
you will have to maintain this.
00:56:20.470 --> 00:56:21.740
There may be significant maintenance.
00:56:22.380 --> 00:56:24.240
So the idea was rejected,
00:56:25.090 --> 00:56:27.580
sort of fact, to do this and maintain it as a separate package,
00:56:28.620 --> 00:56:29.380
at least for me.
00:56:29.640 --> 00:56:31.380
I don't see this as a long-term,
00:56:31.660 --> 00:56:33.660
this back thing as a long-term useful thing,
00:56:33.940 --> 00:56:35.300
but maybe in the first year,
00:56:35.700 --> 00:56:35.900
you know,
00:56:36.020 --> 00:56:37.520
as people are searching for 3.15.
00:56:37.720 --> 00:56:38.480
Of course, of course,
00:56:38.720 --> 00:56:38.800
yeah.
00:56:39.340 --> 00:56:40.380
And then in three years, you tell them,
00:56:40.380 --> 00:56:41.180
like, what are you doing 3.14?
00:56:41.290 --> 00:56:41.660
Get out of here.
00:56:42.040 --> 00:56:43.440
But, but, but,
00:56:43.740 --> 00:56:43.920
but,
00:56:44.080 --> 00:56:44.520
ooh,
00:56:44.860 --> 00:56:46.120
maybe we can talk about this later.
00:56:46.400 --> 00:56:48.820
But like, one interesting thing,
00:56:48.940 --> 00:56:51.340
well, actually, let me segue into it smoothly,
00:56:51.540 --> 00:56:51.740
smoothly.
00:56:51.900 --> 00:56:53.880
Let's do some smooth segue into it.
00:56:54.440 --> 00:56:58.360
So one interesting thing that is also important to mention,
00:56:58.900 --> 00:57:02.500
And I can see that you have the profiling.sampling documentation in the sidebar.
00:57:02.660 --> 00:57:06.180
So let's open that for the people viewing this in YouTube or whatever.
00:57:06.700 --> 00:57:09.060
So the documentation here is huge,
00:57:09.280 --> 00:57:09.360
huge,
00:57:09.620 --> 00:57:10.140
huge documentation.
00:57:10.580 --> 00:57:11.040
Why is that?
00:57:11.260 --> 00:57:12.660
Because this does a lot of things.
00:57:12.880 --> 00:57:15.980
It does a lot of things, a lot of things, by the way, that only profiler can do.
00:57:16.120 --> 00:57:17.500
So it's not just about being fast.
00:57:17.600 --> 00:57:20.060
It does all sorts of other things.
00:57:20.380 --> 00:57:24.380
So as opposed to like cprofile that like has like,
00:57:24.560 --> 00:57:25.799
you know, the PSTAT format
00:57:25.820 --> 00:57:26.920
and can tell you like basic,
00:57:27.300 --> 00:57:28.920
you know, statistics about your program or whatever.
00:57:29.410 --> 00:57:31.920
Like this thing is a full beast.
00:57:32.030 --> 00:57:32.860
It comes with everything,
00:57:33.080 --> 00:57:33.160
right?
00:57:33.410 --> 00:57:37.480
So here Tachyon can create like, like flame graphs.
00:57:37.770 --> 00:57:39.500
It can create all sorts of visualizations.
00:57:39.770 --> 00:57:41.060
Like, so it has that top model.
00:57:41.170 --> 00:57:44.040
So like, like when you do top and you see the processes life.
00:57:44.420 --> 00:57:46.580
So Tachyon has this like, like Tachyon life.
00:57:47.060 --> 00:57:49.740
Well, I can show you like the functions life of your application,
00:57:50.180 --> 00:57:51.100
like almost like a top,
00:57:51.310 --> 00:57:52.940
it can create like heat maps.
00:57:53.200 --> 00:58:01.040
So you can see lines with different colors depending if they are more hot or cold, if they are more slow or fast.
00:58:01.860 --> 00:58:05.120
You can do differential frame graphs, which is very cool.
00:58:05.300 --> 00:58:12.900
You can run your program and then change a bit and run it again and then compare what became faster and slower.
00:58:13.080 --> 00:58:16.840
So it can tell you what functions were faster and what functions got slower.
00:58:17.300 --> 00:58:20.420
Another very cool thing that this does is that it does async audio,
00:58:20.640 --> 00:58:21.180
which is the first
00:58:21.420 --> 00:58:25.920
profiler right now that like can also do async audio because like, you know,
00:58:26.080 --> 00:58:26.560
the even loop
00:58:26.780 --> 00:58:28.060
does all sorts of weird things.
00:58:28.900 --> 00:58:29.260
Right.
00:58:29.400 --> 00:58:30.680
And it's often one thread,
00:58:31.110 --> 00:58:32.780
basically kind of async call stack.
00:58:32.890 --> 00:58:33.700
So it's like, what are you doing?
00:58:33.820 --> 00:58:34.740
Like, well, we're dispatching.
00:58:35.320 --> 00:58:35.540
Yeah,
00:58:35.720 --> 00:58:35.800
exactly.
00:58:35.960 --> 00:58:36.060
Exactly.
00:58:36.100 --> 00:58:36.660
What is that?
00:58:36.720 --> 00:58:36.980
What is that?
00:58:37.110 --> 00:58:37.200
Exactly.
00:58:37.520 --> 00:58:37.580
Yeah.
00:58:38.040 --> 00:58:38.940
Or like, you know, it's like,
00:58:39.160 --> 00:58:39.280
oh,
00:58:40.040 --> 00:58:40.200
you know,
00:58:40.570 --> 00:58:42.140
but it's also important because I think
00:58:42.200 --> 00:58:45.660
I do changes a bit the, what means slow or fast.
00:58:45.820 --> 00:58:50.480
Like, for example, imagine that you're downloading a 3GB file,
00:58:50.810 --> 00:58:51.820
but the server is slow.
00:58:52.220 --> 00:58:57.680
So you're spending a lot of time waiting for that 3GB file, not because you are crunching
00:58:57.730 --> 00:58:58.200
your CPU,
00:58:58.440 --> 00:58:59.680
it's because your server is slow.
00:58:59.920 --> 00:59:02.340
But your application is slow because the server is slow.
00:59:02.660 --> 00:59:04.540
So how do you know that that is happening?
00:59:04.980 --> 00:59:09.200
Well, in the normal world, if that was the case, you would be like, I don't know, reading
00:59:09.290 --> 00:59:09.720
or something.
00:59:10.040 --> 00:59:13.300
Like, you will see that the profile will show you that you are reading, reading, reading,
00:59:13.540 --> 00:59:14.880
so you can't imagine what's going on.
00:59:15.200 --> 00:59:15.820
But in this case,
00:59:16.000 --> 00:59:20.220
because the event loop will only be waking up when the server has something to tell you.
00:59:20.640 --> 00:59:20.860
Meanwhile,
00:59:21.060 --> 00:59:22.220
that will never appear.
00:59:22.540 --> 00:59:25.140
So like if you can only see the coroutines that are running,
00:59:25.600 --> 00:59:30.100
the fact that you are waiting for the server to send your data will never appear in your flame graph because it's not happening.
00:59:30.480 --> 00:59:37.040
So you will never know that what is slow about your program is that your program is waiting for a three-hour file to appear,
00:59:37.360 --> 00:59:37.460
right?
00:59:37.820 --> 00:59:38.060
Yeah,
00:59:38.820 --> 00:59:41.100
this is especially important for a sampling profiler,
00:59:41.300 --> 00:59:41.380
right?
00:59:41.680 --> 00:59:43.320
Exactly, exactly, because you are not tracing it.
00:59:43.400 --> 00:59:48.600
Also tracing in the even loop is very weird because you keep changing for task to task, so it's very confusing.
00:59:49.100 --> 00:59:51.760
But here in Taki has these two modes.
00:59:52.240 --> 00:59:54.780
One of them is like tasks that are running.
00:59:55.000 --> 00:59:56.680
So you can see only running tasks.
00:59:56.840 --> 00:59:59.900
So in that case, for instance, if you're waiting for the 3D guy file, it will not appear.
01:00:00.300 --> 01:00:02.200
But you can see that meanwhile you're doing other things.
01:00:02.600 --> 01:00:05.720
But also it has this mode, which is all the tasks that I'm waiting for.
01:00:06.180 --> 01:00:07.980
So every time you sample,
01:00:08.140 --> 01:00:10.320
you see every single task that is going on.
01:00:10.660 --> 01:00:11.660
So you can see that like,
01:00:12.000 --> 01:00:12.080
oh,
01:00:12.300 --> 01:00:13.740
I'm waiting for this 3G file.
01:00:14.140 --> 01:00:15.020
It's just that it's not advancing.
01:00:15.280 --> 01:00:17.820
So because it's not advancing, it will appear in more samples.
01:00:18.310 --> 01:00:22.220
And then you will know that like that guy is like all the time being waiting for.
01:00:22.230 --> 01:00:23.460
So like, you know that it's happening.
01:00:24.420 --> 01:00:27.900
And then apart from that, it also has like dump mode when you can have like,
01:00:28.240 --> 01:00:28.360
you know,
01:00:28.570 --> 01:00:31.900
imagine that your application is frozen because you have a deadlock or perhaps,
01:00:32.060 --> 01:00:32.400
you know,
01:00:32.800 --> 01:00:36.580
you know, it's just waiting for IO that never arrives and you want to know where it's going.
01:00:37.040 --> 01:00:41.020
so you can use profiling.sampling dump
01:00:41.360 --> 01:00:43.200
and then you pass the PID of your application
01:00:43.670 --> 01:00:46.040
and then it will show you what's going on.
01:00:46.110 --> 01:00:47.100
It will show you the stack trace.
01:00:47.560 --> 01:00:48.540
Imagine the stack trace,
01:00:48.760 --> 01:00:50.500
like if an exception is thrown,
01:00:50.660 --> 01:00:51.820
but instead of throwing an exception,
01:00:51.890 --> 01:00:52.920
it will show you where it is.
01:00:53.300 --> 01:00:53.760
And of course,
01:00:53.980 --> 01:00:55.180
this works on everywhere.
01:00:55.480 --> 01:00:56.400
So it works on macOS,
01:00:56.590 --> 01:00:57.220
it works on Windows,
01:00:57.430 --> 01:00:58.160
it works on Linux.
01:00:58.820 --> 01:00:59.500
Same mechanism,
01:00:59.800 --> 01:01:00.360
same guarantees,
01:01:00.620 --> 01:01:01.040
same everything.
01:01:02.279 --> 01:01:03.140
Okay, that's awesome.
01:01:03.300 --> 01:01:03.380
Now,
01:01:03.590 --> 01:01:05.099
you threw out a bunch of stuff there
01:01:05.260 --> 01:01:07.140
And I think it's amazing how much is here,
01:01:07.320 --> 01:01:09.740
but let's just go back really quick and just,
01:01:10.180 --> 01:01:11.040
let's just focus on this.
01:01:11.200 --> 01:01:11.900
I think this is,
01:01:12.060 --> 01:01:12.920
this kind of game changing.
01:01:12.920 --> 01:01:13.600
Yeah, there is so much.
01:01:13.780 --> 01:01:15.400
There is, you need a second podcast,
01:01:15.640 --> 01:01:15.740
man.
01:01:15.820 --> 01:01:16.980
Like you need to invite us a second time.
01:01:17.000 --> 01:01:17.560
What do we even do?
01:01:17.680 --> 01:01:19.480
Like we got to just, we're going to go, we're going to go,
01:01:19.680 --> 01:01:19.960
keep going.
01:01:20.420 --> 01:01:21.600
So the attach,
01:01:22.120 --> 01:01:22.320
this is,
01:01:22.980 --> 01:01:24.120
there's something running in production.
01:01:24.600 --> 01:01:24.800
Let me,
01:01:25.440 --> 01:01:27.480
right now it's doing whatever it is that we're like,
01:01:27.640 --> 01:01:27.780
uh,
01:01:28.060 --> 01:01:28.640
what is this doing?
01:01:28.780 --> 01:01:29.100
What's going on?
01:01:29.180 --> 01:01:29.840
How do we fix this?
01:01:29.960 --> 01:01:30.940
Or just what's going on?
01:01:30.980 --> 01:01:32.200
you can attach to it,
01:01:32.740 --> 01:01:34.460
grab a pstat file,
01:01:34.900 --> 01:01:36.220
take that away and work on it, right?
01:01:36.590 --> 01:01:36.680
Without,
01:01:36.940 --> 01:01:39.740
I mean, it probably does affect the performance somehow,
01:01:40.439 --> 01:01:42.060
but not nearly the same.
01:01:42.660 --> 01:01:43.320
No, no, it's free.
01:01:43.600 --> 01:01:43.880
It's free.
01:01:43.880 --> 01:01:44.160
It's free.
01:01:44.650 --> 01:01:44.760
Okay.
01:01:45.000 --> 01:01:45.320
It's free.
01:01:45.920 --> 01:01:47.740
The key way it's free is because,
01:01:47.900 --> 01:01:47.980
well,
01:01:48.170 --> 01:01:48.500
okay, sorry.
01:01:48.600 --> 01:01:49.240
It's free by default.
01:01:49.560 --> 01:01:51.540
There is something that you can do.
01:01:51.880 --> 01:01:53.160
We can't go into detail in a second,
01:01:53.340 --> 01:01:57.900
but there's something that you can do to kind of like have more exact data
01:01:58.440 --> 01:01:59.780
that will slow it down a bit.
01:02:00.120 --> 01:02:00.400
But still,
01:02:00.740 --> 01:02:02.580
for most of the sampling rates,
01:02:02.740 --> 01:02:03.360
it's mostly free.
01:02:03.870 --> 01:02:04.440
So by default,
01:02:04.900 --> 01:02:07.280
Acheon will never stop your application.
01:02:07.530 --> 01:02:09.420
It will just always read what is going on.
01:02:09.540 --> 01:02:12.500
Of course, the trade-off is that you can do partial reads.
01:02:12.580 --> 01:02:14.360
So perhaps it's reading,
01:02:14.560 --> 01:02:17.420
a structure that's changing, so it gets some weird stuff.
01:02:17.620 --> 01:02:18.040
So Acheon,
01:02:18.540 --> 01:02:21.680
most of the time, will detect that that is happening and it will throw away the sample.
01:02:21.890 --> 01:02:23.180
So it will say, okay, this is a bad sample.
01:02:23.420 --> 01:02:24.560
And it will not read it.
01:02:24.700 --> 01:02:27.520
But of course, that means that you have less samples if that is happening a lot.
01:02:27.600 --> 01:02:29.840
That can happen, for example, you have a lot of generators,
01:02:30.590 --> 01:02:32.360
because generators tend to be a bit like,
01:02:32.700 --> 01:02:33.940
you know, you get partial reads.
01:02:34.270 --> 01:02:35.100
So sometimes it happens.
01:02:35.820 --> 01:02:36.300
But by default,
01:02:36.540 --> 01:02:38.060
it will only read from the application.
01:02:38.210 --> 01:02:38.700
And the application,
01:02:38.960 --> 01:02:41.000
first of all, the application doesn't know that this is happening.
01:02:41.440 --> 01:02:41.680
And,
01:02:42.050 --> 01:02:45.600
you know, unless you only have one CPU and then you are giving the CPU to both programs,
01:02:45.860 --> 01:02:47.600
which in today's computers,
01:02:48.150 --> 01:02:48.820
that will never happen.
01:02:49.820 --> 01:02:53.080
I don't know, you're running a Docker container with only one CPU core or whatever.
01:02:53.160 --> 01:02:53.480
but like,
01:02:53.660 --> 01:02:55.200
you know, you run the profile in another,
01:02:55.420 --> 01:02:57.580
you know, sidecar or whatever, like that would ever happen.
01:02:58.320 --> 01:02:59.140
So unless that's the case,
01:02:59.320 --> 01:03:01.500
like the application will never know and it will run at full speed.
01:03:02.400 --> 01:03:07.820
If you want to avoid this kind of partial read situation and you want to show that every read is consistent
01:03:08.180 --> 01:03:12.700
and increase the kind of like exactness and, you know,
01:03:12.880 --> 01:03:14.100
trustworthiness of the data,
01:03:14.640 --> 01:03:17.180
then there is this mode where you can pass does this blocking.
01:03:17.640 --> 01:03:19.780
And what does this blocking will do is that before reading,
01:03:19.940 --> 01:03:22.400
it will stop the application for a split millisecond,
01:03:22.720 --> 01:03:23.260
very quickly,
01:03:23.570 --> 01:03:25.400
it will do the read and it will restart it.
01:03:25.680 --> 01:03:25.800
So,
01:03:26.120 --> 01:03:28.440
of course, if you do the default sampling rate
01:03:28.640 --> 01:03:31.420
or even sampling rates up to 10,000 Hz,
01:03:31.840 --> 01:03:33.340
that is a very small amount of time.
01:03:33.560 --> 01:03:35.540
We are talking about slowdowns of 1% to 2%.
01:03:36.200 --> 01:03:38.180
So 1% to 2%, not two times.
01:03:38.260 --> 01:03:39.760
It's 1% to 2%. It's nothing.
01:03:40.260 --> 01:03:42.480
If you've done profiling before, it's much,
01:03:42.980 --> 01:03:43.600
much lower.
01:03:43.620 --> 01:03:43.680
It's like,
01:03:44.260 --> 01:03:44.480
whatever,
01:03:44.880 --> 01:03:45.020
man.
01:03:45.450 --> 01:03:46.280
What are you talking about?
01:03:48.380 --> 01:03:51.040
We would make Parthons 315 5% faster,
01:03:51.180 --> 01:03:52.180
and then that's fine, right?
01:03:52.420 --> 01:03:55.780
But like if you ramp the sample rate really high,
01:03:55.960 --> 01:03:58.780
like let's say 100,000 samples per second,
01:03:59.080 --> 01:04:02.800
then we are talking about like more, like 20% or 30% or 80%.
01:04:02.940 --> 01:04:05.980
I think that you do it at crazy rates, like 1 million samples,
01:04:06.540 --> 01:04:07.600
if your machine can do it,
01:04:07.660 --> 01:04:10.700
because now it depends on your machine to be able to do that.
01:04:11.100 --> 01:04:14.240
Then we are talking about two times lower or something like that.
01:04:14.560 --> 01:04:17.140
But like you see, like you need to like go really, really crazy.
01:04:17.640 --> 01:04:18.980
So in production, for example,
01:04:19.160 --> 01:04:23.680
you can sample perfectly at 100 or 1000 hertz and have a slow
01:04:23.780 --> 01:04:24.800
dance of one or two percent.
01:04:24.980 --> 01:04:26.160
And it's fine. It's acceptable.
01:04:27.280 --> 01:04:29.020
And this thing that we're talking
01:04:29.180 --> 01:04:29.280
about,
01:04:29.540 --> 01:04:30.820
this thing is so important.
01:04:31.080 --> 01:04:33.860
You don't understand how important this is. Like I'm free
01:04:33.860 --> 01:04:35.780
to say this is not a trade secret.
01:04:35.940 --> 01:04:38.820
Like my employer used this thing all the fucking time
01:04:39.420 --> 01:04:40.220
for making a lot of money.
01:04:40.500 --> 01:04:45.779
So you cannot imagine how critical it is to understand why your programs
01:04:45.800 --> 01:04:48.700
like are slow or how your programs can be optimized.
01:04:49.190 --> 01:04:51.820
Because a lot of the time you just need to see live traffic.
01:04:52.170 --> 01:04:53.720
Like if you have or like live data,
01:04:53.920 --> 01:04:53.980
right?
01:04:54.260 --> 01:04:58.720
If you never know how your application is really going to behave or what is slow
01:04:58.910 --> 01:05:00.540
until you measure the real deal.
01:05:00.730 --> 01:05:02.240
You cannot measure the fake deal.
01:05:02.410 --> 01:05:03.940
Like you need to measure the real deal.
01:05:04.360 --> 01:05:07.880
And for that, you need to attach because most of the times you don't have access to,
01:05:08.210 --> 01:05:10.460
oh, I just want to launch my application on the lab profiler.
01:05:10.620 --> 01:05:13.780
No, you need to like, you just have it there and it's doing something weird.
01:05:14.140 --> 01:05:15.600
And then you attach and then you look a bit.
01:05:15.740 --> 01:05:16.320
and then you detach,
01:05:16.540 --> 01:05:16.660
right?
01:05:16.820 --> 01:05:17.180
Even this,
01:05:17.680 --> 01:05:18.920
let's say for example,
01:05:19.720 --> 01:05:21.940
that blocking was 50% slowdown.
01:05:22.100 --> 01:05:23.680
You could do it for like a bunch of seconds
01:05:23.940 --> 01:05:25.220
and then detach and you know,
01:05:25.420 --> 01:05:26.020
here is all good.
01:05:26.300 --> 01:05:27.300
And then you got the information
01:05:27.760 --> 01:05:28.960
and then you can reason about it.
01:05:29.260 --> 01:05:29.740
And then you can manage - That
01:05:29.740 --> 01:05:31.180
information is so good,
01:05:31.340 --> 01:05:31.540
Pablo.
01:05:32.040 --> 01:05:32.360
The in
01:05:32.360 --> 01:05:34.300
-production information is so much different.
01:05:34.820 --> 01:05:34.880
It's
01:05:34.880 --> 01:05:35.940
key, it's key, it's key.
01:05:36.180 --> 01:05:38.200
Like for this
01:05:38.200 --> 01:05:39.480
year, I mean, this is kind of the segue
01:05:39.720 --> 01:05:40.140
and the secret
01:05:40.140 --> 01:05:40.520
and whatever,
01:05:40.760 --> 01:05:41.360
but like one
01:05:41.360 --> 01:05:43.980
of the things I've been working on here,
01:05:44.780 --> 01:05:48.720
had solar training for like this past year is precisely all about getting,
01:05:49.260 --> 01:05:49.880
well, not all about,
01:05:50.080 --> 01:05:50.480
but like, like,
01:05:50.660 --> 01:05:55.160
but a lot of my work here has been to ensure that getting that information and much
01:05:55.280 --> 01:05:57.380
more from light data is what you do.
01:05:57.900 --> 01:05:59.780
And we can go into why it's insane here. But like,
01:06:00.220 --> 01:06:00.820
part of the
01:06:00.940 --> 01:06:03.960
reason is because this information is gold,
01:06:04.280 --> 01:06:06.140
like, it's just the best thing that you can do,
01:06:06.320 --> 01:06:06.460
because
01:06:06.920 --> 01:06:08.820
it's telling you literally what you need to change.
01:06:09.120 --> 01:06:09.840
It's almost cheating.
01:06:10.120 --> 01:06:10.640
You know what I mean?
01:06:14.340 --> 01:06:14.740
You
01:06:14.760 --> 01:06:21.360
bad change this thing and then you go and like literally today today right we activated a mega
01:06:21.560 --> 01:06:25.720
version of this which is not tacky only something else but we activated a version of this at work
01:06:26.120 --> 01:06:30.740
in one of the machines that the data scientists used to like the supercomputers that we have here
01:06:30.740 --> 01:06:35.480
so here we have a lot of supercomputers and then like people do their their you know the experiments
01:06:35.760 --> 01:06:41.380
or things with trading and you know that's like i don't know it's war like it's huge machines
01:06:41.660 --> 01:06:43.820
like hundreds and hundreds of gigabytes of memory,
01:06:44.400 --> 01:06:45.380
like hundreds of cores.
01:06:45.880 --> 01:06:47.040
So it's like war, right?
01:06:47.310 --> 01:06:48.520
And then for the first time,
01:06:48.690 --> 01:06:50.460
because this is so ridiculously complicated
01:06:50.660 --> 01:06:53.860
and these machines are so big that like it's so different to do this thing at scale,
01:06:54.440 --> 01:06:57.180
we managed to have a version of this that is tracing all programs.
01:06:57.450 --> 01:06:59.020
So not just one, all of them,
01:06:59.240 --> 01:06:59.320
right?
01:06:59.620 --> 01:07:01.740
And using this live data, literally today,
01:07:02.180 --> 01:07:05.900
we saw one of these applications that was doing something slightly odd.
01:07:06.120 --> 01:07:06.300
It's like,
01:07:06.520 --> 01:07:10.240
why is this application taking like an entire core of the machine
01:07:10.280 --> 01:07:11.540
when it should be doing nothing.
01:07:12.040 --> 01:07:13.220
And I was like, hmm.
01:07:13.680 --> 01:07:17.380
And we look at that and it turns out that in some specific case
01:07:17.410 --> 01:07:20.260
that was happening a lot, but nobody thought that this was going to happen,
01:07:20.680 --> 01:07:23.980
one of the algorithms was underperforming slightly
01:07:24.200 --> 01:07:26.880
because it was accidentally quadratic in one way, right?
01:07:27.140 --> 01:07:28.820
Which is like, again, it's very difficult to see.
01:07:28.940 --> 01:07:31.760
If I show you the code, no person on the planet will see it.
01:07:31.920 --> 01:07:33.840
It is so ridiculously complicated to see.
01:07:34.120 --> 01:07:36.220
But we could see it as clear as day on the data.
01:07:36.440 --> 01:07:37.420
It was so obvious,
01:07:37.620 --> 01:07:38.160
so obvious,
01:07:38.320 --> 01:07:39.020
so obvious.
01:07:39.480 --> 01:07:41.300
And bam, we remove it and ba-da-boom,
01:07:41.580 --> 01:07:44.660
we have a core on every single machine of our supercomputer free now.
01:07:45.100 --> 01:07:45.740
That's awesome.
01:07:46.100 --> 01:07:46.720
This is ridiculous.
01:07:47.090 --> 01:07:50.680
You know how this is like a huge amount of money saved.
01:07:51.060 --> 01:07:53.800
So getting this data is the best thing that you can do.
01:07:54.160 --> 01:07:55.460
Listen, everyone listening to this,
01:07:55.680 --> 01:07:56.600
just grab this thing,
01:07:56.940 --> 01:08:00.260
put it to your application and your manager will be happy forever.
01:08:00.660 --> 01:08:03.040
Like it's just the easiest way to make your manager happy.
01:08:03.200 --> 01:08:07.460
Just go measure the shit and fix the thing and don't tell anyone the trick.
01:08:07.760 --> 01:08:08.540
And they're like, damn,
01:08:08.880 --> 01:08:09.440
you just fix it.
01:08:09.520 --> 01:08:10.900
You can do it to other teams.
01:08:11.440 --> 01:08:13.460
You're a performance wizard is what you've become.
01:08:13.640 --> 01:08:14.060
Look at you.
01:08:14.140 --> 01:08:14.420
Exactly, exactly.
01:08:14.780 --> 01:08:16.819
Just attach it to other people's programs.
01:08:17.380 --> 01:08:18.520
And then you see what's low.
01:08:18.600 --> 01:08:20.460
And suddenly you have a change in their code.
01:08:20.759 --> 01:08:21.540
And they say, wow,
01:08:21.740 --> 01:08:22.640
how did you do?
01:08:22.779 --> 01:08:23.060
And like,
01:08:23.460 --> 01:08:23.700
damn.
01:08:24.720 --> 01:08:25.100
Incredible.
01:08:25.339 --> 01:08:27.600
Just do it before everyone knows how to do it.
01:08:28.140 --> 01:08:28.960
What the secret is.
01:08:28.980 --> 01:08:30.720
No, like you knew from this podcast.
01:08:31.140 --> 01:08:32.359
You knew it because you're here.
01:08:32.759 --> 01:08:34.779
Don't worry, I'll delete the episode really soon.
01:08:35.060 --> 01:08:35.480
Oh, damn,
01:08:35.720 --> 01:08:35.779
damn.
01:08:36.720 --> 01:08:38.480
That way the secret will be safe.
01:08:38.900 --> 01:08:39.000
No.
01:08:39.180 --> 01:08:39.480
Yes.
01:08:41.060 --> 01:08:42.380
And this is just part of Python.
01:08:42.839 --> 01:08:43.440
It's just free stuff.
01:08:43.660 --> 01:08:44.700
I mean, the stuff that you're doing,
01:08:44.830 --> 01:08:47.020
the multi was obviously extending this,
01:08:47.160 --> 01:08:49.140
but this just comes with Python 3.15,
01:08:49.319 --> 01:08:49.380
right?
01:08:49.680 --> 01:08:51.259
Yes, this comes with Python 3.15.
01:08:51.540 --> 01:08:51.700
Sorry.
01:08:52.020 --> 01:08:54.259
I think it's really important to emphasize
01:08:55.140 --> 01:08:57.160
how, what a superpower this is,
01:08:57.290 --> 01:08:58.980
because I think in my experience,
01:08:59.520 --> 01:09:02.359
even if you have some tooling available
01:09:02.720 --> 01:09:04.580
for this sort of production profiling,
01:09:04.859 --> 01:09:06.680
it's not always the case that you have access
01:09:06.700 --> 01:09:08.700
that in the worst possible time
01:09:08.920 --> 01:09:10.420
when things are blowing up
01:09:10.470 --> 01:09:11.520
and you need something like that.
01:09:12.060 --> 01:09:12.799
It's the opposite.
01:09:13.529 --> 01:09:14.819
You don't have these tools ready
01:09:15.040 --> 01:09:16.759
when you realize that you need it, right?
01:09:17.020 --> 01:09:17.940
And what's amazing about this
01:09:17.970 --> 01:09:19.279
is that all you need
01:09:19.290 --> 01:09:20.440
is to have 3.15 installed
01:09:20.900 --> 01:09:22.240
and running in your application.
01:09:22.650 --> 01:09:24.299
And this opens up all sorts of possibilities.
01:09:24.720 --> 01:09:25.799
Like you can just go there
01:09:26.120 --> 01:09:26.480
and grab,
01:09:26.640 --> 01:09:27.020
you know,
01:09:27.100 --> 01:09:29.060
when things are happening,
01:09:29.299 --> 01:09:29.799
you can go there
01:09:30.540 --> 01:09:31.060
and sort of grab
01:09:31.779 --> 01:09:33.000
some sort of profile out of that.
01:09:33.580 --> 01:09:34.779
But I think it also opens up
01:09:34.779 --> 01:09:35.800
a lot of possibilities
01:09:35.799 --> 01:09:36.839
for the future.
01:09:37.680 --> 01:09:39.520
I mean, you can even implement
01:09:39.839 --> 01:09:41.220
continuous profiling with this thing
01:09:41.270 --> 01:09:42.060
if you wanted to,
01:09:42.440 --> 01:09:44.759
where you do this as a long-term thing
01:09:45.100 --> 01:09:47.500
on a lower sort of sampling frequency.
01:09:47.880 --> 01:09:48.779
But I think it's also important
01:09:48.830 --> 01:09:49.900
to emphasize here that
01:09:50.140 --> 01:09:52.420
Tachyon is not just a production profiler.
01:09:52.740 --> 01:09:54.160
So it has several modes
01:09:54.320 --> 01:09:55.700
where you attach to your process,
01:09:55.980 --> 01:09:56.600
to your live process,
01:09:56.820 --> 01:09:57.620
and that's super powerful
01:09:57.880 --> 01:09:59.260
and that's useful in many cases.
01:09:59.540 --> 01:10:00.820
But you can also load up
01:10:00.980 --> 01:10:01.980
your sort of application.
01:10:02.190 --> 01:10:03.200
It has a mode where
01:10:03.400 --> 01:10:04.800
you can just load up an application
01:10:04.800 --> 01:10:08.200
and then you start profiling as the program starts
01:10:08.640 --> 01:10:10.740
and it finishes at some duration
01:10:11.360 --> 01:10:12.380
and then you look at that data.
01:10:12.660 --> 01:10:14.800
So this is also very useful potentially
01:10:15.040 --> 01:10:16.160
in your development environment.
01:10:16.620 --> 01:10:18.780
Different reporters which are designed
01:10:19.120 --> 01:10:21.320
for development sort of processes,
01:10:21.640 --> 01:10:22.120
for example,
01:10:22.340 --> 01:10:23.080
you're looking for,
01:10:23.420 --> 01:10:27.320
you want to compare two different profiling snapshots.
01:10:27.700 --> 01:10:28.860
You know, like you make code changes
01:10:28.940 --> 01:10:31.960
and you want to compare how well those changes are,
01:10:32.320 --> 01:10:33.640
how much faster, how much slower
01:10:34.220 --> 01:10:35.180
are making your application.
01:10:35.680 --> 01:10:37.460
So I definitely wouldn't sort of
01:10:37.500 --> 01:10:39.520
lock it back in and like, you use this
01:10:39.720 --> 01:10:41.480
when you have performance issues in production.
01:10:42.580 --> 01:10:43.340
You can use it
01:10:43.660 --> 01:10:45.400
absolutely for any sort of
01:10:45.560 --> 01:10:46.340
profiling situation.
01:10:46.820 --> 01:10:46.920
Yeah,
01:10:47.080 --> 01:10:49.680
like a CLI sort of situation like you're talking about, maybe.
01:10:49.860 --> 01:10:50.280
Exactly.
01:10:50.440 --> 01:10:51.360
So you can use it
01:10:52.920 --> 01:10:53.200
actually
01:10:53.380 --> 01:10:54.821
we have the run here on the
01:10:55.580 --> 01:10:57.320
screen. So you can run a script or you can import
01:10:57.320 --> 01:10:59.500
a module with it and then you basically go from there.
01:10:59.800 --> 01:11:01.360
And there you can have, that can be
01:11:01.720 --> 01:11:03.620
your test suite or it can be a sample
01:11:03.920 --> 01:11:07.620
thing or it can be a CLI which you're looking at sort of improving the performance.
01:11:08.110 --> 01:11:08.500
By the way,
01:11:08.760 --> 01:11:11.040
since you were talking about quantum mechanics before.
01:11:11.900 --> 01:11:12.800
We're on the string theory now.
01:11:12.930 --> 01:11:13.340
No, just kidding.
01:11:13.370 --> 01:11:13.600
Go ahead.
01:11:13.760 --> 01:11:13.960
Yeah.
01:11:14.010 --> 01:11:14.240
No, no.
01:11:14.360 --> 01:11:14.620
Well, well,
01:11:14.780 --> 01:11:15.720
well, well, well, well, well, well.
01:11:16.580 --> 01:11:17.540
We're in relativity now.
01:11:17.710 --> 01:11:18.800
Do you know why it's called tachyon?
01:11:19.080 --> 01:11:20.460
I was wondering, but I haven't asked yet.
01:11:20.600 --> 01:11:20.960
No, why?
01:11:21.300 --> 01:11:21.660
I don't know.
01:11:22.340 --> 01:11:27.000
So a tachyon is the name of hypothetical particles that go faster than light.
01:11:27.560 --> 01:11:30.860
So like a photon is a particle that goes at the speed of light because,
01:11:31.140 --> 01:11:31.380
you know, it's
01:11:31.820 --> 01:11:33.340
the particles that made up light.
01:11:33.480 --> 01:11:33.660
So,
01:11:33.920 --> 01:11:34.940
you know, that's a photon.
01:11:35.820 --> 01:11:39.300
And, you know, special relativity doesn't say that this is not possible.
01:11:39.460 --> 01:11:39.860
It just says,
01:11:40.080 --> 01:11:44.820
like, if you're a photon, you cannot go faster than the speed of light or slower than the speed of light because you are light.
01:11:45.100 --> 01:11:45.660
And if you are,
01:11:45.960 --> 01:11:47.500
you know, you have mass,
01:11:47.700 --> 01:11:51.400
like a normal thing, you cannot go faster than the speed of light or at the speed of light.
01:11:51.580 --> 01:11:54.600
So the speed of light is like a limit, but you can also don't achieve it.
01:11:54.920 --> 01:11:59.340
But then special relativity also says, well, you could also be faster than the speed of light.
01:11:59.460 --> 01:12:00.920
And what happens if that is true,
01:12:01.120 --> 01:12:03.700
you cannot go at or lower than the speed of light.
01:12:03.940 --> 01:12:05.500
And of course, we have not seen such a thing.
01:12:05.760 --> 01:12:08.940
So that is very likely not true because among other things,
01:12:09.120 --> 01:12:10.920
tachyons will travel back in time,
01:12:11.520 --> 01:12:12.840
which is like perhaps a bit crazy.
01:12:12.980 --> 01:12:14.360
But in theory,
01:12:14.600 --> 01:12:18.100
you know, we have to talk about those possibilities and to investigate them.
01:12:18.130 --> 01:12:19.100
So we have to name them.
01:12:19.520 --> 01:12:22.440
And then the name for those particles is called tachyons,
01:12:23.280 --> 01:12:25.340
which is tachy, which is fast in Greek.
01:12:25.800 --> 01:12:28.420
So that's where it comes from.
01:12:28.760 --> 01:12:31.160
That's awesome. And it has its own logo as well.
01:12:31.490 --> 01:12:31.740
Oh yeah,
01:12:31.900 --> 01:12:32.960
Mike Jimenez did that.
01:12:33.390 --> 01:12:37.640
So Mike has been the person who has done some of the logos for the releases.
01:12:37.770 --> 01:12:39.740
So she did the 310 and 311.
01:12:40.180 --> 01:12:43.600
She's an ML researcher at World DeepMind.
01:12:44.120 --> 01:12:47.080
And she's also a very good artist.
01:12:47.480 --> 01:12:48.820
And she does a bunch of these logos.
01:12:49.020 --> 01:12:50.460
So she created the Tachyon logo.
01:12:50.680 --> 01:12:51.160
Very cool.
01:12:51.520 --> 01:12:53.520
I do want to say one more thing about the attach.
01:12:54.020 --> 01:12:56.440
Something that has always bothered me about profiling.
01:12:56.540 --> 01:12:59.340
and I've sort of solved it by writing code,
01:12:59.450 --> 01:13:01.220
like disable the profile or it start
01:13:01.460 --> 01:13:04.140
and then enable it for certain things I'm interested in.
01:13:04.480 --> 01:13:07.140
But there's just so much overhead about the startup
01:13:07.740 --> 01:13:09.120
of your app that's just noise,
01:13:09.410 --> 01:13:09.540
right?
01:13:09.660 --> 01:13:11.120
Like all the imports,
01:13:11.300 --> 01:13:11.660
all the other,
01:13:12.040 --> 01:13:13.080
you're like, I don't care about that.
01:13:13.130 --> 01:13:14.920
I just want to know once it's up and running,
01:13:15.420 --> 01:13:16.220
what does this take?
01:13:16.360 --> 01:13:16.840
You know what I mean?
01:13:17.420 --> 01:13:20.320
And for an app that's sort of up and running for a while,
01:13:20.720 --> 01:13:21.840
just attach to it,
01:13:22.030 --> 01:13:23.760
do the couple of things you're going to do
01:13:23.860 --> 01:13:25.440
that invoke that and then detach,
01:13:25.960 --> 01:13:27.560
gets a really focused view, I would imagine.
01:13:27.980 --> 01:13:28.460
Right. Yes,
01:13:28.660 --> 01:13:29.220
absolutely.
01:13:29.660 --> 01:13:31.580
So getting a little bit short on time,
01:13:32.060 --> 01:13:33.820
but one more thing I want to ask you about,
01:13:34.330 --> 01:13:35.540
because there's this profiling mode,
01:13:35.560 --> 01:13:36.600
which I don't fully understand.
01:13:37.140 --> 01:13:39.680
I know you guys can tell me we have wall clock mode,
01:13:39.980 --> 01:13:42.300
CPU mode, we have GIL mode,
01:13:42.310 --> 01:13:43.520
and we have exception mode.
01:13:43.570 --> 01:13:44.340
They all sound useful.
01:13:44.710 --> 01:13:45.420
But what are these?
01:13:45.570 --> 01:13:45.980
What are these?
01:13:46.610 --> 01:13:46.700
Yeah,
01:13:46.880 --> 01:13:47.240
yeah, yeah.
01:13:47.660 --> 01:13:47.840
So,
01:13:48.180 --> 01:13:48.440
you know,
01:13:48.740 --> 01:13:49.280
okay,
01:13:49.840 --> 01:13:50.220
interesting.
01:13:50.460 --> 01:13:52.500
So the idea here is that,
01:13:52.680 --> 01:13:52.840
you know,
01:13:53.280 --> 01:13:55.920
the profiler will look at your application
01:13:55.940 --> 01:13:57.840
time to time, I will say like, okay, what's going on?
01:13:58.060 --> 01:14:00.560
And it will basically take notes of what
01:14:00.600 --> 01:14:01.220
application is doing,
01:14:01.420 --> 01:14:03.040
right? So that's the basics.
01:14:03.520 --> 01:14:06.300
Okay, so these modes are basically filters or the
01:14:06.340 --> 01:14:06.580
samples.
01:14:06.980 --> 01:14:07.720
The reason is the following.
01:14:07.920 --> 01:14:10.580
Imagine that this is very common. Okay, I will tell you an
01:14:10.680 --> 01:14:11.900
actual case that happened here, right?
01:14:12.120 --> 01:14:14.840
So let's say you have two threads.
01:14:15.320 --> 01:14:16.020
Let's say one of those
01:14:16.120 --> 01:14:18.020
threads is waiting for the network.
01:14:18.280 --> 01:14:20.500
So it's blocking on select or listen or whatever,
01:14:20.680 --> 01:14:20.760
right?
01:14:21.360 --> 01:14:23.560
And then the other thread is doing,
01:14:24.000 --> 01:14:24.380
I don't know,
01:14:24.680 --> 01:14:25.860
calculations or whatever it is,
01:14:26.080 --> 01:14:26.140
right?
01:14:26.410 --> 01:14:28.180
And then you profile those threads.
01:14:28.350 --> 01:14:32.040
And then if you can look individually at every thread, but if you look at the whole thing
01:14:32.280 --> 01:14:35.280
together, it will tell you, okay, half of the time you're spending listening on the
01:14:35.360 --> 01:14:38.600
network and half of the time you're calculating pi, right?
01:14:38.980 --> 01:14:39.940
So in this case,
01:14:40.400 --> 01:14:41.280
the network is noise.
01:14:41.620 --> 01:14:44.480
I don't care about that because I cannot improve the network.
01:14:44.610 --> 01:14:45.080
You know what I mean?
01:14:45.120 --> 01:14:46.220
I cannot just make it fast.
01:14:46.320 --> 01:14:47.340
Or let's say I'm sleeping.
01:14:47.470 --> 01:14:48.760
I call sleep, sleep five.
01:14:49.100 --> 01:14:50.980
then the provider will correctly tell you,
01:14:51.320 --> 01:14:53.260
you are waiting five seconds on Sleep 5.
01:14:53.590 --> 01:14:55.380
But like, okay, but that's not very useful.
01:14:55.450 --> 01:14:55.960
You know what I mean?
01:14:56.020 --> 01:14:57.160
Like that, I cannot fix that,
01:14:57.400 --> 01:14:57.460
right?
01:14:57.600 --> 01:14:58.280
As part of the,
01:14:58.480 --> 01:14:59.720
and that can be written in a file
01:15:00.280 --> 01:15:00.600
or like,
01:15:00.760 --> 01:15:01.660
you know, IO in general.
01:15:02.140 --> 01:15:03.860
So, but that is through the steel,
01:15:04.100 --> 01:15:04.160
right?
01:15:04.220 --> 01:15:05.480
Because if you wait five seconds,
01:15:05.650 --> 01:15:06.840
your program will take five seconds.
01:15:06.960 --> 01:15:08.400
So if you want to know why is my program
01:15:08.900 --> 01:15:09.780
waiting five seconds,
01:15:10.180 --> 01:15:11.500
it's because it's sleeping for five seconds.
01:15:11.610 --> 01:15:12.620
So that's the correct answer.
01:15:12.880 --> 01:15:14.620
It's sometimes it's a good answer, sometimes it's not.
01:15:14.940 --> 01:15:17.760
So by default, it will tell you that's called wall clock.
01:15:18.000 --> 01:15:23.260
It's called wall clock because if you look at a clock in the wall, it will tell you that your application took five seconds,
01:15:23.500 --> 01:15:23.620
right?
01:15:23.880 --> 01:15:24.780
Even if it wasn't sleeping.
01:15:25.120 --> 01:15:25.580
But of course,
01:15:25.800 --> 01:15:27.900
sometimes we don't want to look at that.
01:15:27.960 --> 01:15:30.660
We want to know when my application is doing something,
01:15:31.080 --> 01:15:32.360
you know, because here it's just waiting,
01:15:32.620 --> 01:15:32.700
right?
01:15:32.760 --> 01:15:34.080
Waiting for data,
01:15:34.760 --> 01:15:36.020
waiting to read disk.
01:15:36.460 --> 01:15:37.720
Maybe your disk is slow.
01:15:38.060 --> 01:15:39.140
Maybe the network's slow.
01:15:39.300 --> 01:15:40.000
Maybe you're sleeping.
01:15:40.560 --> 01:15:42.960
So you want to know when your application is doing something.
01:15:43.300 --> 01:15:44.540
That is called CPU time.
01:15:44.760 --> 01:15:46.420
So it's not time that you will measure.
01:15:46.600 --> 01:15:49.780
like how much time is actually your application running in the CPU.
01:15:50.150 --> 01:15:51.800
So when you run it in mode CPU,
01:15:52.490 --> 01:15:55.600
you are filtering all those things that your application is waiting.
01:15:55.860 --> 01:16:00.480
So you're filtering all the samples when you're sleeping or waiting for network.
01:16:00.910 --> 01:16:04.180
And then you only get the samples when your application is doing something,
01:16:04.440 --> 01:16:06.000
which are the ones that you can actually improve,
01:16:06.280 --> 01:16:06.340
right?
01:16:06.440 --> 01:16:10.180
Because like now I can look at a better algorithm or I can look at better module
01:16:10.740 --> 01:16:10.960
or,
01:16:11.240 --> 01:16:11.340
you know,
01:16:11.500 --> 01:16:12.240
I'm parsing JSON.
01:16:12.920 --> 01:16:13.100
So,
01:16:13.500 --> 01:16:13.660
you know,
01:16:13.840 --> 01:16:15.900
that both are interesting because for instance,
01:16:16.060 --> 01:16:16.920
let's say you're parsing JSON,
01:16:17.080 --> 01:16:18.340
you need to read the JSON.
01:16:18.660 --> 01:16:20.220
And for that, you need to wait for your drive
01:16:20.330 --> 01:16:21.680
or the network to give you the JSON.
01:16:22.060 --> 01:16:23.480
That might be interesting to know
01:16:23.840 --> 01:16:26.080
that 30% of the whole time is you waiting
01:16:26.280 --> 01:16:28.400
for the JSON to be given to you.
01:16:28.740 --> 01:16:30.120
So that is all useful information.
01:16:30.740 --> 01:16:30.900
Now,
01:16:31.360 --> 01:16:32.960
once you know that, you cannot improve that
01:16:33.040 --> 01:16:33.900
because that is, you know,
01:16:34.300 --> 01:16:35.820
you tear or the file system,
01:16:36.360 --> 01:16:38.980
but you can know that you're parsing JSON and that's slow.
01:16:39.010 --> 01:16:41.620
So you can say, okay, I can use a better JSON parser, right?
01:16:41.640 --> 01:16:45.200
Right, or JSON or RISM rather than built in JSON
01:16:45.220 --> 01:16:45.820
and see how that works.
01:16:45.900 --> 01:16:46.000
Yeah.
01:16:46.480 --> 01:16:46.860
Exactly, exactly.
01:16:47.020 --> 01:16:48.320
So that is wall and CPU.
01:16:48.540 --> 01:16:50.560
And then we have like the most exotic ones,
01:16:50.740 --> 01:16:52.100
which is like GIL mode.
01:16:52.260 --> 01:16:54.500
So GIL mode tells you how much time
01:16:54.660 --> 01:16:56.060
are you spending on the GIL.
01:16:56.480 --> 01:16:57.440
And this is a bit of a different,
01:16:58.100 --> 01:16:59.920
is a bit of the same idea.
01:17:00.140 --> 01:17:02.960
The idea here is that if you have,
01:17:03.140 --> 01:17:03.400
for instance,
01:17:03.620 --> 01:17:05.240
some program that is spending time
01:17:05.480 --> 01:17:06.900
without the gill, like for example,
01:17:07.160 --> 01:17:07.440
NumPy,
01:17:07.600 --> 01:17:07.680
right?
01:17:08.360 --> 01:17:10.620
You cannot optimize NumPy because that's a SQL,
01:17:11.020 --> 01:17:11.120
right?
01:17:11.160 --> 01:17:12.860
Like you cannot even see what's going on there normally.
01:17:13.580 --> 01:17:14.460
Spoilers in a second.
01:17:14.800 --> 01:17:17.920
But like you cannot see inside that because no Python code.
01:17:18.360 --> 01:17:20.620
So what happens is that because you, you,
01:17:20.840 --> 01:17:20.900
you,
01:17:21.220 --> 01:17:22.200
that thing is running without the
01:17:22.320 --> 01:17:23.720
guild, you don't have access to it.
01:17:23.860 --> 01:17:24.360
So by default,
01:17:24.640 --> 01:17:27.420
it will tell you who is calling that code and it will associate
01:17:27.620 --> 01:17:29.520
all the time that you're spending in NumPy to a caller.
01:17:29.820 --> 01:17:29.960
Right?
01:17:30.060 --> 01:17:32.960
So let's say the function calling NumPy is called call NumPy.
01:17:33.280 --> 01:17:35.880
So it will tell you that call NumPy is 10 seconds.
01:17:36.200 --> 01:17:36.280
Right.
01:17:36.520 --> 01:17:39.520
But let's say that for 10 of those, for nine of those seconds,
01:17:40.080 --> 01:17:41.260
the guild is not
01:17:41.360 --> 01:17:41.500
held,
01:17:41.740 --> 01:17:43.800
of course that is on CPU, so it's doing actual work.
01:17:44.080 --> 01:17:47.080
But because the GIL is not held, some other thread could do something,
01:17:47.500 --> 01:17:47.580
right?
01:17:47.860 --> 01:17:52.780
So you may have a situation where you have many threads and then you want to know who
01:17:52.940 --> 01:17:57.560
is holding the GIL to know what is preventing other threads to run.
01:17:57.930 --> 01:18:02.320
Because if you filter all the samples by only the samples that happen with the gill,
01:18:02.540 --> 01:18:02.760
you
01:18:02.790 --> 01:18:04.620
know that that thing, as long as it's running,
01:18:04.940 --> 01:18:05.900
nothing else is running,
01:18:06.340 --> 01:18:06.460
right?
01:18:06.700 --> 01:18:07.680
So that's interesting.
01:18:08.060 --> 01:18:09.320
So that's another way to cut.
01:18:09.540 --> 01:18:10.160
You may say, well,
01:18:10.340 --> 01:18:11.580
that sounds very complicated.
01:18:11.920 --> 01:18:13.620
Well, it's more complicated than CPU or wall.
01:18:13.860 --> 01:18:15.200
is harder to understand.
01:18:15.640 --> 01:18:18.620
But if you are, you know, already very deep into the weeds
01:18:18.860 --> 01:18:20.440
and like you are very advanced,
01:18:20.580 --> 01:18:21.220
and this is a bit of,
01:18:21.920 --> 01:18:23.360
I want to say this very explicitly,
01:18:23.600 --> 01:18:25.360
like something that we try to do with Tachyon
01:18:25.820 --> 01:18:28.440
is that it tries to be useful
01:18:28.660 --> 01:18:31.000
to a very big range of developers.
01:18:31.360 --> 01:18:31.680
Tachyon,
01:18:31.880 --> 01:18:33.460
if you just want a quick answer,
01:18:33.860 --> 01:18:35.120
you just want to know what is going on,
01:18:35.320 --> 01:18:36.040
just tell me what is wrong.
01:18:36.360 --> 01:18:37.260
Like it will give you that.
01:18:37.360 --> 01:18:38.560
You don't need to know flame graphs.
01:18:38.900 --> 01:18:39.740
You don't need to know,
01:18:40.220 --> 01:18:41.060
like to read this thing.
01:18:41.060 --> 01:18:42.380
It will tell you like, bam, bam, bam.
01:18:42.460 --> 01:18:43.840
These are the five functions that are slower.
01:18:44.160 --> 01:18:44.280
Bam.
01:18:44.480 --> 01:18:45.080
And then you go and look,
01:18:45.320 --> 01:18:45.380
right?
01:18:45.660 --> 01:18:50.520
If you are a bit more advanced, you can start asking for frame graphs or heat maps or,
01:18:50.680 --> 01:18:50.760
you
01:18:50.760 --> 01:18:50.780
know,
01:18:50.960 --> 01:18:51.920
read a bit of these things.
01:18:52.340 --> 01:18:53.760
And if you're really an expert,
01:18:54.180 --> 01:18:56.680
there is like extremely complicated things that you can ask
01:18:56.780 --> 01:18:57.200
this profiler.
01:18:57.420 --> 01:18:57.840
Like, for instance,
01:18:58.320 --> 01:19:00.860
the exception mode is the same thing, but it's asking the question,
01:19:01.020 --> 01:19:02.860
how much time I spend handling exceptions?
01:19:03.400 --> 01:19:03.980
So like errors,
01:19:04.160 --> 01:19:05.840
how much error handling is costing me?
01:19:06.080 --> 01:19:07.120
Of course, you will say, well,
01:19:07.280 --> 01:19:08.380
that probably is nothing.
01:19:09.000 --> 01:19:09.100
But,
01:19:09.280 --> 01:19:09.440
you know,
01:19:09.620 --> 01:19:10.000
if you are already like...
01:19:10.000 --> 01:19:10.840
It can be a lot.
01:19:11.120 --> 01:19:11.240
Yeah,
01:19:11.420 --> 01:19:11.680
exactly.
01:19:11.740 --> 01:19:13.180
You don't know if you are fine tuning.
01:19:13.840 --> 01:19:16.540
And also exceptions are used for much more than exceptions.
01:19:16.610 --> 01:19:16.980
For instance,
01:19:17.200 --> 01:19:18.140
as you very well know,
01:19:18.580 --> 01:19:20.220
the iterator protocol raises,
01:19:20.440 --> 01:19:21.680
you know, a stop iteration
01:19:22.240 --> 01:19:23.320
as exception control.
01:19:23.450 --> 01:19:24.060
So that can be,
01:19:24.300 --> 01:19:24.440
you know.
01:19:24.800 --> 01:19:28.420
And then there is also opcode mode where you can look at individual opcodes.
01:19:28.650 --> 01:19:31.960
So there is very complicated questions that you can ask your program.
01:19:32.300 --> 01:19:32.940
And in particular,
01:19:33.460 --> 01:19:39.520
CPU wall exception and GIL are filters over the samples such that you
01:19:39.480 --> 01:19:43.460
can like know what is your application exactly doing and what is not doing.
01:19:43.620 --> 01:19:43.740
Yeah.
01:19:43.900 --> 01:19:44.340
Very interesting.
01:19:44.660 --> 01:19:48.760
I mean, it's super simple example of exceptions costing is I'm parsing a CRV,
01:19:48.980 --> 01:19:49.340
a CSV,
01:19:49.540 --> 01:19:49.820
and it's
01:19:49.920 --> 01:19:50.760
coming in as text.
01:19:51.040 --> 01:19:51.840
And I think it's a number.
01:19:51.920 --> 01:19:54.360
So we do try this thing.
01:19:54.860 --> 01:19:55.160
Excellent.
01:19:55.520 --> 01:19:56.300
Well, I guess it's,
01:19:56.460 --> 01:19:57.480
you know, set it to none, right?
01:19:58.000 --> 01:20:02.320
You could regex check that or some, you could do something else instead of an exception in
01:20:02.380 --> 01:20:02.880
a tight loop.
01:20:03.080 --> 01:20:03.380
Exactly.
01:20:03.380 --> 01:20:04.380
You get a big bonus.
01:20:04.940 --> 01:20:05.340
Absolutely.
01:20:05.520 --> 01:20:05.660
Absolutely.
01:20:06.000 --> 01:20:06.920
That's exactly it.
01:20:06.940 --> 01:20:07.040
Right.
01:20:07.480 --> 01:20:13.420
And there's some modes when you don't filter by it, like there's a secret mode that only runs when you use one of the outputs,
01:20:13.800 --> 01:20:14.280
which is Gecko.
01:20:15.000 --> 01:20:18.420
So we don't advertise the mode because it only makes sense to this mode,
01:20:18.780 --> 01:20:19.680
to this format,
01:20:19.840 --> 01:20:20.460
which is Gecko.
01:20:20.960 --> 01:20:22.040
But it's called All.
01:20:22.340 --> 01:20:25.160
And what All does is that it records all the samples,
01:20:25.460 --> 01:20:28.800
but it kind of like colors the sample with what was going on there.
01:20:29.160 --> 01:20:31.260
So if the sample is on CPU, it says CPU.
01:20:31.480 --> 01:20:31.940
If it's not.
01:20:32.240 --> 01:20:37.200
So in that way, you can see kind of like a bunch of lanes of what every thread was doing.
01:20:37.600 --> 01:20:41.520
So for example, if you have two threads and the threads are both calculating CPU,
01:20:41.830 --> 01:20:42.880
they will fight for the guild.
01:20:43.170 --> 01:20:45.060
So they will kind of move the guild back and forward.
01:20:45.470 --> 01:20:49.320
And then you can see in the timeline what thread has the guild and for how much.
01:20:49.680 --> 01:20:50.420
So that you can see, oh,
01:20:50.660 --> 01:20:50.840
you know,
01:20:51.080 --> 01:20:52.840
this thread ran for like 20 milliseconds.
01:20:53.060 --> 01:20:54.800
And then this other thread ran for like 10 milliseconds.
01:20:54.990 --> 01:20:56.760
And you can see how it's been bounced.
01:20:57.100 --> 01:20:59.720
So you can like calculate maximum throughput and things like that.
01:21:01.140 --> 01:21:03.420
Yeah, that contention is a big challenge of multi-threading.
01:21:04.080 --> 01:21:04.160
Exactly.
01:21:04.660 --> 01:21:06.640
You said that stuff doesn't scale linearly,
01:21:06.970 --> 01:21:08.640
like 10 threads don't give you 10x performance,
01:21:08.790 --> 01:21:10.160
and that's a good reason right there.
01:21:10.470 --> 01:21:10.560
Exactly,
01:21:10.780 --> 01:21:13.260
and this can give you a more or less good answer for that,
01:21:13.390 --> 01:21:14.000
perhaps not perfect,
01:21:14.200 --> 01:21:14.740
but...
01:21:14.890 --> 01:21:15.960
You guys, there is so much here.
01:21:16.290 --> 01:21:17.200
There is so much here.
01:21:17.760 --> 01:21:19.440
But we're also so much out of time, I think.
01:21:19.880 --> 01:21:21.120
I mean, people should check this out.
01:21:21.260 --> 01:21:23.120
There's really a lot that you can...
01:21:23.140 --> 01:21:24.600
I'm super impressed with this,
01:21:24.760 --> 01:21:25.760
Pablo and Laszlo.
01:21:25.820 --> 01:21:26.200
Well, thank you.
01:21:26.360 --> 01:21:26.860
Thank you very much.
01:21:27.520 --> 01:21:28.040
Thank you.
01:21:28.160 --> 01:21:31.980
There was a really good presentation on PyCon this year,
01:21:32.160 --> 01:21:33.680
which has a little bit more info.
01:21:34.300 --> 01:21:36.840
I also remember because you mentioned how much there is here.
01:21:36.850 --> 01:21:43.380
And I remember working on the slide and I was just panicking how we're going to get all this information into this half an hour talk.
01:21:43.540 --> 01:21:43.920
What's our talk?
01:21:44.900 --> 01:21:45.220
30 minutes?
01:21:45.370 --> 01:21:45.860
Oh, uh-oh.
01:21:46.740 --> 01:21:47.000
No, no,
01:21:47.160 --> 01:21:47.980
it's impossible.
01:21:49.700 --> 01:21:50.140
Yeah,
01:21:50.930 --> 01:21:51.440
that presentation.
01:21:51.630 --> 01:21:54.880
But I also recommend that for anyone who's interested.
01:21:55.010 --> 01:21:57.380
I think that has a pretty good overview.
01:21:57.900 --> 01:21:58.060
Yeah,
01:21:58.320 --> 01:22:00.120
I'll definitely link to your talk on YouTube.
01:22:00.680 --> 01:22:03.440
There is only one thing I want to add before we go,
01:22:03.780 --> 01:22:07.420
which I think is interesting and it's also a bit, you know, selfish,
01:22:07.920 --> 01:22:09.940
which is that, so Tachyon is really good.
01:22:10.640 --> 01:22:13.960
The only interesting challenge of Tachyon is that,
01:22:14.640 --> 01:22:17.780
you know, if your application spends a bunch of time in C code,
01:22:18.120 --> 01:22:21.580
like NumPy or, you know, Pandas or Polars or like nowadays,
01:22:21.880 --> 01:22:25.060
like all the machine learning stuff like CUDA or whatever it is.
01:22:25.580 --> 01:22:25.880
Unfortunately,
01:22:26.280 --> 01:22:26.360
again,
01:22:26.520 --> 01:22:28.560
like Tachyon is a Python profiler.
01:22:28.610 --> 01:22:29.820
So it only sees Python, right?
01:22:30.160 --> 01:22:32.540
It is who calls that C code, but not the C code.
01:22:32.820 --> 01:22:38.820
And of course, you can say as a user, you are not super interested in what is happening on the guts of NumPy.
01:22:39.040 --> 01:22:40.180
But sometimes you are, right?
01:22:40.280 --> 01:22:45.160
Because what is going on or why my application is low, sometimes you need to just go there.
01:22:45.320 --> 01:22:47.640
That's the unfortunate life we live in.
01:22:47.840 --> 01:22:51.280
And Tachyon cannot analyze C code because it's a Python profiler.
01:22:51.340 --> 01:22:52.160
It's not a C profiler.
01:22:52.580 --> 01:22:55.420
On the other side, a C profiler cannot analyze Python.
01:22:55.680 --> 01:22:57.980
So you can only see C code.
01:22:58.180 --> 01:23:00.000
So you will see the internals of CPython,
01:23:00.480 --> 01:23:01.780
but it will not have Python names.
01:23:01.820 --> 01:23:03.500
It will not tell you full bar batch.
01:23:03.540 --> 01:23:05.200
It will tell you by valuable frame default,
01:23:05.560 --> 01:23:06.280
and that's not useful.
01:23:06.700 --> 01:23:08.540
So that is one big challenge.
01:23:08.700 --> 01:23:12.080
And the other is that the moment you want to go into the world of C profilers,
01:23:12.520 --> 01:23:13.520
complexity explodes.
01:23:14.620 --> 01:23:17.160
It's just like, if this is hard, which is hard,
01:23:17.600 --> 01:23:18.700
when you want to do C profiles,
01:23:19.080 --> 01:23:20.680
it's like 10 times harder.
01:23:20.800 --> 01:23:22.860
And then when you want to do both at the same time,
01:23:23.040 --> 01:23:24.380
it's a million times harder.
01:23:24.760 --> 01:23:25.720
So unfortunately,
01:23:25.900 --> 01:23:28.680
we didn't want to add that complexity to the standard library because,
01:23:29.220 --> 01:23:32.020
you know, it's a lot of complexity for core developers to maintain.
01:23:32.320 --> 01:23:33.660
And it's also really,
01:23:33.940 --> 01:23:34.120
really,
01:23:34.420 --> 01:23:35.080
really hard.
01:23:35.400 --> 01:23:38.620
It's extremely hard and it requires a lot of care and,
01:23:38.780 --> 01:23:39.740
you know, bugs will happen.
01:23:40.020 --> 01:23:41.240
So we didn't want to like put.
01:23:41.480 --> 01:23:45.520
So one of the things I've been working on here at Hudson River Trading is a version of
01:23:45.640 --> 01:23:47.300
Tachyon is like the big brother of Tachyon,
01:23:47.480 --> 01:23:48.320
which is called Cronon,
01:23:48.840 --> 01:23:49.580
which is also physics
01:23:49.600 --> 01:23:55.500
related because like the in quantum mechanics if you quantize time the particle of time is called
01:23:55.500 --> 01:23:59.900
a chronon so you know because it's time because it's processed so we call it chron and this is a
01:24:00.000 --> 01:24:06.800
profiler we have been using for nine months already here for with extreme success extreme success like
01:24:06.800 --> 01:24:11.220
the whole company is using it all the time and this profiler does everything that tackian does
01:24:11.470 --> 01:24:18.480
but it's also multiple versions of python so you can profile 312 313 314 315 all that you want and
01:24:18.500 --> 01:24:20.500
also profile C code and kernel code.
01:24:20.680 --> 01:24:22.340
So you can see literally everything.
01:24:22.620 --> 01:24:23.300
Like you can see something
01:24:23.520 --> 01:24:23.780
like,
01:24:24.180 --> 01:24:25.720
oh, your application is calling,
01:24:26.020 --> 01:24:28.980
I don't know, calculating Python and that goes into NumPy
01:24:29.180 --> 01:24:30.440
and that's creating a bunch of arrays.
01:24:30.860 --> 01:24:33.720
But one of those arrays just trigger a page fall in the kernel
01:24:33.960 --> 01:24:37.880
and then the kernel is like trying to find memory for you. So you can see how much that takes.
01:24:38.820 --> 01:24:39.700
And it supports the streaming.
01:24:39.920 --> 01:24:43.480
So you can have like Grafana dashboard with all the profile and
01:24:43.500 --> 01:24:46.580
The Flame Graph is changing as your application is running,
01:24:46.780 --> 01:24:48.780
so you don't need to even wait for it.
01:24:49.050 --> 01:24:50.320
You don't need to attach even.
01:24:50.610 --> 01:24:53.760
You can just keep it 24-7 and see the data as it comes in.
01:24:54.820 --> 01:24:56.260
It supports all sorts of formats,
01:24:56.560 --> 01:24:57.420
all sorts of tools.
01:24:58.300 --> 01:25:00.500
It works on macOS as well and Linux.
01:25:02.050 --> 01:25:05.680
So we are going to make it open source soon in a month or two.
01:25:06.020 --> 01:25:08.820
Yes, and that's going to be a big deal, big deal.
01:25:09.120 --> 01:25:09.380
Cool.
01:25:09.480 --> 01:25:09.980
Secret sauce.
01:25:10.720 --> 01:25:11.740
Yeah, that sounds amazing.
01:25:12.100 --> 01:25:12.260
I mean,
01:25:12.500 --> 01:25:12.620
already,
01:25:12.820 --> 01:25:13.940
Takion is very amazing.
01:25:14.480 --> 01:25:16.840
Alas, it will be known as profiling dot sampling.
01:25:17.480 --> 01:25:18.280
Yes, yes, yes.
01:25:18.600 --> 01:25:19.280
We now know.
01:25:19.280 --> 01:25:20.100
Most people call it Takion.
01:25:20.240 --> 01:25:21.340
Most people call it Takion.
01:25:21.680 --> 01:25:22.340
Call it Takion.
01:25:22.480 --> 01:25:23.300
That's the real name.
01:25:23.440 --> 01:25:28.000
The other one is the, you know, the name that you need to use when you go to boring meetings
01:25:28.580 --> 01:25:28.860
with, like,
01:25:29.140 --> 01:25:30.920
suits and, like, you know, companies.
01:25:31.100 --> 01:25:31.440
Companies, oh,
01:25:31.680 --> 01:25:32.360
who are you Takion?
01:25:32.440 --> 01:25:33.900
No, no, I'm profiling dot sampling.
01:25:34.030 --> 01:25:34.100
Oh,
01:25:34.360 --> 01:25:34.600
okay, okay.
01:25:35.080 --> 01:25:35.480
But, like, no,
01:25:35.660 --> 01:25:35.780
no.
01:25:36.120 --> 01:25:36.760
When you go to the party.
01:25:36.870 --> 01:25:37.500
We trust this one,
01:25:37.670 --> 01:25:37.760
yeah.
01:25:37.980 --> 01:25:39.880
It's not one of the more weird physics things.
01:25:40.400 --> 01:25:44.000
Everyone should just call it Tachyon until it spreads and it just becomes the name.
01:25:44.300 --> 01:25:44.360
Yeah,
01:25:44.540 --> 01:25:44.580
yeah.
01:25:44.820 --> 01:25:47.380
No one will understand what you're talking about profiling or something.
01:25:47.620 --> 01:25:50.440
If you talk to your manager, you say, oh, I use profiler or something.
01:25:50.780 --> 01:25:51.040
Oh,
01:25:51.200 --> 01:25:51.660
okay.
01:25:51.940 --> 01:25:52.380
But like, no,
01:25:52.620 --> 01:25:56.900
with your friends, you say, I use Tachyon and I fucking did it.
01:25:57.020 --> 01:25:57.200
Oh,
01:25:57.400 --> 01:25:57.480
okay.
01:25:57.720 --> 01:25:58.180
That's right.
01:25:58.460 --> 01:25:59.340
Who is this Tachyon?
01:25:59.480 --> 01:26:00.240
Why does he solve it?
01:26:00.260 --> 01:26:01.000
Yeah, who is this guy?
01:26:01.140 --> 01:26:01.580
Who is this guy?
01:26:02.000 --> 01:26:02.080
Well,
01:26:02.400 --> 01:26:02.960
thank you guys,
01:26:03.140 --> 01:26:03.460
Pablo,
01:26:03.900 --> 01:26:04.200
Laszlo.
01:26:04.380 --> 01:26:04.920
Thanks for being on the show.
01:26:05.460 --> 01:26:07.340
Awesome work on getting this out to the world.
01:26:07.480 --> 01:26:09.160
This is going to be super cool for everyone.
01:26:09.420 --> 01:26:09.820
It's going to be great.
01:26:10.000 --> 01:26:11.280
Thank you so much for hiring us,
01:26:11.440 --> 01:26:13.780
and I hope people enjoy it.
01:26:13.920 --> 01:26:14.540
I'm sure they will.
01:26:14.940 --> 01:26:15.160
All right.
01:26:15.540 --> 01:26:15.820
Thanks a lot.
01:26:16.180 --> 01:26:16.440
Bye, guys.
01:26:16.440 --> 01:26:16.720
See you.
01:26:16.980 --> 01:26:17.300
Thank you.
01:26:18.220 --> 01:26:20.640
This has been another episode of Talk Python To Me.
01:26:21.020 --> 01:26:21.740
Thank you to our sponsors.
01:26:21.980 --> 01:26:23.240
Be sure to check out what they're offering.
01:26:23.440 --> 01:26:24.800
It really helps support the show.
01:26:25.600 --> 01:26:27.140
This episode is brought to you by Sentry.
01:26:27.460 --> 01:26:31.500
Sentry's new MCP server connects your error reports straight to your coding agent.
01:26:32.040 --> 01:26:33.060
Claude Code reads the errors,
01:26:33.500 --> 01:26:34.100
finds the bugs,
01:26:34.360 --> 01:26:34.960
and fixes them.
01:26:35.480 --> 01:26:38.480
Get started at talkpython.fm/sentry.
01:26:39.120 --> 01:26:41.120
If you or your team needs to learn Python,
01:26:41.310 --> 01:26:49.320
we have over 270 hours of beginner and advanced courses on topics ranging from complete beginners to async code, Flask, Django,
01:26:49.720 --> 01:26:49.880
HTML,
01:26:50.360 --> 01:26:51.400
and even LLMs.
01:26:51.720 --> 01:26:52.100
Best of all,
01:26:52.340 --> 01:26:54.000
there's no subscription in sight.
01:26:54.560 --> 01:26:56.240
Browse the catalog at talkpython.fm.
01:26:56.920 --> 01:27:00.300
And if you're not already subscribed to the show on your favorite podcast player,
01:27:00.500 --> 01:27:01.580
what are you waiting for?
01:27:02.220 --> 01:27:04.020
Just search for Python in your podcast player.
01:27:04.110 --> 01:27:05.000
We should be right at the top.
01:27:05.400 --> 01:27:06.840
If you enjoy that geeky rap song,
01:27:07.030 --> 01:27:08.260
you can download the full track.
01:27:08.500 --> 01:27:10.320
The link is actually in your podcast below or share notes.
01:27:11.220 --> 01:27:11.780
This is your host,
01:27:12.000 --> 01:27:12.460
Michael Kennedy.
01:27:12.790 --> 01:27:13.920
Thank you so much for listening.
01:27:14.150 --> 01:27:14.940
I really appreciate it.
01:27:15.360 --> 01:27:16.060
I'll see you next time.
01:27:34.160 --> 01:27:35.220
Get old.
01:27:36.740 --> 01:27:38.720
We tapped into that modern vibe.
01:27:39.160 --> 01:27:40.120
Overcame each storm.
01:27:40.940 --> 01:27:42.220
Talk Python To Me.
01:27:42.310 --> 01:27:43.600
I sync is the norm.