Travis Oliphant

WEBVTT
NOTE
Transcription provided by Podhome.fm
Created: 7/6/2024 4:45:45 PM
Duration: 3136.713
Channels: 1
1
00:00:00.080 --> 00:00:02.100
Did you guys get started with this? What's the
2
00:00:03.199 --> 00:00:09.745
what's driving you? We both listen to a lot of podcasts, and, you know, I've been using Python for a while. Chris has been interested
3
00:00:10.945 --> 00:00:14.165
He's recently started using it, and there just haven't been any
4
00:00:14.625 --> 00:00:23.430
Python podcasts out in the wild for a while now because all the ones that had been around stopped producing any new content. So we decided, hey, why not us?
5
00:00:23.890 --> 00:00:24.390
Cool.
6
00:00:24.770 --> 00:00:30.304
Yeah. It was it was definitely 1 of those things where I feel like Python has such a thriving community.
7
00:00:30.845 --> 00:00:41.160
And, you know, I I listen to other podcasts for other stacks like the Ruby Rogues and things like that. And I just thought to myself, you know, it is it is a a crying shame that
8
00:00:41.460 --> 00:00:52.445
Python doesn't have something in this space to sort of, like, to to do outreach to people. Because there are a lot of people who while they're working out or driving or whatever the case may be, they listen to podcast to keep informed.
9
00:01:07.015 --> 00:01:14.450
Hello, and welcome to podcasts dot in it. Thank you for joining us. Today, we are recording on April 28, 2015.
10
00:01:15.230 --> 00:01:21.155
Your hosts, as usual, are Tobias Macy and Chris Patti. And tonight we're interviewing Travis Oliphant.
11
00:01:22.975 --> 00:01:24.835
Travis, why don't you introduce yourself?
12
00:01:25.695 --> 00:01:26.195
Oh,
13
00:01:26.570 --> 00:01:29.790
thanks. I appreciate being here, Tobias and Chris. It's a pleasure.
14
00:01:30.250 --> 00:01:34.350
My name is Travis Oliphant. I've been, working with Python since 1998.
15
00:01:36.165 --> 00:01:36.665
98
16
00:01:37.365 --> 00:01:42.825
about. Oh, for a long time. I have been a I was a scientist really or someone passionate about engineering and science and
17
00:01:43.620 --> 00:01:51.960
came across Python as a graduate student. Really loved it. Ended up, getting pulled in to and getting addicted to the open source community and the
18
00:01:52.485 --> 00:01:57.705
ability to change the world through, interactions with the community and have been a part of the scientific ecosystem
19
00:01:58.405 --> 00:01:59.145
ever since.
20
00:02:00.005 --> 00:02:03.590
So I'm curious what is priority to create NumPy and SciPy
21
00:02:04.130 --> 00:02:07.270
and why you chose Python for creating those libraries?
22
00:02:07.970 --> 00:02:13.655
Yeah. So, so SciPy was my real passion when I was a graduate student. I was studying electrical
23
00:02:15.635 --> 00:02:17.495
engineering and then biomedical imaging
24
00:02:18.115 --> 00:02:30.645
at the Mayo Clinic. And I had 5 dimensional derivatives I need to take, and I knew wanted to do it at a high level and not have to write c code all the time. I could write c. I could write, you know, Pascal, but I really like this expressivity of Python.
25
00:02:31.105 --> 00:02:31.765
I had an
26
00:02:32.145 --> 00:02:40.590
experience basically at the I started to use it, and then a year later, I came back to code I'd written previously and I could still read it, which is the opposite experience I'd had with Pearl 3 years earlier
27
00:02:40.890 --> 00:02:50.655
when I'd written some Pearl code to do some high level manipulation of scatter scatterometer data coming off the tape. And then I went back and tried to look at the code that I had written. I didn't understand it, my own code.
28
00:02:51.194 --> 00:02:51.694
So
29
00:02:52.075 --> 00:02:57.349
that that kind of for me, I remember that moment when a year later, I looked about the same code. I said, I get this still.
30
00:02:58.450 --> 00:03:04.245
And I went, and and I have this really unusual feeling of of kind of loving to program, like, and just having fun with it.
31
00:03:04.785 --> 00:03:09.910
I don't know because of maybe it gave me it felt powerful. It felt like I could do things quickly and I could connect.
32
00:03:10.790 --> 00:03:14.490
People have expressed this in many ways. Like, it fit in your head. It didn't get in your way.
33
00:03:15.350 --> 00:03:18.890
It lets you it left room for you to think about your problem rather than the programming.
34
00:03:19.235 --> 00:03:22.694
I don't know exactly all the reasons, but for me, I was hooked. And
35
00:03:23.075 --> 00:03:30.760
and I and I got excited as a student to have to to be able to create more. So I I looked around and said, I wanna be able to do
36
00:03:31.300 --> 00:03:40.194
order numbers equation solving. I wanted to simulate the MRI an MRI machine, and I needed the the the fundamentals of MRI is the block equations, which simulate the magnetization
37
00:03:40.495 --> 00:03:42.114
vector. It's a fairly straightforward,
38
00:03:42.415 --> 00:03:45.620
simulation, but I needed an or an ordinary differential equation
39
00:03:46.420 --> 00:03:49.080
solver. Wasn't didn't exist in Python.
40
00:03:49.620 --> 00:03:56.495
So I went out and found 1 on the Internet. It was written in Fortran, and I connected it to the Python interpreter by writing a c extension
41
00:03:57.275 --> 00:04:03.040
by hand, basically. I wrote a c extension to Python by hand to connect that Fortran code so I can call it from high level
42
00:04:03.599 --> 00:04:04.500
and specify
43
00:04:05.599 --> 00:04:12.985
the equations in Python but have this Fortran solver ultimately do it. That was sort of the start of SciPy. I ended up connecting integration
44
00:04:13.364 --> 00:04:14.504
solver and,
45
00:04:14.885 --> 00:04:18.025
some optimization libraries. I released them as Minpak in 1999,
46
00:04:19.044 --> 00:04:19.544
roughly.
47
00:04:21.290 --> 00:04:26.910
So I was just excited about the possibility to create this this this cool library that lets you do high level coding and programming
48
00:04:27.675 --> 00:04:34.095
but in a language that was open. And you could then share and other people would use it. So that was a start. I gotta get excited with SciPy.
49
00:04:34.715 --> 00:04:39.910
Then as I as I graduated and I got a job as an assistant professor at Ringling University,
50
00:04:40.449 --> 00:04:46.925
there was some energy around the basically, in the community, there was a a a new array object being promoted called numarray.
51
00:04:47.305 --> 00:04:51.085
And in the when I had joined the Python ecosystem Numeric written by Jim Huguenen
52
00:04:51.705 --> 00:04:58.660
was, the array object that was being that I used and built sci fi around. So number a was this other array object that was kind of emerging,
53
00:04:59.199 --> 00:05:08.730
promoted by the Space Science Telescope Institute. It had some features, improved features they needed. And then the then 1 day, I saw, basically a
54
00:05:09.690 --> 00:05:17.790
a library came out. A I'd always wanted. It was a morphology library. And I was a medical imaging student and I really wanted a morphology library in sci fi,
55
00:05:18.345 --> 00:05:20.205
but it came out for Numarray.
56
00:05:20.665 --> 00:05:25.645
And that at that point, I went, oh, we're building these 2 communities of different libraries on different array objects.
57
00:05:26.400 --> 00:05:28.340
And I I became frustrated.
58
00:05:28.960 --> 00:05:30.500
I was like, oh, this is not good.
59
00:05:31.039 --> 00:05:35.865
And so, that happened to coincide a few a few months or 2 later with a class
60
00:05:36.245 --> 00:05:45.010
that I was teaching canceled, and so I was left without a class. I had 4 months of no class and just my research to work on. And so couple that with this
61
00:05:45.470 --> 00:05:53.845
itch and this passion to merge these communities and the feeling that I've been I've I've been the only 1 kind of around long enough. I felt that would be able to do something about this. So
62
00:05:54.224 --> 00:05:57.044
I kinda felt an obligation, a duty, and then a desire
63
00:05:57.585 --> 00:05:58.565
to work on it.
64
00:05:59.585 --> 00:06:00.965
That's that's really great.
65
00:06:01.460 --> 00:06:04.360
You know, it's it's interesting. I worked for the Human Genome Project
66
00:06:04.820 --> 00:06:07.080
quite a number of years back at this point. And
67
00:06:07.460 --> 00:06:18.835
I I at that point in time, everyone there was sort of graduating on from this kinda crusty old pearl, and everyone was really excited about the potential for for using Python
68
00:06:19.350 --> 00:06:20.090
in that
69
00:06:20.470 --> 00:06:21.610
sphere. So
70
00:06:21.990 --> 00:06:29.715
based on the timeline that you're mentioning, you know, it seems like there were a whole bunch of things sort of coalescing and coming together to really sort of
71
00:06:30.095 --> 00:06:32.355
provide for kind of a Python
72
00:06:32.735 --> 00:06:39.100
explosion in the sciences that happened all around the same time. That's right. It's exactly right. There was. This is about 2006
73
00:06:39.880 --> 00:06:44.940
that I, wrote NumPy, which built out of numeric and pulled in the features of of numarray.
74
00:06:45.514 --> 00:06:47.854
So to me, that was a very important critical
75
00:06:48.555 --> 00:06:57.030
moment because it was a it's a challenging thing to do. I I I'm not a I was not trained as a developer, as a computer scientist, as a scientist. Cared about what people did with
76
00:06:57.570 --> 00:07:05.065
the code. I learned enough c and written and seen enough of the Python c if you gotta be dangerous. Became a core Python developer at the time to actually move and and
77
00:07:05.605 --> 00:07:07.465
and try to promote some of the Python,
78
00:07:07.845 --> 00:07:12.710
or the NumPy array structure into Python itself to avoid the any future problem of Eridium compatibility.
79
00:07:13.330 --> 00:07:14.070
So a lot
80
00:07:14.370 --> 00:07:18.195
of work I did around that time was a real learning experience for me, to kind of understand,
81
00:07:18.574 --> 00:07:31.099
kind of what, software engineering and and architecture is like. I made mistakes, but but I know what I know now at the time I would've done things a bit differently. But, you know, you that's the great thing about the Python community is people, you know, they they they dive in where they
82
00:07:31.560 --> 00:07:38.545
can and they and they support what they can do and then move from there and iterate. So, yes, about that time so around that time also, you know, John Hunter had created Matplotlib.
83
00:07:39.805 --> 00:07:44.870
In 2001, I we put together scipy out of the early work I've done in 99.
84
00:07:45.410 --> 00:07:47.110
I kind of organized a library of
85
00:07:47.569 --> 00:07:52.205
tools. So you started to get this critical mass, and then IPython interactive,
86
00:07:52.505 --> 00:07:54.445
environment was also emerging at the time.
87
00:07:54.985 --> 00:08:00.530
And after 2006, kinda creating NumPy really helped kinda solidify this is the array object, everyone agrees.
88
00:08:01.190 --> 00:08:03.410
I did a lot of work to try to, you know,
89
00:08:03.790 --> 00:08:11.455
just organize community and and and really by and the work was a lot of writing code, a lot of writing examples, a lot of writing documentation, a lot of writing
90
00:08:11.995 --> 00:08:19.560
emails to try to encourage everybody to use it. And I remember when John Hunter removed his support for number,
91
00:08:20.020 --> 00:08:24.185
right, in numeric. It was just NumPy support in map. And that was about 2007,
92
00:08:24.485 --> 00:08:29.765
and that was a big deal. That and then sort of everybody followed suit, and then there's the explosion occurred of
93
00:08:30.410 --> 00:08:33.230
a lot of things came together as you said. It was a and definitely
94
00:08:33.690 --> 00:08:40.964
1 of the great things about the Python ecosystem is how many people have participated in making it great. And the scientific computing system is no different.
95
00:08:41.345 --> 00:08:42.964
Although it does take individuals
96
00:08:43.425 --> 00:08:44.324
taking risks,
97
00:08:44.865 --> 00:08:47.365
it was not good for my academic career to spend time on NumPy.
98
00:08:47.750 --> 00:08:50.070
Nobody none of my people folks who would,
99
00:08:50.630 --> 00:08:52.810
who would eventually vote on my tenure application
100
00:08:53.750 --> 00:08:54.730
cared about NumPy.
101
00:08:55.670 --> 00:08:59.225
But I felt like it was the right thing to do and the and the important thing to do for the world.
102
00:08:59.845 --> 00:09:00.345
So
103
00:09:01.045 --> 00:09:02.665
We're certainly glad you did.
104
00:09:03.204 --> 00:09:07.399
We're definitely all happy that you did that because it has certainly become the basis of a
105
00:09:07.700 --> 00:09:14.935
large amount of scientific and numerical code in Python, and even people who aren't necessarily doing any hard science
106
00:09:15.715 --> 00:09:19.895
or incredibly complicated maths have definitely found some uses for it.
107
00:09:20.595 --> 00:09:25.210
Yeah. And what's in you you look at and then, you know, Penn is a similar story. Right? You've talked to Wes. You'll hear,
108
00:09:25.590 --> 00:09:26.250
you know,
109
00:09:26.790 --> 00:09:37.825
a data frame story and a lot of other folks are using data frames, but he wrote 1 and got his his good employer to let him open source what he had worked on. And now it's become the standard for people doing data analysis.
110
00:09:38.605 --> 00:09:42.209
So, you know, the store and, you know, John Hunter before him with with map.lib
111
00:09:42.670 --> 00:09:45.250
and and Fernando Perez and Brian Granger on IPython.
112
00:09:45.949 --> 00:09:50.675
You just see, you know, the scikit learn team is another 1 that just exploded since 2009.
113
00:09:51.535 --> 00:09:54.355
Like, it's just fantastic to see all this,
114
00:09:54.735 --> 00:10:02.019
but it does take effort. I guess that's the thing I wanna definitely emphasize. Somebody has to have the courage to do and to act even in the face of uncertainty.
115
00:10:02.959 --> 00:10:21.360
And but once you do that, then and not everything you do also succeeds. Right? Sometimes you write something that's it's the wrong direction, but if you're listening open to feedback, you can usually get it. NumPy, everybody wanted it. Everybody was excited about it. They were unsure it could be done, but once I started making progress and people could see we were I was gonna make it, then I got a lot of support.
116
00:10:22.780 --> 00:10:23.520
I'll bet.
117
00:10:24.145 --> 00:10:33.350
So for those who want in the know, can you provide a brief description of what data science is and how you got involved in it? Data science, man. That's a term that was coined by DJ Patil.
118
00:10:34.450 --> 00:10:37.910
And, so it's being used as a as a coalescing point,
119
00:10:38.450 --> 00:10:51.280
for a general concept of anybody that takes data and tries to get it from and tries to get insight from it, it kind of becomes this most people who are data scientists do a little bit of applied mathematics, a little bit of system administration,
120
00:10:51.740 --> 00:10:53.520
a little bit of coding, programming.
121
00:10:54.380 --> 00:10:58.080
They kinda put it all together and have to kinda little do a little bit of all 3.
122
00:10:58.485 --> 00:11:04.825
So, you know, data science is a popular term, but I think a lot of people don't really know what it means or or what it is.
123
00:11:05.365 --> 00:11:15.580
If I look back when I was doing graduate school, I was taking scaturometry data from satellites and estimating wind speed. And that was data science to a degree, but so science has been doing data science for a long time.
124
00:11:15.925 --> 00:11:30.149
But currently today, it's kind of it's it's more popular because now the business professionals are doing it too. The marketer the marketing folks doing it too. The people look at their logs are saying, oh, we gotta get information from this, know what to sell to people. So it's got more money around it now and so people
125
00:11:30.450 --> 00:11:35.190
use it. The cool thing is that, you know, these tools we use for the scientists like NumPy
126
00:11:36.035 --> 00:11:37.654
have now this it's application
127
00:11:38.115 --> 00:11:40.134
to a much larger group of people.
128
00:11:40.435 --> 00:11:44.855
And Python, because it's so accessible to people besides just scientists and programmers,
129
00:11:45.230 --> 00:11:49.330
it's accessible to even business analysts with the right tools around it.
130
00:11:50.190 --> 00:11:54.770
Now they're in now now you this whole ecosystem is now made available to a lot of more people.
131
00:11:55.235 --> 00:11:56.214
So that's exciting.
132
00:11:56.675 --> 00:11:59.255
Absolutely. And, you know, talking about the the
133
00:11:59.635 --> 00:12:06.280
I have 2 things to say. The first is it's interesting. You talk about, you know, data science kind of becoming this almost sort of like supercharged
134
00:12:06.900 --> 00:12:10.520
term that the marketers have picked up. It's funny how many of those there are.
135
00:12:10.980 --> 00:12:12.680
It seems like they've really
136
00:12:13.084 --> 00:12:22.705
become more and more common in recent years, things like Cloud or DevOps and it's like Yep. You have this core group of people like yourself who built this thing that does a thing
137
00:12:23.600 --> 00:12:50.525
you know, and it it fills a niche and suddenly, like, a term gets applied to it and then it kinda takes on a life of its own and takes off. So Right. That's that's, you know And, you know, for data science, pandas has been really helpful to kinda orient. Pandas built on top of NumPy and orients it towards a particular audience that needs like, NumPy is a multidimensional array. Right? And, you know, I'm very excited about the multidimensional aspect of it. I've been doing, you know, 5 dimensional derivative calculations when grad school. That's why it was in my head
138
00:12:51.165 --> 00:12:53.905
at the time. But Pandas is just it's a 2 dimensional table.
139
00:12:54.285 --> 00:13:04.530
It's very it's a very simple structure, and you could do it in NumPy, but Pandas added an API on top and a couple of operations. A few operations are very simple, and it made it more accessible.
140
00:13:04.990 --> 00:13:07.410
And that's kind of been the I think Python generally
141
00:13:07.875 --> 00:13:19.850
makes it easy to create objects and structures that make coding and calculations accessible to others. And there's still a rich opportunity in a lot of spaces to do that. That's what's exciting. It's it's not done, not by any means.
142
00:13:20.310 --> 00:13:23.130
You know, I've I've always been passionate about creating
143
00:13:23.910 --> 00:13:32.055
technology solutions that let even more get built around it. You know, kind of looking at the fundamental blockers that are causing disconnects and then
144
00:13:32.355 --> 00:13:34.600
try to resolve some of those so that we can
145
00:13:35.160 --> 00:13:36.380
get get a cooperative
146
00:13:36.759 --> 00:13:38.779
explosion on top of additional ideas.
147
00:13:39.319 --> 00:13:53.805
So that's still available right now. In Python, I it's still the right language to do it in. It's still the the it's got its challenges. It's, you know, there's lots of things that I'd love to see different in Python, but it's still it's got a critical mass, it's accessible, people can understand it, and and
148
00:13:54.440 --> 00:13:56.620
it's it's still a great language for all of that.
149
00:13:57.240 --> 00:14:02.300
Absolutely. And I and I really think sort of dovetailing what you were just saying in the point you made previously
150
00:14:02.920 --> 00:14:07.065
about these tools having wider, you know, and broader and broader applicability.
151
00:14:07.765 --> 00:14:21.420
I've been seeing in the last few years, I work in the infrastructure as code space. I do a lot of work with, you know, things like Chef and Ansible and SaltStack and and all that kind of thing. And it's been really interesting watching people who build
152
00:14:22.125 --> 00:14:25.505
sort of care build care and feed for infrastructure
153
00:14:26.285 --> 00:14:31.025
sort of really start to leverage some of these amazing tools for, you know, data science
154
00:14:31.370 --> 00:14:32.190
in terms of,
155
00:14:33.050 --> 00:14:33.550
visualization
156
00:14:34.010 --> 00:14:37.150
and because it's really easy to end up with these huge
157
00:14:37.930 --> 00:14:55.700
masses of data that it can become very ungainly. And all of a sudden, when you can apply these, you know, hey, scientists have been dealing with this stuff for, you know, longer than we've been alive, and we now have these amazing computational tools that make it easier than falling off a log. It's it's really cool to see
158
00:14:56.165 --> 00:14:56.985
what happens
159
00:14:57.765 --> 00:15:02.345
when people from all kinds of different, you know, disciplines, fields, and and industries
160
00:15:02.965 --> 00:15:06.360
start to leverage this stuff and and only goodness can result.
161
00:15:06.820 --> 00:15:09.560
I agree. Python has allowed a lot of people to cooperate
162
00:15:09.860 --> 00:15:10.600
in building
163
00:15:11.220 --> 00:15:18.045
interesting tools that otherwise wouldn't talk to each other. That's a very interesting aspect of Python. Yeah. It definitely seems to have become sort of the,
164
00:15:18.605 --> 00:15:21.370
lingua franca of anybody who's trying to
165
00:15:22.410 --> 00:15:31.885
do integrations between a large variety of problem domains because of the, you know, myriad different libraries that are available for it and the power
166
00:15:32.345 --> 00:15:34.765
and flexibility of those libraries.
167
00:15:35.305 --> 00:15:45.360
Yes. Python got its got its strength as glue. I think we've just started to see how superglued that will be. It's sort of, you know, a glue that becomes like Legos.
168
00:15:48.125 --> 00:16:09.625
It's funny that you mentioned Legos because when you're talking about leveraging these core components that can get built out in more general ways, 1 of the architects where I work, he likes to use that term. But the truth is it's a really good term, especially for things like this that, you know, people just combine in really interesting ways. And you look at a little kid with a pile of Legos, and it's gonna be kind of amazing what they come up with sometimes.
169
00:16:10.089 --> 00:16:10.589
Yes.
170
00:16:11.209 --> 00:16:12.649
Yes. Yeah. To do
171
00:16:13.449 --> 00:16:15.310
just like with Legos, sometimes
172
00:16:16.490 --> 00:16:20.435
like, 1 thing about being Pythonic. Right? I've I've heard that word for a lot of years.
173
00:16:20.975 --> 00:16:23.155
1 angle of being Pythonic is,
174
00:16:23.615 --> 00:16:28.580
having the right structure in your Legos. Right? They can connect to each other. Right? Right. Right?
175
00:16:29.280 --> 00:16:33.380
You wanna make sure you can build on top and and have different layers of abstractions.
176
00:16:34.320 --> 00:16:39.825
So, yeah, that's, it's powerful. So I'm really excited still to be a part of the Python community and,
177
00:16:40.285 --> 00:16:45.170
I know that's, it's it's been a a tremendous ride and I'm still pretty passionate about it.
178
00:16:46.529 --> 00:16:56.275
Great. Great. So can you tell us some of the story of how Continuum Analytics came to be and sort of your position with that company? Yeah. So I started in academia
179
00:16:57.375 --> 00:16:58.995
and really love open source.
180
00:16:59.375 --> 00:17:01.555
Realized that but I also love markets.
181
00:17:01.970 --> 00:17:06.950
I I I my mind, open source is too valuable to the world to be left to just volunteerism,
182
00:17:07.490 --> 00:17:25.409
and so I wanna figure out and support any efforts in the marketplace to connect what we buy to what to what gets built in open source as well. So I I left Acone about 7 years ago, went to the industry and started working as a consultant. And in the process saw a lot of opportunities that for, you know, open source was solving problems that businesses needed.
183
00:17:25.870 --> 00:17:27.809
And so and over that time,
184
00:17:28.415 --> 00:17:30.195
built up some ideas particularly,
185
00:17:31.135 --> 00:17:36.995
saw an opportunity in bringing these tools to the data analytics world. And so, Peter and I created a continuum
186
00:17:37.950 --> 00:17:38.530
to connect
187
00:17:38.990 --> 00:17:47.330
the the scientific codes that we are very aware of and have been a part of to the larger data analysis problems, in particular to helping
188
00:17:48.005 --> 00:17:49.144
experts everywhere,
189
00:17:50.404 --> 00:17:52.345
build tools faster and easier.
190
00:17:52.804 --> 00:18:11.085
Our our motto is to connect expertise to data, our mission, our our our goal. A lot of people believe that there's gonna be a magic I don't know if they believe it or they hope it. I mean, I would love it too if there were some magic predictor that could take your data and just give you results. You have that here's the day input data input and then output comes exactly what I need to do next, you know, action statements.
191
00:18:12.025 --> 00:18:13.710
We're a long way from being there.
192
00:18:14.029 --> 00:18:37.230
I think in some cases, you can get close, but, generally, what's needed is the ability for people who understand the domain to quickly put together solutions they can iterate with quickly because the data changes, the problems change, and you come to an answer soon and quick and as fast as possible. And Python's a deal for that. So our our mission is to use Python and build on Python to make it easy to be able to build solutions
193
00:18:37.785 --> 00:18:39.645
that includes dashboards and visualizations
194
00:18:40.585 --> 00:18:41.145
and and,
195
00:18:41.545 --> 00:18:42.765
even to full applications
196
00:18:43.865 --> 00:18:46.205
very, very quickly from the large data they have.
197
00:18:46.679 --> 00:19:05.059
And so that that was that was the idea around continuum and so we we wanted to support open source. Our you know, 1 of our metrics of success is how many open source projects we're contributing to and and and releasing, because that's 1 of our reasons for existing is to give, to keep building this community that we that I find so compelling in the world.
198
00:19:06.880 --> 00:19:11.700
Well, I think by that metric, you you guys can be considered to be a raging success. I mean,
199
00:19:12.240 --> 00:19:38.130
I'm not really as as as well versed in in this area as my cohost here is, and Tobias was filling me in on I'd I'd heard of of Numba and NumPy, but I had no idea that you folks have a a whole little, you know, mini universe of projects that you folks have have brought into the into the community. It's it's it's kind of kind of impressive and really cool. I appreciate that. I mean, they they are we are we do try to focus a little bit around visualization.
200
00:19:38.430 --> 00:19:41.570
Voci is our visualization tool, and the goal there is to bring
201
00:19:42.510 --> 00:19:48.275
to make you not have to write JavaScript. You can write Python and then have interactive visualization in the browser.
202
00:19:48.815 --> 00:19:59.370
And then, you know, Blaze, I'm really excited about Blaze. Blaze is a topic for another podcast, though. It's all about helping people write expressions at a high level and translate into the back end where the data sits.
203
00:19:59.750 --> 00:20:03.050
It solves the problem of currently today when people have data problems.
204
00:20:03.590 --> 00:20:12.830
The first question they ask is where is my data because that defines how they talk about their data problems. If it's in a database, they write a SQL statement. If it's in CSV files, write some parsing code.
205
00:20:13.309 --> 00:20:13.890
If it's
206
00:20:14.429 --> 00:20:34.929
in HDFS, then they either do do spark or they write some hive or they they do something different depending on where it is. We don't think it should be that way. We think you should should have an understanding of your data as a as a high level object, data frame, or array, and then write your code. And then the back end, it's where it sits is a is a implementation detail and you can map your expression to wherever it sits.
207
00:20:35.230 --> 00:20:37.890
So much like SQLAlchemy does that for data and databases,
208
00:20:38.429 --> 00:20:40.850
Blaze does this for data everywhere.
209
00:20:42.190 --> 00:20:43.394
That's that's really
210
00:20:43.855 --> 00:20:44.355
neat.
211
00:20:46.174 --> 00:20:52.860
So when you say so when you I I like your your analogy of of of likening it to, to sequel alchemy.
212
00:20:53.640 --> 00:20:55.900
I guess I guess a question that I have is
213
00:20:56.440 --> 00:21:09.030
what I I guess what I I what I really need to do is after the show, go go look it up and and get some more detail. But can you give me, like, a a potential use case? Like, what is a real world problem that someone might have
214
00:21:09.410 --> 00:21:11.510
that Blaze would elegantly solve?
215
00:21:11.890 --> 00:21:23.970
There's a couple that it elegantly solves today. I mean, 1 is I wanna translate data 1 form to another. A lot of people don't try out the tools that are available, don't know the difference between a Cassandra or a Hive or a Spark or a or a SQLite
216
00:21:24.429 --> 00:21:45.120
implementation or using b calls and and PI tables. You know, they typically get get stuck with the current data format they have and there's and they're and they don't realize they could be having huge performance gains, they use a slightly different back end and maybe adjust the approach they're taking. And Blaze makes it easy to try them. You can take your data. You can write a high level expression, use your query, and then,
217
00:21:45.820 --> 00:21:54.485
then you can quickly port it. Say, okay. I'm doing this with SQLite or I'm doing it in my table in in Oracle. I'm doing it with a bunch of CSV files. What if I just convert all this to HDF 5 files,
218
00:21:54.865 --> 00:21:56.725
which is a scientific data format,
219
00:21:57.039 --> 00:21:59.779
and then ran my query. Could I could I be faster?
220
00:22:00.480 --> 00:22:02.740
And it lets you test that out very, very quickly.
221
00:22:03.440 --> 00:22:13.695
You know, within, you know, a few lines of code, you're right the back end is switched and your same query runs on that on that new set. So it lets you it lets you switch back ends very quickly and migrate data quickly.
222
00:22:14.430 --> 00:22:25.165
There's a sub product in the Blaze ecosystem called Odo. Odo does the the extract, you know, the the copying of data. Just as it's almost like you're saying copy, but you don't have to do all the work. You just say, here's my,
223
00:22:25.545 --> 00:22:34.290
URL to this to the CSV file collection in this directory, and then here's my URL to a bunch of tables in Hive. And it'll copy them. It'll just do the transforms for
224
00:22:34.750 --> 00:22:43.575
you. And then you can run a query on it. And a query is a table expression. So if you're used to the Pandas API, you can write a Pandas API like an expression and now it runs
225
00:22:44.035 --> 00:22:52.635
on the data in Hive. You don't have to learn Spark. You don't have to learn, MapReduce. You don't have to learn a new system. You can just use the same high level interface.
226
00:22:53.275 --> 00:23:01.860
That is super cool. And that's definitely a problem that a lot of people in a lot of fields are having, you know, like, where they realize, gee, our performance stinks and, you
227
00:23:02.240 --> 00:23:03.140
know, we're using
228
00:23:03.520 --> 00:23:09.540
a SQL database for this. But really, really, the way this data is being used isn't relational at all. Let's go
229
00:23:09.924 --> 00:23:11.065
explore a NoSQL,
230
00:23:11.525 --> 00:23:19.450
but that's not something as trivial as saying, hey. Change the back end out. Like, there's a lot of work involved there. And so to be able to sort of, like,
231
00:23:19.910 --> 00:23:35.890
toggle a toggle and try something else, that's that's potentially life changing for people. Yeah. That's really amazing. It Definitely. Our goal. And we and and in many cases, we we reach it. Right? There I'm not gonna say it's it's, like, mapping to a different dataset. If you know a larger dataset, you have transfer it. You have to actually
232
00:23:36.190 --> 00:23:41.445
you do have to connect it to a different you know, pulling it all out of the database. It's a simple 1 liner, but it might take 3
233
00:23:42.245 --> 00:23:46.424
hours. Right? There it's a transfer if your dataset's really big.
234
00:23:47.125 --> 00:23:56.870
But the idea is you can separate operational efforts from the coding involved and the mindset of the person. Because a lot of times, just finding those people that can do all that is really hard. Absolutely.
235
00:23:58.035 --> 00:23:59.175
Definitely just really
236
00:23:59.875 --> 00:24:01.815
reduces the amount of friction that
237
00:24:02.675 --> 00:24:06.100
and inertia that your data has. Yes. Exactly.
238
00:24:06.560 --> 00:24:09.940
Exactly. We kinda have this mindset of, you know, we're trying to reduce silos.
239
00:24:10.480 --> 00:24:11.620
You know, a lot of data
240
00:24:12.164 --> 00:24:15.945
reduce silos. People don't want silos. They wanna connect data from multiple sources.
241
00:24:16.565 --> 00:24:19.465
And, to get insights quickly, you need to be able to
242
00:24:19.765 --> 00:24:25.100
write ideas at a high level. Our big goal with Blaze is to turn the Internet into its own into a single database.
243
00:24:25.480 --> 00:24:27.659
Right? Even there is gonna take some effort. But
244
00:24:28.200 --> 00:24:32.365
but that the idea is why every data anywhere, any URL should essentially
245
00:24:32.745 --> 00:24:34.765
be like a table in a universal database.
246
00:24:36.840 --> 00:24:39.580
And you can access it from a pandas like API in Python.
247
00:24:41.400 --> 00:24:50.445
That's that's definitely really cool. I wonder if you might say a few words about, Okari because I really 1 of the things that really struck me when I came to the Python community was
248
00:24:50.905 --> 00:24:55.325
IPython notebook. I mean, it just this it it kinda I don't know if you you remember,
249
00:24:55.909 --> 00:25:03.370
I think you're old just by the sound of your voice, you sound like you might be old enough to remember this. There was a cartoon years years years ago with Tennessee Tuxedo
250
00:25:03.965 --> 00:25:09.425
and, mister Whoopi's 3 d b b, just like magic blackboard that he could use to sketch out
251
00:25:09.725 --> 00:25:17.970
anything in the animations, explain how things worked in the whole 9 yards. And IPython Notebook really kind of reminded me of that and the idea
252
00:25:18.590 --> 00:25:23.605
of being able to sort of, like, even add an additional dimension of that in in the realm
253
00:25:24.225 --> 00:25:26.644
of science and data analysis and experimentation
254
00:25:27.105 --> 00:25:34.930
seems like it fits the the metaphor even even more fully. Yeah. It's it's an amazing phenomenon. I've really it's been awesome to watch it,
255
00:25:35.230 --> 00:25:45.924
kind of emerge. And Fernando Perez and Brian Granger are both good friends of mine. We've we've been in this we've been in this together. We often, talk about how we're kind of old guard here in this in this fight for 18 years.
256
00:25:46.784 --> 00:25:47.284
But
257
00:25:47.670 --> 00:25:55.397
I'm IPython actually started with Jenko Houser or something called IPP that Fernando grabbed and used as the interface. He's
258
00:25:55.845 --> 00:25:56.745
the the key,
259
00:25:57.285 --> 00:26:04.025
superpower of Fernando is he's constantly trying to figure out how to improve the user experience. And Brian Granger joined him with a similar, desire.
260
00:26:04.530 --> 00:26:11.110
And, there's a guy named William Stein who actually built an early version of a JavaScript front end to the Python kernel
261
00:26:11.625 --> 00:26:13.405
and it's currently existing in SageMathLab.
262
00:26:13.705 --> 00:26:18.205
So he has this whole Sage interface which has been it's it's really geared towards the the,
263
00:26:19.065 --> 00:26:19.565
mathematician
264
00:26:19.960 --> 00:26:22.140
to the, kind of replacement for Mathematica.
265
00:26:22.919 --> 00:26:23.820
But he's, you know,
266
00:26:24.120 --> 00:26:28.460
he's a amazing guy and he since adopted IPython notebook as well, but he had that early thing in 2007
267
00:26:28.895 --> 00:26:40.730
and that inspired Fernando and Brian to go, let's build this. But they they architected around a kernel. You have a running kernel then an interface that could be swapped out. But that interface has cut captured the attention of a lot of people.
268
00:26:41.030 --> 00:26:44.250
People recognize that it's a way to communicate information quickly.
269
00:26:44.625 --> 00:26:48.085
At Continuum, we've constantly been looking at what are the bottlenecks to really
270
00:26:48.545 --> 00:26:49.765
enable collaboration
271
00:26:50.145 --> 00:26:51.045
and shared,
272
00:26:51.665 --> 00:27:07.585
understanding across a large number of because to me, that's the secret to getting insight from data. You're not gonna get insight from data because you just happen to get get lucky. It really is about taking what you know and interacting with the data and interviewing exploring and talking to somebody else about it, and you want that process, the whole workflow to be as seamless as possible.
273
00:27:08.044 --> 00:27:08.865
So we
274
00:27:09.179 --> 00:27:13.520
knew about IPython Notebooks, so have been very interested in kind of empowering people to use that more effectively
275
00:27:14.059 --> 00:27:21.235
and working with the IPython team to help them. So Akari really was a was a was a notion of the notebook's great, but it's not enough.
276
00:27:21.535 --> 00:27:25.875
You have to have the not only just the the interface, you have to have your environment, your code environment.
277
00:27:26.290 --> 00:27:36.085
Yep. All the dependencies that are needed to run that workflow have to be available and installed for you. The data that you that that work with depends on also has to be available and installed for you, and you and you want quickly available.
278
00:27:36.465 --> 00:27:45.290
So, you know, we started working with with those those ideas in mind and and came up with kind of an initial cloud based solution, and we also have an on premise solution that we we we install.
279
00:27:46.070 --> 00:27:50.330
We're still iterating. We have a lot of things we're improving with that basic idea.
280
00:27:51.165 --> 00:27:56.465
Mostly right now, we're we're we're resource constrained. We're looking to hire people because we don't have enough people to help us support
281
00:27:56.925 --> 00:28:03.679
the the the success we're having in some of these initiatives. So that's been a constraint for us. It's just getting the right people who can help.
282
00:28:03.980 --> 00:28:08.640
But yeah. So Akari is all about, again, collaboration, helping people leverage the IPython notebook.
283
00:28:09.245 --> 00:28:19.880
So I've you know, that that phenomenon is is is is a real 1, and a lot of people have seen that this, it can change their approach. I it can for some people, it's replacing Excel workflows, Excel workbook,
284
00:28:20.740 --> 00:28:26.005
flows. They'll instead of having a bunch of Excel sheets, they'll use an IPython notebook to express their work.
285
00:28:26.645 --> 00:28:29.545
Super excited about the the future potential of it.
286
00:28:30.725 --> 00:28:35.990
Great. And I have 1 1 last, last question in the realm of projects that you folks
287
00:28:36.710 --> 00:28:43.130
have come out with. Numba is 1 of the things that I had actually learned about before I even actively started using Python
288
00:28:43.505 --> 00:28:51.365
just because it made such a splash in the sort of general computing news. The idea of a, you know, JIT compiled
289
00:28:52.270 --> 00:28:59.010
hyper optimized Python for a numeric computation is is really, really super cool. And you talk about wide applications,
290
00:28:59.310 --> 00:29:05.375
I mean, that's being used in everything from, obviously, data science all the way out to games and, you know,
291
00:29:05.675 --> 00:29:06.895
all kinds of areas.
292
00:29:07.420 --> 00:29:31.500
That really, it's quite an achievement. So what led you folks to to build it, and what are some of the challenges, and and where do you see it going? It's a lot of fun. What led to it, honestly, was my desire when I wrote NumPy, the first thing I wrote actually and remember I talked about SciPy starting with bunch of modules I wrote? 1 of the first ones I wrote in 1998 was something called SIFIs, which is a bunch of special functions.
293
00:29:31.960 --> 00:29:37.975
Things like airy function and Bessel functions and a whole host of these scientific functions that nobody cares about unless you're in physics,
294
00:29:38.835 --> 00:29:46.340
but then show up in various ways. I wanted all these available to Python users and to do it so NumPy has this thing called the universal function,
295
00:29:46.720 --> 00:29:54.115
but to build a universal function you had to write c code. And I always wanted to be able to say kind of have a Python expression of the function
296
00:29:54.735 --> 00:30:02.500
and then create a virtual a a NumPy new func just writing Python code. So I've always wanted that, and I you can never do it
297
00:30:03.040 --> 00:30:08.394
because, frankly, it needs a compiler. You need to be able to a compiler that can take Python code and produce machine code
298
00:30:08.695 --> 00:30:13.755
to do that. So kinda with that in mind, I I came across the LLVM library, and
299
00:30:14.390 --> 00:30:15.370
it really helps.
300
00:30:15.750 --> 00:30:25.995
It's a great it's a great library system. There's there's issues in terms of compatibility with back versions and so forth, but but what it allowed is that I saw a lot of companies like Apple and NVIDIA
301
00:30:26.455 --> 00:30:29.355
using LLVM as a common compiler framework
302
00:30:29.975 --> 00:30:30.460
and
303
00:30:30.779 --> 00:30:37.600
realized there's an opportunity to use that to make the process compiling simpler. I did a compiler course after I wrote the first version of Numba.
304
00:30:38.135 --> 00:30:43.515
Fortunately, Numba's had 3 reversions since then with a larger team and people with more compiler expertise than I had.
305
00:30:43.895 --> 00:30:47.590
But I I was just crazy enough to think that I could do something once I had LLVM.
306
00:30:48.049 --> 00:30:52.230
Like I joke about, a compiler is easy to write if you don't have to write the parser or the code generator.
307
00:30:54.505 --> 00:31:01.245
And with Python, I don't have to write the parser because I get bytecode out of Python and I don't have to write the code generator because l v m does it for me. So it's truly just translating
308
00:31:01.545 --> 00:31:02.510
bytecode to
309
00:31:03.070 --> 00:31:04.850
lvm intermediate representation.
310
00:31:05.550 --> 00:31:08.930
There's still a lot of challenges and most of the challenges are semantic and definitional
311
00:31:09.470 --> 00:31:14.495
in terms of what are we really doing because, you know, taking arbitrary Python code and making it faster
312
00:31:15.034 --> 00:31:18.880
is a really, really hard problem. You can't do it in general.
313
00:31:19.360 --> 00:31:32.635
Right? What you have to do, you can have subsets of code and particularly, like, the kind of code that uses NumPy arrays and scalars and just if statements and so forth. You can make that fast. And there's no reason to write Fortran or c if that's the kind of code you're writing.
314
00:31:33.255 --> 00:31:39.750
So it was recognizing that and see what can we do. Let's make progress in here. It was inspired a little bit by a conversation with the Pew Pew community.
315
00:31:40.610 --> 00:31:51.865
Some people think there's a conflict. There isn't a conflict. There's just different ways of looking at the world. I actually see a way to forward to working with those folks now that maybe in a later point I could talk about. I'm pretty excited about it, actually.
316
00:31:52.440 --> 00:32:00.220
You know, I wrote a blog post because they PyPI was exciting to a lot of people. Wow. Future of Python. PyPI. That's awesome. But they were unaware
317
00:32:00.600 --> 00:32:00.915
or
318
00:32:01.795 --> 00:32:11.520
of the the the deep roots of the numeric Python scientific computing ecosystem and how connected to c code and Fortran code and all the c extensions that it required.
319
00:32:12.140 --> 00:32:18.400
Like, to really move that community over to PyPI would take an enormous effort, and they they seemed unaware of that challenge.
320
00:32:18.860 --> 00:32:28.385
And so my approach say, look. We'll start doing the other way. Well, you know, maybe we'll meet in the middle somewhere. Right? We'll start with take the NumPy sci fi community and start adding JIT compilation
321
00:32:29.010 --> 00:32:38.765
and then add those features in that direction. So it's a different approach, same problem. It's all possible, but, again, how much effort it's gonna take? You know, with the right 1, 000, 000 of dollars, we can do everything.
322
00:32:40.425 --> 00:32:48.860
But, you know, how do we do this in a community in a way that we can meet in the middle? And I think there's actually some really powerful solutions in the that that could be accomplished if we work together.
323
00:32:49.799 --> 00:32:59.575
I'll bet. It's it's really interesting that you say that. It it really occurs to me that so often in technology, we get this, like, you know, I'm gonna do it this way. I'm gonna do it that way. And it's like,
324
00:32:59.955 --> 00:33:09.160
whether it is that way or not, people assume that it's like, you know, pistols at 20 paces, when in reality, it's, you know, it's just a different approach to the problem and and and
325
00:33:10.155 --> 00:33:14.015
it it it benefits no 1 to have this kind of bizarre
326
00:33:14.315 --> 00:33:21.030
everyone is in constant competition. I mean, healthy competition sort of like, you know, I can do better than that is is good, but,
327
00:33:21.490 --> 00:33:24.390
you know, not collaborating for the sake of partisanship
328
00:33:24.690 --> 00:33:26.995
makes no sense. I I came from
329
00:33:27.375 --> 00:33:35.770
many years in in Ruby, I learned Python, and and all of a sudden I'm realizing you guys have some really cool toys in this side of the pool. I mean,
330
00:33:36.070 --> 00:34:04.285
it just makes sense. There's no reason to to block yourself off from from what other people are doing. It may not be what you would choose. It may not even be the right answer for you, but at least be aware of it and be open to it. Totally agree. So actually right now, I'm I I see a lot of, in our future is integrating with with the Java Stack and with the R community and with like, it's really you know, a lot of people are trying to solve the same problems and how do we do it in a way that's cooperative so we can share each other's successes.
331
00:34:04.985 --> 00:34:10.285
That's that to me would be the dream. It's it's difficult. There's some real challenges there, but there are also
332
00:34:10.940 --> 00:34:14.320
good there are solutions at times, especially if you're looking for them.
333
00:34:14.620 --> 00:34:18.960
And very, very transformative solutions if you if you keep searching.
334
00:34:19.425 --> 00:34:25.845
So that drives me. I'm excited about that. I like to I guess maybe all computer programs are lazy, some people say.
335
00:34:26.224 --> 00:34:28.070
We all wanna just take advantage of other people's
336
00:34:29.030 --> 00:34:36.890
work. And that's certainly I mean, SciPy was a making available old Fortran code, you know, bring it back to life and connect it to the modern user.
337
00:34:37.425 --> 00:34:41.045
And that was a big part of what made sci fi, and it's still a big part of sci fi.
338
00:34:41.825 --> 00:34:42.645
And that's
339
00:34:43.025 --> 00:34:47.630
you gotta keep doing that. You know? Let's let's instead of having to reinvent the wheel, let's figure out how to connect with each other.
340
00:34:48.730 --> 00:34:57.605
So switching gears a bit. Can you explain a bit about what Numfocus is and how that got started? Oh, yeah. I'd love to, actually. So Numfocus when I worked as a consultant,
341
00:34:58.385 --> 00:34:58.625
I,
342
00:34:59.185 --> 00:34:59.925
I recognized
343
00:35:00.385 --> 00:35:03.990
the the challenges of being in a company and then also supporting community.
344
00:35:04.450 --> 00:35:13.295
Because I do I'm passionate about community, but I also am passionate about markets and I and I have to feed my family, I have to have a job. And sometimes the the pressures are
345
00:35:13.674 --> 00:35:23.430
they cause people to be a little suspicious too. Like, our company is doing this. What does it mean? And so I wanted to create an organization that was fully community run and and and funded and and and supported.
346
00:35:23.890 --> 00:35:38.910
So the if people had concerns you know, if you you wanna support companies. I think some companies are doing great things. Go buy their stuff, help them, make them successful. But there are other people who wanna just support the community. So I wanted to make sure there was a place where Sandeep Python could be community supported.
347
00:35:39.530 --> 00:35:58.340
And people could understand if they had any concerns, they could they could just focus on 1 in the community side. Then it also becomes a place that people can have different ideas in the marketplace. You know? My company, your company, we have different ideas. We can cooperate together through an organization. So same time, if we found a continuum, I also found a numb focus and got together with leading, you know, with, Brenda Perez and John Hunter and Perry Greenfield and Jared and Anthony
348
00:36:00.855 --> 00:36:08.315
John Hunter and Perry Greenfield and Jared and Anthony Scopats. And we created NumFOCUS as a community centric organization,
349
00:36:08.615 --> 00:36:12.860
much like the PSF, much like the Apache Apache Foundation, much like the Linux Software Foundation.
350
00:36:14.280 --> 00:36:17.955
Continuum basically hired an executive director and gave her full time to work on Unfocus.
351
00:36:18.435 --> 00:36:20.695
That's kind of our commitment to the community, and
352
00:36:20.995 --> 00:36:25.815
that's been a successful approach. She's been able to really rally a community around PyData,
353
00:36:26.530 --> 00:36:30.470
around the John Hunter Technical Fellowship, around women in science technology events,
354
00:36:30.930 --> 00:36:33.330
and really help that grow into a,
355
00:36:34.375 --> 00:36:38.555
what should and what we and and the fiscal sponsorships for SymPy and for
356
00:36:38.934 --> 00:36:39.434
IPython
357
00:36:39.815 --> 00:36:40.555
and for
358
00:36:40.855 --> 00:36:42.395
several other projects.
359
00:36:42.775 --> 00:36:43.835
And then, you know,
360
00:36:44.440 --> 00:36:44.940
eventually,
361
00:36:45.240 --> 00:36:50.940
we're we're finally getting fiscal sponsorship for NumPy itself. 1 of the challenges is as I step back away from NumPy,
362
00:36:51.615 --> 00:36:59.714
you leave a little bit of a vacuum and and then kind of there's a few people. There's like a committee of people who really make it work and keep running the NumPy and Cypher ecosystems,
363
00:37:00.730 --> 00:37:06.910
but helping them get a fiscal sponsorship together so we can fund them. So the mission of NumFOCUS is to fund the tools everybody uses
364
00:37:07.210 --> 00:37:11.245
and to be a rallying point for raising money to help these tools keep going.
365
00:37:12.185 --> 00:37:14.525
So it's a fully support if it's a 501c3
366
00:37:14.825 --> 00:37:15.325
nonprofit,
367
00:37:15.785 --> 00:37:26.410
you can jump in and participate as a as a member and donate. You can, use your time. You can participate as a in 1 of the Pi Data events. There's a lot of ways to participate and become a supporter of the community.
368
00:37:28.214 --> 00:37:31.275
That's that's really very cool. I think I think that
369
00:37:32.535 --> 00:37:33.275
so much
370
00:37:33.815 --> 00:37:34.954
how should I say this?
371
00:37:35.335 --> 00:37:38.530
I look at so many technology stacks like like,
372
00:37:39.090 --> 00:37:43.670
Linux as an example, Linux on the desktop. And I think to myself, you know, it is such a shame
373
00:37:44.025 --> 00:37:47.645
that these folks can't come together and just agree on
374
00:37:48.025 --> 00:37:53.690
some standards, some interfaces. Like like, you know, you can have a completely different way of doing things, but
375
00:37:54.170 --> 00:37:56.829
being able to to come together and
376
00:37:57.690 --> 00:38:00.270
support the infrastructure that you both use
377
00:38:00.865 --> 00:38:01.365
and
378
00:38:02.465 --> 00:38:03.605
figure out a way
379
00:38:03.985 --> 00:38:04.725
to not
380
00:38:05.585 --> 00:38:11.380
inconvenience and fragment your user base, that's definitely a wonderful thing. It sounds like that's part of what NumFocus,
381
00:38:11.840 --> 00:38:13.940
in addition to offering physical support,
382
00:38:14.320 --> 00:38:21.355
is trying to do. Is that is that right? Oh, absolutely. Yeah. It's it's definitely a place where conversations can take place that between projects,
383
00:38:21.734 --> 00:38:27.310
where people can know there's a there's an interesting body of folks who care about the overall the overall experience
384
00:38:27.850 --> 00:38:37.435
and wanna make sure that scikit learn and IPython and NumPy and SciPy are all kind of talking together as even though they're independent projects, that they have standards interfaces they agree on.
385
00:38:37.895 --> 00:38:44.240
A lot of that energy sort of happened in fact, you know, I, Fernando Perez, when we we organized NumFOCUS, he had the thought of, you know, confederation
386
00:38:45.099 --> 00:38:45.839
of of federate
387
00:38:46.220 --> 00:38:49.365
groups coming together and having a place in a in a common
388
00:38:49.745 --> 00:39:00.170
organization. He used references to the founding of the country, actually, the United States of America. Oh, wow. Different states, and they only need to come together and there'd be a group. And so, you know, there that's been a part of it. It's figuring out how do we
389
00:39:00.550 --> 00:39:13.430
we open source definitely has a community and a and a individual spirit and it needs to, like, retain that, but there does it helps to have organizations that are supportive of that, but a place for people to come together and and, take action together.
390
00:39:14.210 --> 00:39:21.670
And there's it's great to be involved, you know, to have a legal entity that can take money, a legal entity that can employ people, a legal entity that can
391
00:39:22.385 --> 00:39:22.885
support
392
00:39:23.185 --> 00:39:32.610
if if something you know, patents and trademark ownership and all those things that are important in to associate with the laws of a of a particular country, but it can represent the community.
393
00:39:32.990 --> 00:39:48.580
There are challenges associated with it. You know, volunteer run is is hard. That's why I've been grateful that we can continue to support an executive you know, Leah Silan is the executive director and she's a a driving force in the community just as an actor. We can continually move things along. Other companies are starting to give resources as well.
394
00:39:49.140 --> 00:39:50.840
So it's exciting to see.
395
00:39:51.140 --> 00:39:58.565
So it's really starting to take off and I'm really pleased about, the response to community to organize it. So just helping sustain itself.
396
00:39:59.045 --> 00:40:04.265
Again, my passion has always been about I love open source. I love to see it work. I love to see it sustained.
397
00:40:04.885 --> 00:40:13.559
You know, I I have 6 kids. Right? And I have 3 kids in undergraduate school. So from the very beginning of my involvement with open source, I've been aware of the need for me to provide for my family.
398
00:40:13.905 --> 00:40:20.964
And so I've had to figure out how do we continue to do this and still put shoes on the kid's feet and put them through now it's put them through college.
399
00:40:21.820 --> 00:40:33.675
And so that's that's been a part of what I you know, I wanted that for everybody. You know, how do we how do we make this this work? And sometimes it's through community and sometimes it's through companies. I think both participate and work together to make it happen.
400
00:40:35.255 --> 00:40:35.755
Absolutely.
401
00:40:36.615 --> 00:40:40.315
So for someone just starting out in the data science and, pardon me, analytics space,
402
00:40:40.620 --> 00:40:42.000
what advice would you give?
403
00:40:42.700 --> 00:40:45.040
Yeah. I would say learn Python as much as you can.
404
00:40:45.420 --> 00:40:47.120
I would say, download Anaconda.
405
00:40:47.580 --> 00:40:58.029
I would say go and, do a do a Google search on data analysis top in Python. Learn learn learning basic data analysis in Python. And there's enormous number of resources on,
406
00:40:58.430 --> 00:41:01.089
IPython Notebooks that'll just teach you and you can walk through.
407
00:41:01.470 --> 00:41:13.910
I would say go to, find a community, find a group, find a local meetup group, or find a local, you know, either Pydata or a data science group or a even the R community can be helpful. You might go there and find a few like minded interested folks.
408
00:41:14.309 --> 00:41:15.690
You can take an online class,
409
00:41:16.390 --> 00:41:17.210
but get involved
410
00:41:18.230 --> 00:41:24.164
and have it and then have something you care about. Like, if you wanna be doing data science, it means you wanna you wanna transform information
411
00:41:24.865 --> 00:41:26.885
into a form to answer a question.
412
00:41:27.265 --> 00:41:31.765
So so think what do you wanna do? Find a problem you're interested in and do the analysis
413
00:41:32.320 --> 00:41:34.740
and just and and use that to guide your studies.
414
00:41:35.840 --> 00:41:42.865
That's great. I think that'll definitely be some very useful advice to a lot of people particularly because of data analysis
415
00:41:43.245 --> 00:41:45.585
and data science being such big
416
00:41:45.965 --> 00:41:47.640
big industries now. And,
417
00:41:48.440 --> 00:41:51.180
you know, the I've been reading a lot of things about
418
00:41:51.640 --> 00:42:09.119
how lots of companies are hiring for it. They don't have enough people to fill the positions that they are trying to fill, and so it's definitely a very lucrative direction for people to go in whether they're just starting off in technology or just getting out of college or if they've been in the industry for years years and are just looking for something new to do.
419
00:42:09.420 --> 00:42:14.615
It is. 1 reason Python's an excellent choice is and don't don't be afraid to learn some programming.
420
00:42:14.915 --> 00:42:21.910
I think a lot of people find that the the the the folks want data scientists, but they want data scientists who can program. Yep. We put together a solution.
421
00:42:22.850 --> 00:42:25.350
We had a lot of applicants for data science roles.
422
00:42:25.650 --> 00:42:30.070
Our needs like, the number of needs where we want someone just to be analyzing data are small.
423
00:42:30.994 --> 00:42:38.055
Where we want people to be able to take the problem somebody's trying to solve, again, because the tools right now are not where they could be and will be in 10 years.
424
00:42:38.510 --> 00:42:40.690
Today, there's still a you gotta do some integration.
425
00:42:41.150 --> 00:42:46.849
You gotta do some gluing together with legacy solutions with legacy data, and that's gonna take some some programming.
426
00:42:47.265 --> 00:43:02.660
Python can do it all for you. So just learn Python and become good at it and don't be afraid to steer around. Maybe you're just interested in Pandas and NumPy, but Don't be afraid to learn a little bit about, you know, the URL, you know, request library or the other libraries and, you know, a parsing library,
427
00:43:02.960 --> 00:43:06.715
a scraping library. Don't be afraid to learn a little more and expand your knowledge.
428
00:43:08.055 --> 00:43:13.280
So of your myriad achievements and projects that you've been involved with over your
429
00:43:13.740 --> 00:43:17.840
life, what are you most proud of? That's a tough question. I mean, honestly,
430
00:43:19.335 --> 00:43:21.515
honestly, it's my kids who I'm most proud of.
431
00:43:22.055 --> 00:43:26.075
That's a tough thing to say to a dad of 6 kids. He's oldest as 20 and youngest as 7.
432
00:43:27.210 --> 00:43:28.670
That tells you how old I am.
433
00:43:29.770 --> 00:43:31.150
So beside my family,
434
00:43:31.450 --> 00:43:32.270
I would say
435
00:43:32.730 --> 00:43:36.805
jury's still out a bit. I'm certainly proud of what NumPy has become.
436
00:43:37.185 --> 00:43:42.805
Certainly, you know, and and realign and mostly because the effort it took to to create it and it was definitely a,
437
00:43:43.530 --> 00:43:46.510
it was 1 of those situations where I did not know the end from the beginning,
438
00:43:46.970 --> 00:43:48.910
had to, you know, leap of faith,
439
00:43:49.290 --> 00:44:00.095
feel this urge, feel this past, feel this need, and take a step in the dark and without a lot of support and then but but have the confidence. And so, you know, and then have it emerge and and be a a real success.
440
00:44:00.690 --> 00:44:08.075
Definitely definitely proud of that. But you know, there's other things I'd like to see more proud of in the future. So hoping to replicate that in in other projects.
441
00:44:09.115 --> 00:44:09.855
That's great.
442
00:44:10.635 --> 00:44:11.135
So
443
00:44:11.595 --> 00:44:13.535
at the end of our episodes, we like to
444
00:44:14.474 --> 00:44:20.540
provide listeners with some pics. And so this can be anything that you find interesting enough to wanna share with other people.
445
00:44:21.080 --> 00:44:28.035
So it could be technology, it could be a movie, it could be a board game, whatever it happens to be. So we'll get it started.
446
00:44:28.655 --> 00:44:32.115
So my first pick is going to be used bookstores.
447
00:44:32.750 --> 00:44:33.970
They are amazing.
448
00:44:34.830 --> 00:44:39.090
There's 1 actually down in East Lyme, Connecticut called the Book Barn,
449
00:44:39.390 --> 00:44:55.205
and I took my kids and my wife there recently, and we walked out with a giant box of books for about a $100, which would otherwise probably have cost us about 5 times that much. So used bookstores are amazing for getting a lot of really good and interesting material for
450
00:44:55.765 --> 00:44:57.945
reading, whether it's fiction or nonfiction
451
00:44:58.245 --> 00:45:02.720
or kids' books. Just great thing to go out and do. Good way to spend the
452
00:45:03.260 --> 00:45:07.599
day. My next choice is going to be the movie cloudy with a chance of meatballs.
453
00:45:08.380 --> 00:45:15.725
The book is a kid's classic, has been for years. The movie came out a couple years ago, and it is hilarious.
454
00:45:16.425 --> 00:45:20.000
My 4 year old loves watching it. He can watch it repeatedly,
455
00:45:20.620 --> 00:45:25.520
and I've seen it a few times, and I still think it's 1 of the funniest kids' movies I've ever seen.
456
00:45:26.540 --> 00:45:32.305
And then going on that theme for funny movies, another really great 1 is Kickin' It Old School with Jamie Kennedy.
457
00:45:33.005 --> 00:45:47.655
And I've watched that movie probably 8 or 9 times, and I still love watching it. It is 1 of the funniest movies I've ever seen. So for anybody who has even a tangential experience with the eighties, it's well worth a watch. Mhmm.
458
00:45:47.955 --> 00:45:49.015
Yeah. Awesome.
459
00:45:49.875 --> 00:45:51.895
Chris, why don't you go ahead? Alrighty.
460
00:45:52.515 --> 00:45:58.980
So this week, the first thing I'd like to pick to to sort of continue along Tobias' theme of comedians
461
00:45:59.520 --> 00:46:05.845
is kids in the hall. For anybody who went to college around the same time I did in the sort of late eighties early nineties,
462
00:46:06.545 --> 00:46:10.730
these guys are in a Canadian comedy troop, and they are just
463
00:46:11.270 --> 00:46:13.050
so funny, so bizarre,
464
00:46:13.590 --> 00:46:21.095
really great stuff. I mean, they've gone on to do lots of other great things. You probably know, if not them, then some of the work they've done in other
465
00:46:21.395 --> 00:46:23.975
venues, but, they still make me
466
00:46:24.355 --> 00:46:26.055
laugh, you know, out loud
467
00:46:26.359 --> 00:46:29.020
20 years later. So that's that says something, I think.
468
00:46:29.320 --> 00:46:36.904
The next pick that I have is the Museum of Fine Arts here in Boston. Every year they do this really cool event called Art in Bloom,
469
00:46:37.204 --> 00:46:39.385
and it's really something kind of neat. Like,
470
00:46:40.164 --> 00:46:42.184
every year they they
471
00:46:43.285 --> 00:46:44.984
they call in floral designers,
472
00:46:45.910 --> 00:46:46.650
oddly enough,
473
00:46:47.030 --> 00:46:49.210
to make floral designs
474
00:46:49.590 --> 00:46:52.570
and pair them with pieces of art all around the museum.
475
00:46:53.175 --> 00:46:57.994
So, for the over the course of a weekend, you get to wander around the Museum of Fine Arts
476
00:46:58.375 --> 00:46:58.875
and
477
00:46:59.175 --> 00:47:02.315
see all these really kind of amazing, creative,
478
00:47:02.680 --> 00:47:04.060
cool floral designs
479
00:47:04.680 --> 00:47:05.580
paired with
480
00:47:05.880 --> 00:47:06.780
great art,
481
00:47:07.320 --> 00:47:09.900
you know, and and the Museum of Fine Arts has some really
482
00:47:10.825 --> 00:47:19.565
timeless pieces. So it's just a really great experience. It's it's sort of like a great, you know, a great evening or day out, and and I highly recommend it.
483
00:47:20.200 --> 00:47:21.820
Then my next pick is,
484
00:47:22.440 --> 00:47:23.980
Saran Bark and the
485
00:47:24.359 --> 00:47:25.900
the the code newbies community.
486
00:47:26.680 --> 00:47:38.550
Talking about, you know, bringing people in and and enabling people to do good work. You know, so many there's been so much discussion out in our field right now about sort of, like, diversity and being welcoming to newcomers.
487
00:47:39.170 --> 00:47:44.515
And she, more than anybody else that I can think of, has really sort of like
488
00:47:44.914 --> 00:47:47.734
walked the walk. She's created a whole little
489
00:47:48.194 --> 00:47:49.894
empire of communities,
490
00:47:50.355 --> 00:47:53.870
sub communities, I guess. They do she does a weekly Twitter chat,
491
00:47:54.250 --> 00:47:55.230
they have a Slack
492
00:47:55.610 --> 00:47:57.870
team channel room, whatever it's called.
493
00:47:58.330 --> 00:48:02.315
They have a discourse forum and it's all oriented towards
494
00:48:02.775 --> 00:48:12.140
helping people get started with programming. And once they do sort of, like, get their foot in the door and get that get that first job or get that dream job in software development. And
495
00:48:12.440 --> 00:48:31.390
just sort of like and also, she does a podcast that's totally amazing and whether you're new or old or have been coding for 20 years like I have, it's really neat and and worth a listen. So kudos to her and all the work she does. And my last pick, because I've been blathering on long enough, is the Apple 27 inch,
496
00:48:31.770 --> 00:48:35.875
Retina Imac 5 k, which I'm sitting in front of as we speak.
497
00:48:36.355 --> 00:48:42.695
I realized that this is gonna out me as a total Apple fanboy and that's okay. I love this machine. It is
498
00:48:43.075 --> 00:48:46.599
fast. The display is just as gorgeous as you might think.
499
00:48:47.140 --> 00:48:48.839
It's beautifully engineered,
500
00:48:49.460 --> 00:48:50.440
well put together,
501
00:48:50.980 --> 00:48:51.800
and just
502
00:48:52.105 --> 00:48:52.925
a really
503
00:48:53.585 --> 00:48:55.885
a real pleasure to set up, use.
504
00:48:56.665 --> 00:49:02.310
It has been a delight since I've gotten it, and I can't recommend it highly enough for anybody who needs
505
00:49:02.850 --> 00:49:08.950
a machine with, you know, a fair bit of horsepower under the hood, but doesn't necessarily wanna go through the
506
00:49:09.255 --> 00:49:11.515
pain of, you know, building their own
507
00:49:11.895 --> 00:49:16.315
PC or or supercharged PC or something like that. It's a really impressive machine.
508
00:49:17.020 --> 00:49:19.440
Travis, why don't you why don't you give us your picks?
509
00:49:19.900 --> 00:49:23.635
Awesome. Wow. Okay. Okay. I'll I'll start with Data Carpentry.
510
00:49:24.355 --> 00:49:25.795
Data Carpentry is,
511
00:49:26.115 --> 00:49:44.535
basically, an organization run by Tracy Teal. Just getting off the ground kind of after the pattern of software carpentry, which was very successful at training scientists how to program. Data carpentry is about training people into various industries how to deal with data and how to how to how to manipulate it, how to use it. So it's it's, they're just getting off the ground, but check them out, datacarpentry.org.
512
00:49:45.315 --> 00:49:46.454
Tracy is an amazing,
513
00:49:46.915 --> 00:49:51.869
participant in the Python community. She was at pie PyCon this year with her family and was,
514
00:49:52.750 --> 00:49:54.849
has been a longtime supporter of Python.
515
00:49:55.710 --> 00:49:57.035
So that's 1.
516
00:49:57.575 --> 00:50:02.315
Second 1 is the, I would say the Brain Science podcast by Ginger Campbell, MD.
517
00:50:02.695 --> 00:50:17.990
She was a emergency room physician who who decided that she really loved science and wanted to go back to her roots and start a science podcast. I love it. It's an old 1. It's been out for a while, but you can still go there and get a lot of great book ideas about how the brain works, and I've just really enjoyed listening to her.
518
00:50:18.470 --> 00:50:19.210
And finally,
519
00:50:19.590 --> 00:50:25.050
a little bit maybe different, but, a favorite book of mine is, Money, Bank, Credit, and Economic Cycles.
520
00:50:25.510 --> 00:50:30.995
It's a bit of a big book. It's pretty thick. It's not light reading. It's definitely for someone who's serious about trying to understand.
521
00:50:31.455 --> 00:50:35.760
I I I feel like I understand the world much better. I understand,
522
00:50:36.300 --> 00:50:50.065
financial systems and banking much better because I've read this book. It's it takes you through the history of money from the Roman Roman days to today and kind of all and and how it works at a fundamental level. So really appreciate that. He's out of Spain. He's a Spanish professor,
523
00:50:50.445 --> 00:50:54.750
at the University of Ray car Juan Carlos Madrid. Really appreciate that
524
00:50:55.210 --> 00:50:57.069
book. So that's that's it.
525
00:50:58.089 --> 00:51:05.535
Great. Well, we really appreciate you taking the time out of your evening to speak with us. It's been a lot of fun, really interesting.
526
00:51:06.315 --> 00:51:18.300
So for anybody who wants to follow you and continue analytics and the work you guys are doing, what would be the best way to find you and keep track of your Yeah. You can follow me on Twitter. I'm at teolephant,
527
00:51:18.760 --> 00:51:26.605
t o l I p h a n t. You can follow PyData, you know, at pydata, p y d a t a. You can come to our website and follow at continuum IO.
528
00:51:27.130 --> 00:51:30.910
Twitter is easy. Facebook, we post too occasionally. You can come to our website.
529
00:51:31.690 --> 00:51:39.145
We're we're at every piloted event. So look for piloted events in in an area near you, and that's a way to kind of follow both the company and the community.
530
00:51:40.325 --> 00:51:41.145
Great. Great.
531
00:51:42.005 --> 00:51:48.340
Great. Alright. Well, it's been a real pleasure. I appreciate everything, you're doing. Thanks for inviting me. Thank you for coming.