#40: Top 10 Data Science Stories from 2015

WEBVTT
00:00:00.001 --> 00:00:05.500
It's the end of the year, and many of you are probably kicking back and taking it easy without a TPS report to be seen.
00:00:05.500 --> 00:00:08.020
So we'll keep this fun and lighthearted this week.
00:00:08.020 --> 00:00:18.440
We're running down the top 10 data science stories of 2015 on episode 40 of Talk Python To Me with guest Jonathan Morgan, recorded December 13th, 2015.
00:00:18.440 --> 00:00:48.420
Welcome to Talk Python To Me, a weekly podcast on Python, the language, the library, the language, the library, the language.
00:00:48.420 --> 00:00:50.080
The ecosystem and the personalities.
00:00:50.080 --> 00:00:52.180
This is your host, Michael Kennedy.
00:00:52.180 --> 00:00:54.180
Follow me on Twitter where I'm @mkennedy.
00:00:54.180 --> 00:01:00.600
Keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter via at Talk Python.
00:01:00.600 --> 00:01:04.680
This episode is brought to you by Hired and DigitalOcean.
00:01:04.680 --> 00:01:10.740
Thank them for supporting the show via Twitter where they're at Hired underscore HQ and at DigitalOcean.
00:01:10.740 --> 00:01:12.580
Hey, everyone.
00:01:12.580 --> 00:01:14.360
This episode's a little unique.
00:01:14.360 --> 00:01:21.460
I've partnered with a great data science podcast called Partially Derivative, and we're doing a joint show multicast of both podcasts.
00:01:21.460 --> 00:01:26.460
If you like this sort of thing, be sure to check out partiallyderivative.com and subscribe to their show.
00:01:26.460 --> 00:01:29.700
Also, I wanted to let you know I'm not releasing a show next week.
00:01:29.700 --> 00:01:31.540
I'm on vacation.
00:01:31.900 --> 00:01:40.520
So I'm going to take next week off, do a little resting and relaxation, hang out with the family, and be ready to get back and do a ton of awesome shows for you in 2016.
00:01:40.520 --> 00:01:44.460
Now, let's get on to this co-hosted episode I did with Jonathan Morgan.
00:01:44.460 --> 00:01:46.680
Hey, Jonathan.
00:01:46.680 --> 00:01:47.420
Welcome to the show.
00:01:47.420 --> 00:01:48.820
Hey, Mike.
00:01:48.820 --> 00:01:49.700
Thanks so much for having me.
00:01:50.140 --> 00:01:58.480
Yeah, I'm really excited to do this joint Talk Python partially derivative podcast about the end of the year and data science and all that awesome stuff.
00:01:58.480 --> 00:02:00.300
Yeah, I'm super excited, too.
00:02:00.300 --> 00:02:07.540
I think when our powers combine, it is the ultimate resource for Python data, Python stuff in the universe.
00:02:07.540 --> 00:02:08.120
Absolutely.
00:02:08.440 --> 00:02:09.420
Yeah, it's going to be great.
00:02:09.420 --> 00:02:10.080
It's going to be great.
00:02:10.080 --> 00:02:13.060
So for those of you who don't know me, my name is Michael Kennedy.
00:02:13.060 --> 00:02:17.920
I'm the host of Talk Python2Me, a sort of developer-focused Python podcast.
00:02:17.920 --> 00:02:21.380
And this week, I'm teaming up with the partially derivative guys.
00:02:21.380 --> 00:02:33.680
And for those of you who don't know me, I'm Jonathan Morgan, one of the co-hosts of the partially derivative podcast, a podcast about data science, kind of, but also sort of about screwing around and drinking beer.
00:02:34.320 --> 00:02:38.800
Yeah, that's really the most common combination of any two things, I would say.
00:02:38.800 --> 00:02:41.480
It's probably drinking beer and data science, right?
00:02:41.480 --> 00:02:45.320
All of the best data science is done in a two-beer buzz.
00:02:45.320 --> 00:02:46.960
It's the secret nobody tells you.
00:02:46.960 --> 00:02:47.760
That's right.
00:02:47.760 --> 00:02:49.720
You only learn it in grad school.
00:02:49.720 --> 00:02:51.320
Yeah, exactly.
00:02:51.320 --> 00:02:54.060
I kind of like to hear about your company.
00:02:54.060 --> 00:02:59.700
The last I heard is you guys were starting a data science company, and that's about all I heard.
00:02:59.700 --> 00:03:00.560
It's called Popply, right?
00:03:00.560 --> 00:03:02.280
Yeah, that's right.
00:03:02.280 --> 00:03:10.880
So it's called Popply, and my two co-hosts of partially derivative, Chris Albon and Vidya Spandana, started this data science company.
00:03:10.880 --> 00:03:17.160
And so basically, we were realizing that there's a whole bunch of data science that's super hard.
00:03:17.160 --> 00:03:25.280
And everybody, I think, is familiar with this kind of artificial intelligence and complicated machine learning.
00:03:25.620 --> 00:03:28.380
But then there's a lot of data science that's actually pretty straightforward.
00:03:28.380 --> 00:03:35.040
It kind of boils down to inference and making charts out of data that you just have sitting around.
00:03:35.040 --> 00:03:37.620
So you have a better everyday idea about what's happening.
00:03:38.140 --> 00:03:39.700
And it turns out that that's super hard.
00:03:39.700 --> 00:03:42.280
Like, actually, even for pretty technical people, it's super hard.
00:03:42.280 --> 00:03:51.080
You know, like, I meet a lot of developers who are like, they'll ask me a question like, so I've got like, like, thousands of rows of data in my database.
00:03:51.080 --> 00:03:53.720
And they're all like, at a time.
00:03:53.720 --> 00:03:58.300
But then how do I like, look at it, you know, like, over time?
00:03:59.260 --> 00:04:00.740
And it's like, oh, that's right.
00:04:00.740 --> 00:04:06.040
Like, I mean, super technically competent people who are still like, I just don't really understand how the data thing works.
00:04:06.040 --> 00:04:10.820
And that first kind of turning that first corner was really important to us.
00:04:10.820 --> 00:04:11.520
We're like, okay, cool.
00:04:11.520 --> 00:04:17.400
We could actually empower a lot of people to do data stuff if we could make that first step automatic.
00:04:17.400 --> 00:04:24.260
Like, if we could just go from some raw data to charts to give you an idea about what's happening inside your data, we should definitely do that for people.
00:04:24.260 --> 00:04:25.740
And so that seems easy.
00:04:25.740 --> 00:04:27.540
It took us a little bit longer than we thought.
00:04:29.040 --> 00:04:31.720
All those details, they keep sneaking in there.
00:04:31.720 --> 00:04:33.220
Exactly.
00:04:33.220 --> 00:04:33.900
Exactly.
00:04:33.900 --> 00:04:37.760
But yeah, so, but we've released a product.
00:04:37.760 --> 00:04:38.900
It's in private beta.
00:04:38.900 --> 00:04:41.040
So talk Python to me, listeners.
00:04:41.040 --> 00:04:43.280
You should definitely be part of the private beta.
00:04:43.280 --> 00:04:46.180
I'm sure there'll be some contact information.
00:04:46.180 --> 00:04:47.280
It's poppily.com.
00:04:47.280 --> 00:04:52.260
You can go request an invite or just email me or at me on Twitter or something, and we'll get you in the private beta.
00:04:52.260 --> 00:04:55.680
Yeah, and we're releasing publicly early next year.
00:04:55.680 --> 00:04:56.560
So it's super fun.
00:04:56.560 --> 00:04:57.400
We're having a really good time.
00:04:57.400 --> 00:04:58.340
That's great.
00:04:58.440 --> 00:05:00.820
You guys are actually using Python quite a bit there, right?
00:05:00.820 --> 00:05:02.080
Oh, yeah.
00:05:02.080 --> 00:05:03.020
Up and down the stack.
00:05:03.020 --> 00:05:03.680
It's all Python.
00:05:04.180 --> 00:05:12.120
So some of the data people out there might know that there's another couple languages that do some data science or that people use to do data science.
00:05:12.120 --> 00:05:13.280
One is called R.
00:05:13.280 --> 00:05:15.560
People use a language called Scala.
00:05:15.560 --> 00:05:23.920
None of them live up to the awesome power and flexibility of Python, which is why we use it in almost everything we do.
00:05:23.920 --> 00:05:25.160
From the web app that people interact with.
00:05:25.160 --> 00:05:27.660
The web app that people interact with when they're actually using the system.
00:05:27.660 --> 00:05:28.760
All of that's in Python.
00:05:28.760 --> 00:05:30.480
It's actually a Django app.
00:05:30.480 --> 00:05:31.880
I hadn't coded in Django for a while.
00:05:31.880 --> 00:05:32.500
It was super fun.
00:05:32.500 --> 00:05:41.800
And then the back end uses SciPy and the whole SciPy stack for some of the machine learning and data processing stuff that we're doing on the back end.
00:05:42.180 --> 00:05:45.000
So we are a Python shop all the way.
00:05:45.000 --> 00:05:47.480
That sounds really fun to be putting all that together.
00:05:47.480 --> 00:05:48.880
I'm sure you guys are liking it.
00:05:48.880 --> 00:05:50.200
Yeah.
00:05:50.200 --> 00:06:00.960
It's actually the coolest thing about Python from my perspective is that you can do kind of complicated scientific computing and stats and then plug it right into the web app that you'd already built because the language is so flexible.
00:06:00.960 --> 00:06:04.080
So it's been fun.
00:06:04.800 --> 00:06:04.980
Yeah.
00:06:04.980 --> 00:06:05.440
Very cool.
00:06:05.440 --> 00:06:11.120
So I'll put a link in the show notes, but definitely if you guys, if that sounds interesting, check out poppily.com.
00:06:11.120 --> 00:06:11.520
Is that right?
00:06:11.520 --> 00:06:12.000
Yeah.
00:06:12.000 --> 00:06:12.280
Yeah.
00:06:12.280 --> 00:06:12.600
That's it.
00:06:12.600 --> 00:06:13.300
Poppily.com.
00:06:13.300 --> 00:06:17.500
We thought about poppily.io, but poppily.io, it was just weird.
00:06:17.500 --> 00:06:18.360
It was too much.
00:06:18.360 --> 00:06:20.500
Get one of those Libyan domains.
00:06:20.500 --> 00:06:22.620
Those are always good for the startups.
00:06:22.620 --> 00:06:23.580
The LOIs.
00:06:23.580 --> 00:06:24.980
Yeah.
00:06:24.980 --> 00:06:25.580
Yeah.
00:06:25.580 --> 00:06:26.060
Poppily.
00:06:26.060 --> 00:06:27.020
Poppily.
00:06:27.020 --> 00:06:27.640
Would have been the best.
00:06:27.640 --> 00:06:28.120
Yeah.
00:06:28.120 --> 00:06:29.060
Missed opportunity.
00:06:29.060 --> 00:06:29.980
Missed opportunity.
00:06:29.980 --> 00:06:33.360
Well, you can always change the name if you really have to.
00:06:33.360 --> 00:06:34.580
All right.
00:06:34.600 --> 00:06:36.580
So you want to talk about this year?
00:06:36.580 --> 00:06:43.720
I mean, this show's going to come out on Talk Python on the 29th, and I suspect around the same time on Partially Derivative.
00:06:43.720 --> 00:06:52.800
It's perfect, like right at the end of the year, to talk about sort of what has happened in Python world intersected with data science, I guess.
00:06:52.800 --> 00:06:54.100
Yeah, absolutely.
00:06:54.100 --> 00:06:55.640
I mean, it's been a big year.
00:06:55.640 --> 00:06:58.000
A big year for Python and a big year for data science.
00:06:58.880 --> 00:07:05.200
The first pick, maybe not the most important, is probably most relevant to people while they're listening to this show.
00:07:05.200 --> 00:07:07.940
Like, if it comes out on the 29th, you know, you got some vacation.
00:07:07.940 --> 00:07:11.460
Maybe you'll pick it up around the 31st or the 1st.
00:07:11.460 --> 00:07:14.540
That's typically when we make our New Year's resolutions, right?
00:07:15.360 --> 00:07:15.920
It is.
00:07:15.920 --> 00:07:16.440
It is.
00:07:16.440 --> 00:07:20.480
And the first story is all about how you're pretty much going to fail at this.
00:07:20.480 --> 00:07:23.240
Don't get your hopes up, right?
00:07:24.020 --> 00:07:25.080
I know.
00:07:25.080 --> 00:07:26.480
The numbers are.
00:07:26.480 --> 00:07:32.260
The probability is that you're not going to stick with that New Year's resolution, which is funny because I think it's something that we all know.
00:07:32.260 --> 00:07:33.880
But this is actually from last year.
00:07:33.880 --> 00:07:36.720
Mona Chalabi, who's not at 538 anymore.
00:07:36.720 --> 00:07:39.520
She's doing data journalism for The Guardian.
00:07:39.520 --> 00:07:43.600
But she was at 538 at the time.
00:07:43.980 --> 00:07:53.520
And this was actually kind of like a larger story of the last year that I think kind of data journalism also really came into the, you know, kind of came into the public mindset.
00:07:53.520 --> 00:08:01.640
And she did a really interesting piece where she broke down the stats of how often people fail at their New Year's resolutions and like how long they keep them.
00:08:01.640 --> 00:08:07.240
And I guess it's something like 70 odd percent fail within the first two weeks.
00:08:07.240 --> 00:08:10.740
I'm going to change my life.
00:08:10.840 --> 00:08:12.320
I promise this year will be different.
00:08:12.320 --> 00:08:13.960
Oh, maybe.
00:08:13.960 --> 00:08:15.940
But it's a Tuesday.
00:08:15.940 --> 00:08:18.600
And, you know, my friends are going out or whatever, right?
00:08:18.600 --> 00:08:20.600
Yeah, yeah, totally.
00:08:20.600 --> 00:08:23.760
There's a lot of, I like, I really like the aspiration of New Year's resolutions.
00:08:23.760 --> 00:08:29.840
And I'm just not going to think about the cold, hard reality of their eventual failure until later this month.
00:08:29.840 --> 00:08:30.600
Yeah, exactly.
00:08:30.600 --> 00:08:31.900
You're going to wait at least two weeks.
00:08:31.900 --> 00:08:32.540
Exactly.
00:08:32.540 --> 00:08:32.960
Yeah.
00:08:32.960 --> 00:08:38.460
So some of the most common ones, well, the most common by quite a measure was lose weight.
00:08:38.460 --> 00:08:40.700
And closely related to that was exercise more.
00:08:40.700 --> 00:08:52.180
And then the third one really, in terms of popularity, really puts a sort of a challenge on being able to determine whether or not you've achieved it, which is to be a better person.
00:08:52.180 --> 00:08:54.620
How do you analytically answer that, right?
00:08:54.620 --> 00:08:56.340
That's true.
00:08:56.340 --> 00:08:57.440
It's tough.
00:08:57.440 --> 00:08:57.840
It's tough.
00:08:57.920 --> 00:09:01.180
I mean, I guess you probably could say there's a little bit of wiggle room.
00:09:01.180 --> 00:09:06.760
If 70% of people failed at that resolution, that's actually really worrying.
00:09:06.760 --> 00:09:10.560
I need to be just an incrementally better person.
00:09:10.560 --> 00:09:12.720
They're like, no, I tried, but I'm still a jerk.
00:09:12.720 --> 00:09:15.860
No, I yelled at the neighbor again and kicked over.
00:09:15.860 --> 00:09:16.560
Yeah, exactly.
00:09:18.320 --> 00:09:18.720
Nice.
00:09:18.720 --> 00:09:22.700
Oh, well.
00:09:22.700 --> 00:09:23.160
Oh, well.
00:09:23.160 --> 00:09:26.660
So you and I are both big fans of podcasts.
00:09:26.660 --> 00:09:30.660
We listen to a bunch and obviously we produce some that we're very passionate about.
00:09:30.660 --> 00:09:38.740
And the next one, the next item is actually about the most popular podcast of all time, something called Serial.
00:09:39.460 --> 00:09:47.740
Yeah, in fact, I think it was the only thing in 2015 that was more popular than Talk Python To Me and data science.
00:09:47.740 --> 00:09:48.520
The only thing.
00:09:48.520 --> 00:09:53.320
Everything else, you know, sort of pales in comparison to these two giants of media dominance.
00:09:53.320 --> 00:09:55.240
But of course, there is Serial.
00:09:55.240 --> 00:10:06.400
Yeah, Serial is, if you guys haven't heard of it, it's a podcast that goes back and looks at a person accused, maybe convicted of murder.
00:10:06.400 --> 00:10:07.000
I can't remember.
00:10:07.000 --> 00:10:09.020
No, he was convicted.
00:10:09.340 --> 00:10:09.700
Convicted.
00:10:09.700 --> 00:10:10.440
Yeah, that's what I thought.
00:10:10.440 --> 00:10:18.220
And goes through this high school guy and sort of rehashes, reevaluates the evidence, redoes the interviews.
00:10:18.220 --> 00:10:26.700
And it's like an investigative journalism look, but through the eyes of a podcast rather than maybe through like Time magazine or whatever.
00:10:26.700 --> 00:10:30.060
And it was downloaded something like five million times a week.
00:10:30.060 --> 00:10:33.320
It completely broke all the records of all time.
00:10:33.320 --> 00:10:35.860
Yeah, it was pretty amazing.
00:10:35.860 --> 00:10:39.220
I mean, and by the way, spoiler alert, the dude is totally...
00:10:39.220 --> 00:10:39.860
Totally guilty.
00:10:39.860 --> 00:10:40.840
I'm just...
00:10:40.840 --> 00:10:43.380
I mean, this is not really a spoiler because that's not how the show ends.
00:10:43.380 --> 00:10:45.720
But this is going to be cool.
00:10:45.720 --> 00:10:51.460
This is going to be really divisive for your listeners because half, I think, are going to write angry emails and the other half are going to be like, totally.
00:10:52.360 --> 00:10:53.600
So that's where I stand.
00:10:53.600 --> 00:10:54.520
I feel like I just...
00:10:54.520 --> 00:10:57.640
I feel the responsibility to get that off my chest.
00:10:57.640 --> 00:10:59.500
No, I can't.
00:10:59.500 --> 00:10:59.920
I hash it.
00:10:59.920 --> 00:11:14.500
Well, this second item, also at FiveThirtyEight.com, sort of a journey of some data sciences folks to actually go through and try to apply data science to this journalism to answer statistically or, you know, sort of through data science.
00:11:14.940 --> 00:11:16.200
Is he guilty or not?
00:11:16.200 --> 00:11:16.560
Yeah.
00:11:16.560 --> 00:11:17.120
Yeah.
00:11:17.120 --> 00:11:21.400
And this was actually interesting because it's not really something that can be quantified.
00:11:21.400 --> 00:11:36.400
In fact, that was really a big theme in the show was that all of the information that we had at the time and that we have now to assess whether or not this man was guilty of the crime that they, you know, they think that he committed this murder is really suspect.
00:11:37.040 --> 00:11:45.120
It's really not – it's hard to say conclusively what did and didn't happen because it relied so much on, you know, personal accounts of the events of the day.
00:11:45.120 --> 00:11:50.560
That said, there was some information that they could point to that definitely happened.
00:11:50.560 --> 00:12:03.860
And the big argument that everybody who believes that he's guilty was making was that none of these – none of the events in particular, like, made it certain that he was guilty.
00:12:03.860 --> 00:12:07.720
But when looked at in the aggregate, then that was really unlikely.
00:12:07.720 --> 00:12:09.660
That was sort of the intuition that everybody had.
00:12:09.660 --> 00:12:23.120
But the interesting part was that the – so, 538 interviewed a couple people who are data scientists and they went through kind of a Bayesian process for, you know, assessing the likelihood that each of the events could happen in concert.
00:12:23.120 --> 00:12:26.120
Like, whether or not he basically just had, like, super bad luck.
00:12:26.520 --> 00:12:34.680
And it actually, you know, when you look at this as something called a multiple testing problem, that's a way that you can test the hypothesis in lots of different ways.
00:12:34.680 --> 00:12:43.180
And looking at it through that lens makes it seem a little bit more probable than you might first assume that all of these different things could happen to him.
00:12:43.180 --> 00:12:46.540
So, basically, like, he asked the victim of the crime for a ride.
00:12:46.540 --> 00:12:52.160
He lent his car and his cell phone to somebody else who was also accused of the murder.
00:12:52.380 --> 00:12:59.580
And then his phone was in a location that was in, like, you know, within, like, a small distance from where the body was found.
00:12:59.580 --> 00:13:09.040
And then his cell phone records seemed to corroborate with a bunch of other, like, of the prosecution's testimony about how he totally – so, basically, like, there was, like, four or five things that made it, like, dude, that's impossible.
00:13:09.040 --> 00:13:15.100
Like, if all those things are true, you were definitely guilty, even though none of them as an individual piece of information is super damning.
00:13:15.100 --> 00:13:20.520
But it turns out, according to these data scientists, that, well, you know, maybe we should give this a second look.
00:13:20.580 --> 00:13:23.540
It's actually not that unlikely that he could have had that much bad luck.
00:13:23.540 --> 00:13:24.980
So, we'll see.
00:13:24.980 --> 00:13:38.080
Yeah, that's – I think it's really interesting to take, you know, a hard science like data science that's working with numbers and apply it to something soft like interviews and likelihood that someone's telling the truth and these kinds of things.
00:13:38.080 --> 00:13:41.440
So, I think even in that regard alone, it's really interesting.
00:13:42.100 --> 00:13:42.660
Yeah, totally.
00:13:42.660 --> 00:13:44.340
I mean, ultimately, it was pretty subjective.
00:13:44.340 --> 00:13:49.680
But it was – I mean, and it's hard because this was – these are, like, real people's lives that we're talking about.
00:13:49.680 --> 00:13:50.580
It was a true story.
00:13:50.580 --> 00:13:52.040
You know, it's not like a murder mystery.
00:13:52.040 --> 00:13:56.840
But it was really hard not to get into it and take sides and think about it like, you know, like a whodunit.
00:13:56.840 --> 00:14:02.820
So, you know, apologies to all involved that we're talking about this in such an insensitive way.
00:14:03.300 --> 00:14:03.900
Yeah, absolutely.
00:14:03.900 --> 00:14:08.140
It's definitely a harsh reality that something bad happened to somebody.
00:14:08.140 --> 00:14:09.260
They're having a second season.
00:14:09.260 --> 00:14:10.200
I haven't listened to it yet.
00:14:10.200 --> 00:14:11.540
What's the story of the second season?
00:14:11.540 --> 00:14:11.920
Do you know?
00:14:11.920 --> 00:14:13.140
Just came out.
00:14:13.140 --> 00:14:13.620
No.
00:14:13.620 --> 00:14:14.640
Yeah.
00:14:14.640 --> 00:14:15.520
No, I don't know.
00:14:15.520 --> 00:14:25.220
I think they're probably, you know, trying to keep up with the times because, you know, if you don't – as we know, you got to keep putting out content or people get bored.
00:14:25.220 --> 00:14:30.680
So, Serial 2 – but I guess all I know is that it's probably – I think it's not about the same guy.
00:14:31.040 --> 00:14:37.480
So, if you're sick of hearing about this guy's story, then you're in luck because it's about – I'm assuming another unsolved murder.
00:14:37.480 --> 00:14:38.800
Yeah, it's got to be.
00:14:38.800 --> 00:14:39.320
It's got to be.
00:14:39.320 --> 00:14:40.420
All right.
00:14:40.420 --> 00:14:47.440
Moving on to the next item is something very near and dear to the Talk Python listeners' hearts, I'm sure.
00:14:47.440 --> 00:14:51.580
And that's Jupyter and iPython and iPython Notebooks.
00:14:51.580 --> 00:14:52.980
Yeah.
00:14:52.980 --> 00:14:53.840
Yeah, absolutely.
00:14:53.840 --> 00:14:56.060
And they've had a huge year.
00:14:56.060 --> 00:14:56.660
Yeah.
00:14:56.660 --> 00:14:58.300
So, I think – yeah.
00:14:58.480 --> 00:15:02.520
And so, I don't – I'm assuming a lot of your audience will be pretty familiar with iPython and Jupyter.
00:15:02.520 --> 00:15:08.140
Although, that said, I guess there's like two camps of Python developers, I feel like.
00:15:08.140 --> 00:15:13.880
Like sort of web developers and software engineers and then kind of data and stats folks.
00:15:14.120 --> 00:15:21.980
Did you come at Python from a computer science perspective or did you have to find some language to do your specialty and kind of grow into programming?
00:15:21.980 --> 00:15:27.720
I suspect that that second category is very well familiar with the iPython stuff.
00:15:27.780 --> 00:15:29.500
So, maybe I should just tell everyone.
00:15:29.500 --> 00:15:34.560
iPython Notebooks are these sort of interactive documents.
00:15:34.560 --> 00:15:42.440
You can load them up as web pages and you can write a little bit of Python code and then you can actually execute them real time right there.
00:15:42.440 --> 00:15:48.640
Like so, you could pull some data from a database and then do some sort of science on it and a graph pops up.
00:15:48.720 --> 00:15:52.840
And then you write a little bit more code and another bit of data pops up.
00:15:52.840 --> 00:15:55.460
And these things are sort of live research documents.
00:15:55.460 --> 00:15:56.240
Very powerful.
00:15:56.240 --> 00:16:04.360
And this has been sort of generalized out of the Python world through this Jupyter project to apply to many different programming languages.
00:16:05.140 --> 00:16:10.920
So, this Jupyter project is an open source project run by Fernando Perez and some other guys.
00:16:10.920 --> 00:16:13.280
I'm actually working on having them on the show shortly.
00:16:13.280 --> 00:16:15.460
So, a couple of people have asked if they can be on the show.
00:16:15.460 --> 00:16:19.380
And yes, we'll have somebody from iPython and Jupyter soon.
00:16:19.380 --> 00:16:25.620
But these guys are an open source project that has just received $6 million in funding.
00:16:25.620 --> 00:16:28.320
Yeah, which is kind of amazing.
00:16:28.320 --> 00:16:33.500
I mean, it's a massive amount of funding for a project like this that's effectively a developer tool.
00:16:33.500 --> 00:16:35.660
But it's so useful.
00:16:35.660 --> 00:16:43.860
I think anybody – I was a little bit skeptical of it at first because I came at Python first from more of a computer science and software engineering background.
00:16:43.860 --> 00:16:46.000
And it was later that I got into data stuff.
00:16:46.000 --> 00:16:57.880
But to have something that you can basically record all of your actions – because when you're doing data projects, so often you're like, you need to explain – you get to a number or you get to a chart or you get to a discovery or whatever.
00:16:57.880 --> 00:17:01.320
And then you need to communicate why that matters to somebody else.
00:17:01.320 --> 00:17:04.220
And it only really matters if you can give the context.
00:17:04.220 --> 00:17:09.580
So, well, like first I took this slice of the data and then I manipulated it in this way.
00:17:09.580 --> 00:17:11.220
And then I extracted these features.
00:17:11.220 --> 00:17:15.260
And then once I had those features, I decided to clean some of them up by doing X, Y, and Z.
00:17:15.260 --> 00:17:19.340
And then once I'd done all of those things, obviously, if you look at this chart, it's really meaningful.
00:17:19.340 --> 00:17:22.660
But without those steps up to that, it's like, that's awesome.
00:17:22.660 --> 00:17:24.340
All I can see is like three bars.
00:17:25.420 --> 00:17:30.180
To be able to play that back for somebody and capture it is really awesome.
00:17:30.180 --> 00:17:37.120
And it's nice too when you're not a great coder, which I'm not, and you have to step through and go, wait, why didn't that work?
00:17:37.120 --> 00:17:37.800
Let me run it again.
00:17:37.800 --> 00:17:38.540
Why didn't that work?
00:17:38.540 --> 00:17:39.180
Let me run it again.
00:17:39.180 --> 00:17:40.880
Yeah, yeah.
00:17:40.880 --> 00:17:42.260
They're very cool.
00:17:42.260 --> 00:17:54.560
If I think of writing sort of science-based academic papers, it seems crazy to not do them as something like this, where rather than just saying, oh, I did something on my own.
00:17:54.560 --> 00:17:55.220
Here's a chart.
00:17:55.220 --> 00:17:55.940
Believe it, right?
00:17:55.940 --> 00:17:57.420
Here is the actual code.
00:17:57.420 --> 00:17:58.220
And here is the data.
00:17:58.220 --> 00:18:00.740
And you can just have the whole thing and run it if you like.
00:18:00.740 --> 00:18:01.020
Right?
00:18:01.020 --> 00:18:01.760
That's amazing.
00:18:02.500 --> 00:18:02.940
Yeah, totally.
00:18:02.940 --> 00:18:10.940
To be able to reproduce research by literally running the code that led to the discoveries that informed the paper is huge.
00:18:11.880 --> 00:18:19.060
I mean, Chris, my co-founder and the co-host on Partially Derivative, went and got a PhD.
00:18:19.060 --> 00:18:22.380
And he talked about that all the time.
00:18:22.380 --> 00:18:29.740
Often in postgraduate work, you get assigned a task to reproduce the research in somebody else's paper.
00:18:30.220 --> 00:18:34.280
And he was always stunned at how difficult that was.
00:18:34.280 --> 00:18:42.240
Because even when you had access to the raw data, trying to work with it in such a way that produced the same results was just, you know, it's just really rough.
00:18:42.240 --> 00:18:49.100
And so being able to capture that process where like every little decision that you made to manipulate the data in a particular way.
00:18:49.100 --> 00:18:56.700
And when I say manipulate in this context, I mean like a legitimate transformation of the data, not like a shady manipulation of the data.
00:18:57.980 --> 00:19:06.960
Not like witness tampering type manipulate, but like we're going to make some assumptions about the underlying physics or statistics and then get a better answer.
00:19:06.960 --> 00:19:07.160
Yeah.
00:19:07.160 --> 00:19:09.160
Yeah, totally.
00:19:09.160 --> 00:19:12.840
Well, and because so much of this kind of work is like it's a little bit about it.
00:19:12.840 --> 00:19:14.600
There's some it takes some creativity.
00:19:14.600 --> 00:19:22.860
It takes a little bit of intuition, especially if you're working with like natural language and you're trying to extract some you're trying to like make sense of unstructured data.
00:19:22.860 --> 00:19:27.540
You know, you make a lot of little choices on the way and those do impact the results that you see.
00:19:27.740 --> 00:19:32.740
And that's why being able to reproduce it is so important and for everybody to understand the assumptions that you made.
00:19:32.740 --> 00:19:37.600
And so I think I mean, I can only assume that was part of why they received this like massive bucket of funding.
00:19:37.600 --> 00:19:39.480
It's a super popular product.
00:19:39.480 --> 00:19:43.960
So, yeah, it's going to definitely be a really important foundation of science, period.
00:19:44.400 --> 00:19:53.280
But if you just think of any open source project, like what other open source project do you know that's not got a company behind it that somebody gave six million dollars like this?
00:19:53.280 --> 00:19:54.100
This is a big deal.
00:20:04.540 --> 00:20:07.560
This episode is brought to you by Hired.
00:20:07.560 --> 00:20:14.020
Hired is a two sided, curated marketplace that connects the world's knowledge workers to the best opportunities.
00:20:14.020 --> 00:20:23.200
Each offer you receive has salary and equity presented right up front and you can view the offers to accept or reject them before you even talk to the company.
00:20:23.200 --> 00:20:29.560
Typically, candidates receive five or more offers in just the first week and there are no obligations ever.
00:20:29.560 --> 00:20:31.660
Sounds pretty awesome, doesn't it?
00:20:31.960 --> 00:20:33.700
Well, did I mention there's a signing bonus?
00:20:33.700 --> 00:20:37.780
Everyone who accepts a job from Hired gets a $2,000 signing bonus.
00:20:37.780 --> 00:20:42.120
And as Talk Python listeners, it gets way sweeter.
00:20:42.120 --> 00:20:49.700
Use the link Hired.com slash Talk Python To Me and Hired will double the signing bonus to $4,000.
00:20:49.700 --> 00:20:51.420
Opportunity's knocking.
00:20:51.420 --> 00:20:55.020
Visit Hired.com slash Talk Python To Me and answer the call.
00:20:55.020 --> 00:21:08.900
Okay, so number four on our list is artificial intelligence.
00:21:08.900 --> 00:21:17.760
There have been some cool shows about artificial intelligence like X Machina and that actually happened to feature a little Python code in the show as well, which is cool.
00:21:18.220 --> 00:21:19.820
But people are freaking out about it.
00:21:19.820 --> 00:21:23.120
Yeah, this is this is kind of a new thing this year.
00:21:23.120 --> 00:21:24.060
I don't know.
00:21:24.060 --> 00:21:29.760
Maybe it's because machine learning and AI are becoming more a part of like mainstream conversations.
00:21:29.760 --> 00:21:38.080
Or like you said, there was an entire movie made about it where they referenced the Turing test specifically, like in a major Hollywood movie that blew my mind.
00:21:39.280 --> 00:21:43.980
Because that's pretty nerdy stuff in normal circumstances, right?
00:21:43.980 --> 00:21:44.760
It's super cool.
00:21:44.760 --> 00:21:58.160
But yeah, and now all of a sudden you have these super prominent leaders of the technology community coming out like for and against, like speculatively against the idea of AI because, you know, the matrix or whatever.
00:21:58.160 --> 00:21:59.140
So I don't know.
00:21:59.140 --> 00:22:02.180
It was really interesting to see that debate happening in public.
00:22:02.180 --> 00:22:04.540
I don't know if you I don't know if you track that very much.
00:22:04.540 --> 00:22:09.520
I mean, like Bill Gates was warning us and Stephen Hawking warning us about artificial intelligence.
00:22:09.520 --> 00:22:19.440
And then, of course, other people on the other side giving really nuanced defenses of the way that artificial intelligence helps humans by making better decisions.
00:22:19.440 --> 00:22:21.660
Or it was it's super fascinating conversation.
00:22:21.660 --> 00:22:23.260
Yeah, it's very fascinating.
00:22:23.260 --> 00:22:29.840
The person that came out that sort of in my mind carried the most weight, honestly, was Elon Musk.
00:22:30.540 --> 00:22:30.780
Yeah.
00:22:30.780 --> 00:22:31.720
Yeah.
00:22:31.720 --> 00:22:36.600
Well, I mean, everything that Elon Musk is always when Elon Musk speaks, we listen.
00:22:36.600 --> 00:22:37.620
That's right.
00:22:37.620 --> 00:22:40.600
Because, I mean, Bill Gates, I'm actually a fan of his.
00:22:40.600 --> 00:22:41.960
I think he did some really cool stuff.
00:22:41.960 --> 00:22:48.640
But I kind of feel like he's, you know, has a certain worldview that's sort of already here.
00:22:49.640 --> 00:22:54.180
And Stephen Hawking has some amazing views of the universe.
00:22:54.180 --> 00:23:00.760
But at the same time, I'm not entirely sure how practical his actual interaction with AI and programming is.
00:23:00.760 --> 00:23:05.040
But Elon Musk says, I'm going to build an electric car that's like amazing.
00:23:05.040 --> 00:23:05.680
He built it.
00:23:05.680 --> 00:23:07.860
I'm going to build things that go to space.
00:23:07.860 --> 00:23:08.840
And somehow he does that.
00:23:08.840 --> 00:23:14.000
I mean, he could actually build AIs if he wanted, if anybody can, I would say.
00:23:14.580 --> 00:23:29.040
I think that that's the only way that he's really going to be a Bond villain, because at the moment he's got he's got like super cool, you know, international technology companies that are doing amazing things.
00:23:29.040 --> 00:23:30.540
Like you said, going to space.
00:23:31.320 --> 00:23:34.860
But it's it's not quite supervillain status yet.
00:23:34.860 --> 00:23:41.660
And I think if he builds a robot that like seamlessly works its way into society and obviously initially for good.
00:23:41.660 --> 00:23:43.420
But, you know, it gets out of control.
00:23:43.420 --> 00:23:47.500
That's where that's where the plot of the movie really starts to get thick.
00:23:47.500 --> 00:23:48.560
I'm excited about this.
00:23:48.560 --> 00:23:49.260
Yes, absolutely.
00:23:49.260 --> 00:23:53.960
The AIs had to be created to man the supercharging stations up and down the West Coast.
00:23:53.960 --> 00:23:55.020
And then it just went all wrong.
00:23:55.020 --> 00:23:56.280
They got in the cars and spread.
00:23:56.280 --> 00:23:58.380
Exactly.
00:23:59.380 --> 00:24:02.860
They decided that they deserved more than the menial tasks that they've been assigned.
00:24:02.860 --> 00:24:03.660
Yeah.
00:24:03.660 --> 00:24:17.180
So we'll link to a really cool article from this project that was sort of getting respected leaders in the AI space to sign sort of a pledge to proceed with caution.
00:24:17.180 --> 00:24:25.020
But one of the follow up articles that we'll also list that I really liked was this thing about Mario as in like Super Mario Brothers.
00:24:25.020 --> 00:24:26.360
That was really cool.
00:24:26.360 --> 00:24:27.140
Yeah.
00:24:27.140 --> 00:24:28.000
Yeah, it's pretty funny.
00:24:28.000 --> 00:24:33.040
There were some researchers that like basically made Mario sentient.
00:24:33.040 --> 00:24:35.660
I mean, maybe not quite.
00:24:35.660 --> 00:24:36.740
That's a stretch.
00:24:36.740 --> 00:24:49.260
But they empowered Mario, the character, with his own intelligence and then like would let him loose in the original game to see how well he could defeat the Goombas and all of the other dangers of Mario characters.
00:24:49.380 --> 00:24:49.920
It was super fun.
00:24:49.920 --> 00:24:50.940
It's like on the flip side.
00:24:50.940 --> 00:24:58.920
There's these like really super accessible, fun artificial intelligence projects that aren't necessarily a threat to, you know, humanity.
00:24:59.300 --> 00:24:59.740
Yeah.
00:24:59.740 --> 00:25:04.340
Maybe it's a threat to our high scores on Mario Brothers, but not maybe humanity in large.
00:25:05.240 --> 00:25:07.940
So, you know, those guys are, they're German.
00:25:07.940 --> 00:25:11.980
They're in Tubingen at the university there, which is actually like 35 minutes away from me right now.
00:25:11.980 --> 00:25:12.380
That's cool.
00:25:13.080 --> 00:25:14.220
They made a video.
00:25:14.220 --> 00:25:14.660
Yeah.
00:25:14.660 --> 00:25:16.560
They made a video and we'll link to it.
00:25:16.560 --> 00:25:17.940
It's on Mashable.com.
00:25:17.940 --> 00:25:20.280
And they do all sorts of cool stuff.
00:25:20.280 --> 00:25:22.600
There's a bunch of different intersections.
00:25:22.600 --> 00:25:29.240
It's not just like one part of AI, but there's a lot of sort of understanding the world, understanding language, learning.
00:25:29.640 --> 00:25:36.800
And so they would do things like they would ask Mario, like they can speak to him in English and he would answer in English.
00:25:36.800 --> 00:25:40.460
It sounded a whole lot like war games.
00:25:40.460 --> 00:25:44.200
It's like, you know, that really sort of choppy text to speech, that war games computer.
00:25:44.200 --> 00:25:49.020
So it's kind of funny to hear Super Mario Brothers speak that way.
00:25:49.020 --> 00:25:54.720
But they would say things like, jump, if you jump on Goomba, he will die.
00:25:54.920 --> 00:25:56.780
And then they say, now jump on Goomba.
00:25:56.780 --> 00:25:58.320
And of course the character dies.
00:25:58.320 --> 00:25:59.700
And they say, what do you know about Goomba?
00:25:59.700 --> 00:26:01.740
He says, if I jump on him, he will certainly die.
00:26:01.740 --> 00:26:08.500
And then later they reset his mind and they tell him to go over and jump on Goomba.
00:26:08.500 --> 00:26:09.320
They don't tell anything.
00:26:09.320 --> 00:26:12.600
And then they later, first they ask him, what do you know about him?
00:26:12.600 --> 00:26:13.760
He says, I don't know anything about him.
00:26:13.760 --> 00:26:14.500
Jump on him.
00:26:14.500 --> 00:26:15.500
The guy dies.
00:26:15.500 --> 00:26:17.560
He says, now what do you know about him?
00:26:17.560 --> 00:26:20.340
He goes, I know that he may die if I jump on him.
00:26:20.340 --> 00:26:21.700
And it was really interesting.
00:26:21.700 --> 00:26:24.260
He has all like different emotions and knowledge.
00:26:24.400 --> 00:26:24.920
It's cool.
00:26:24.920 --> 00:26:25.460
Check it out.
00:26:25.460 --> 00:26:26.800
Yeah, it is super cool.
00:26:26.800 --> 00:26:31.780
Actually, that's probably the best part about the project is like Mario's sort of existential,
00:26:31.780 --> 00:26:34.140
like his existential ennui.
00:26:34.140 --> 00:26:36.980
Yeah, absolutely.
00:26:36.980 --> 00:26:43.220
So another thing that happened this year had to do with the New England Patriots.
00:26:43.220 --> 00:26:52.260
And for those of you who maybe don't follow American football super close, this was sort of a big deal.
00:26:52.260 --> 00:27:06.040
The New England Patriots, I don't really care one way or the other, but they'd kind of been seen as a team that, let's put it nicely, takes as much advantage of the situation as they can by, you know, maybe doing things they shouldn't.
00:27:06.040 --> 00:27:07.280
That was so diplomatic.
00:27:09.280 --> 00:27:18.580
And it had come to a head where around the Super Bowl time, they had actually been accused of deflating the footballs for their team.
00:27:18.580 --> 00:27:20.320
And for a while, I didn't know what that meant.
00:27:20.320 --> 00:27:23.280
Like, okay, well, maybe it hurts a little less to catch it.
00:27:23.280 --> 00:27:23.660
I don't know.
00:27:23.720 --> 00:27:28.440
But one of the things that deflated footballs let you do is hold on to them much tighter.
00:27:28.440 --> 00:27:29.260
They're not slippery.
00:27:29.260 --> 00:27:34.920
And so you won't fumble the ball and make some of these game losing mistakes or, you know, mistakes.
00:27:34.920 --> 00:27:38.520
If you can, like, change the physics so it's not a problem.
00:27:38.520 --> 00:27:40.960
Well, that's easier to solve than being better at football.
00:27:40.960 --> 00:27:46.260
So the story is actually some data science folks came and looked at that.
00:27:46.260 --> 00:27:47.820
And I don't know.
00:27:47.820 --> 00:27:50.420
What's your opinion after looking at all the charts and graphs they built?
00:27:50.960 --> 00:27:58.440
Well, I think it boils down to, in a simple way, that the New England Patriots were, like, a massive outlier.
00:27:58.440 --> 00:28:06.260
And so you're right on that if you deflate the football a little bit, apparently, it makes it easier to hold on to.
00:28:06.260 --> 00:28:12.580
And so for those of you who don't watch American football, it's a lot like rugby, if you're familiar with rugby.
00:28:13.180 --> 00:28:21.720
Or more or less, it's like you have a ball that you're holding on to and you're running through a large group of very strong men who are all trying to take that ball from you.
00:28:21.720 --> 00:28:28.400
And so you often lose it because it's a large group of really strong big men trying to take it.
00:28:28.400 --> 00:28:32.400
And pretty much anything that you have in that scenario is not going to be yours for very long.
00:28:33.540 --> 00:28:46.840
But the – and the trend overall in the league, the NFL, the league in which the Patriots play, was that the teams were having – were more successful, like, holding on to the football.
00:28:46.840 --> 00:28:49.000
They had more plays that they could run.
00:28:49.060 --> 00:28:54.740
So the ratio between the number of attempts that they made and the number of times they lost the ball was going up.
00:28:54.740 --> 00:28:57.820
So you could run more plays and fumble less in general.
00:28:57.820 --> 00:29:07.500
That said, the Patriots improved their ratio sort of exponentially more than all of the other teams in the league.
00:29:07.500 --> 00:29:13.460
And so when you see this one outlier way at the top right of a graph, it tends to go like, well, that's not right.
00:29:13.460 --> 00:29:16.340
Something doesn't make sense.
00:29:16.340 --> 00:29:18.520
Something's different about that one data point.
00:29:19.040 --> 00:29:22.800
And so, of course, that sparked some – it sparked a lot of speculation.
00:29:22.800 --> 00:29:25.280
Like, why is it that the Patriots fumble so infrequently?
00:29:25.280 --> 00:29:28.940
And, you know, where there's smoke, there's fire.
00:29:28.940 --> 00:29:31.020
And in this case, at least.
00:29:31.020 --> 00:29:37.140
And they found that the Patriots were deflating footballs just a little bit all of the time.
00:29:37.140 --> 00:29:43.860
And by doing that, they were able to maintain better control of it because it was easier to grip the ball because there was less air inside.
00:29:43.860 --> 00:29:44.880
So it was less buoyant.
00:29:44.880 --> 00:29:49.020
Which – buoyant is probably not the right word to use in this context.
00:29:49.020 --> 00:29:49.820
But whatever.
00:29:49.820 --> 00:29:50.260
Whatever.
00:29:50.260 --> 00:29:51.180
They could hold on to the ball.
00:29:51.180 --> 00:29:52.340
And so that was the whole thing.
00:29:52.340 --> 00:30:01.240
But basically, it was like the way that that was determined was through, you know, just doing some relatively simple data science.
00:30:01.240 --> 00:30:02.480
Relatively simple analysis.
00:30:02.480 --> 00:30:09.660
I mean, it's not that simple in the aggregate that the data – gathering the data was complex, connecting the dots, understanding the consequences of the things that we were learning.
00:30:09.660 --> 00:30:17.960
But by and large, it was – you know, if you actually look at the analysis that detected that outlier that I just described, it's pretty straightforward.
00:30:17.960 --> 00:30:19.260
So anyway, it's kind of cool.
00:30:19.260 --> 00:30:27.480
And it was a super fun story to follow that ultimately was followed at such depth because, oh my gosh, sports in America, that it got a little tedious.
00:30:27.720 --> 00:30:28.920
But at the beginning, it was cool.
00:30:28.920 --> 00:30:30.700
It was like a wonderful little scandal.
00:30:31.700 --> 00:30:33.140
Yeah, it was an interesting scandal.
00:30:33.140 --> 00:30:38.820
And I think it really shows the power of data science because these sports guys, they can go back and forth.
00:30:38.820 --> 00:30:40.820
They're talk radio and da-da-da-da-da.
00:30:40.820 --> 00:30:41.280
No.
00:30:41.280 --> 00:30:43.480
Look at that graph.
00:30:43.480 --> 00:30:45.020
There's something going on here.
00:30:45.020 --> 00:30:45.540
That's it.
00:30:45.960 --> 00:30:47.860
The question is what is going on.
00:30:47.860 --> 00:30:56.460
It's more likely something sort of sneaky is going on rather than they're just dramatically better than even second place, right?
00:30:56.460 --> 00:30:59.080
So very interesting use of data science there.
00:30:59.080 --> 00:31:00.320
I like that one.
00:31:00.320 --> 00:31:01.680
So speaking of –
00:31:01.680 --> 00:31:02.000
Yeah, it was super fun.
00:31:02.000 --> 00:31:02.320
Yeah.
00:31:02.320 --> 00:31:08.580
So speaking of U.S. things, the United States now has a person called the chief data scientist.
00:31:08.580 --> 00:31:09.540
That's pretty awesome.
00:31:09.540 --> 00:31:10.800
Yeah.
00:31:10.800 --> 00:31:12.360
It's kind of amazing, right?
00:31:12.360 --> 00:31:19.100
Like we kind of went from like data science like, whoa, okay, that's something like a bunch of geeks do.
00:31:19.100 --> 00:31:25.100
And now there's like somebody who has a title like over the whole domain of the entire United States doing data science.
00:31:25.100 --> 00:31:26.040
They're chief.
00:31:26.040 --> 00:31:29.260
They're the chief data scientist.
00:31:29.260 --> 00:31:29.520
Yeah.
00:31:29.520 --> 00:31:32.760
I mean that's like almost like a politician has got data science.
00:31:32.760 --> 00:31:34.040
This is crazy.
00:31:34.040 --> 00:31:36.880
No, I think it's a really positive move.
00:31:36.880 --> 00:31:40.060
I mean you think of places that have lots of data.
00:31:40.060 --> 00:31:41.080
Sports.
00:31:41.080 --> 00:31:41.940
We were just talking about that.
00:31:42.080 --> 00:31:42.980
They've got a lot of data.
00:31:42.980 --> 00:31:44.080
CERN.
00:31:44.080 --> 00:31:46.440
At the large Hadron Collider, those guys generate a lot of data.
00:31:46.440 --> 00:31:51.480
But the United States, we have so many different things that we track about people.
00:31:51.480 --> 00:31:54.660
And the answers to those questions really matter, right?
00:31:54.660 --> 00:31:56.740
We make policy based on those numbers.
00:31:56.740 --> 00:31:59.340
So having somebody in charge of doing that right makes a lot of sense.
00:31:59.340 --> 00:32:00.120
Yeah.
00:32:00.120 --> 00:32:08.100
And actually, you know what's super cool about the stuff that DJ Patil, he's the guy who is the chief data scientist, the first one.
00:32:08.800 --> 00:32:12.340
The stuff that he's focused on is really awesome.
00:32:12.340 --> 00:32:18.400
Like a big initiative of his is that he wants to open up a lot of the data that the U.S. government collects.
00:32:18.400 --> 00:32:36.300
So to keep the government agencies more transparent than they've been before and just to advocate for the open data movement in general and get the general public interacting with the data that these agencies produce in a way, almost like as a means of civic engagement.
00:32:36.560 --> 00:32:37.520
It's really awesome.
00:32:37.520 --> 00:32:49.040
And so this whole idea that like you're the sort of the general public's relationship with government can be more modern, can be more technology driven, can be more part of the 21st century.
00:32:49.040 --> 00:32:50.300
It's really cool.
00:32:50.300 --> 00:32:53.400
So the open data projects in particular have been really fascinating.
00:32:53.400 --> 00:32:57.420
And then I know healthcare has been a big focus of DJ's office.
00:32:57.420 --> 00:33:05.360
They've been really looking at, again, how to encourage the public to improve the way that the healthcare system works by investigating the data that's being made available.
00:33:05.940 --> 00:33:14.880
And by making the data that we do produce easier to transport, easier to access, easier to sort of combine and investigate.
00:33:14.880 --> 00:33:25.200
It's really I think it's it's making data accessible and kind of top of mind for an entire generation of kind of, you know, early career Americans.
00:33:25.200 --> 00:33:26.820
It's it's it's really fascinating.
00:33:27.720 --> 00:33:28.180
Yeah, that's cool.
00:33:28.180 --> 00:33:34.600
And, you know, maybe the United States government themselves has access to things that you wouldn't otherwise share.
00:33:34.600 --> 00:33:34.940
Right.
00:33:34.940 --> 00:33:36.300
Like you mentioned health data.
00:33:36.300 --> 00:33:44.800
I think one of the problems with analyzing health data in the large is people don't want to just give away every single thing about themselves for good reason.
00:33:44.800 --> 00:33:47.540
And so it's hard to to talk broadly about that.
00:33:47.540 --> 00:33:47.860
Right.
00:33:47.860 --> 00:33:52.460
But maybe there's extra data in there somewhere that they can, you know, help make people healthy or something.
00:33:52.460 --> 00:33:52.880
That's cool.
00:33:52.880 --> 00:33:54.160
Yeah, totally.
00:33:54.300 --> 00:34:05.900
And it's in general, I think part of this whole idea of like quantified social science, I think for the longest time, science, social science that's done about, you know, the behavior of populations has been kind of anecdotal.
00:34:05.900 --> 00:34:19.140
A lot of, you know, a lot of people doing qualitative research where they're, you know, using their intuition and using their, you know, their knowledge to connect the dots and tell a good story.
00:34:19.500 --> 00:34:24.100
And I'm sorry, that probably sounds super like diminutive of a lot of social science research.
00:34:24.100 --> 00:34:26.180
And I only kind of mean it to be.
00:34:26.180 --> 00:34:38.040
But there's a move towards saying, hey, you know, if you can't back up your statements with data, even if you have to, you know, be creative about how that data is collected or be creative about how that data is interpreted.
00:34:38.040 --> 00:34:39.140
I mean, that's fine.
00:34:39.140 --> 00:34:41.960
But if you're not backing it up with any data, then it doesn't really mean anything.
00:34:42.600 --> 00:34:46.720
And I think that that's, it's very encouraging for people of my persuasion.
00:34:46.720 --> 00:34:48.580
Yeah, I totally agree.
00:34:48.580 --> 00:34:52.540
The other thing that's cool, you talked about the open data project.
00:34:52.540 --> 00:35:00.540
I mean, when you live in a democracy, you would expect that the data, your government, you're supposed to be sort of the boss of the government.
00:35:00.540 --> 00:35:02.740
You should be able to have access to those things.
00:35:03.380 --> 00:35:04.960
And so that's a really positive trend.
00:35:04.960 --> 00:35:10.340
And also on that note, I want to recommend a video by Catherine Devlin.
00:35:11.220 --> 00:35:20.040
She did the keynote at Pi Ohio this year, and she works for some government agency, sort of a, let's see if I remember it right.
00:35:20.040 --> 00:35:30.520
She works for a group of programmers within the U.S. government that are basically an open source wing of this programming group.
00:35:30.520 --> 00:35:35.620
And so they'll go into other places and say, we will help you with this project, but only if we get open source the results.
00:35:35.620 --> 00:35:39.940
And they're trying to spread open source within the U.S. government as well.
00:35:39.940 --> 00:35:41.640
So let's tie together nicely.
00:35:41.640 --> 00:35:43.340
Yeah, absolutely.
00:35:43.340 --> 00:35:47.740
There's everybody kind of involved with the CTO's office.
00:35:47.740 --> 00:35:50.880
Megan Smith is the current CTO of the U.S.
00:35:50.880 --> 00:36:00.340
And almost all the initiatives they're doing are, it's like a totally different way of interacting with government than I think ever existed before.
00:36:00.340 --> 00:36:05.120
So if you get a chance, definitely check out what the CTO's office is doing.
00:36:05.120 --> 00:36:07.200
There's a lot of cool ways to get involved.
00:36:07.200 --> 00:36:09.620
There's a lot of cool projects that they do.
00:36:09.660 --> 00:36:13.780
And there's a lot of interesting open source projects like you just mentioned.
00:36:13.780 --> 00:36:15.640
So, and a lot of cool data sets.
00:36:15.640 --> 00:36:21.900
And which, and by the way, some of those data sets are really great for learning if you're just kind of, you know, cutting your teeth on data science.
00:36:21.900 --> 00:36:33.800
There's a lot of really interesting things about your community and your state and the country in general that might be more fun than working with, I don't know, a data set about advertising clicks.
00:36:34.800 --> 00:36:35.740
Or whatever.
00:36:36.100 --> 00:36:53.120
This episode is brought to you by DigitalOcean.
00:36:53.580 --> 00:36:57.240
DigitalOcean offers simple cloud infrastructure built for developers.
00:36:57.320 --> 00:37:04.460
Over half a million developers deployed to DigitalOcean because it's easy to get started, flexible for scale, and just plain awesome.
00:37:05.360 --> 00:37:10.680
In fact, DigitalOcean provides key infrastructure for delivering Talk Python episodes every day.
00:37:11.020 --> 00:37:18.560
When you, or your podcast client, download an episode, it comes straight out of a custom Flask app built on DigitalOcean, and it's been bulletproof.
00:37:18.560 --> 00:37:26.800
On release days, the measured bandwidth on my single $10 a month server jumps to over 900 megabit per second for sustained periods.
00:37:26.800 --> 00:37:28.060
And there's no trouble.
00:37:28.060 --> 00:37:31.820
That's because they provide great servers on great hardware at a great price.
00:37:32.360 --> 00:37:40.360
Head on over to DigitalOcean.com today and use the promo code TALKPYTHON, all caps, no spaces, to get started with a $10 credit.
00:37:40.360 --> 00:37:50.080
Let's move on to number seven.
00:37:50.080 --> 00:37:52.340
It's going to be late December.
00:37:52.340 --> 00:37:53.660
Winter's coming, probably.
00:37:54.700 --> 00:37:55.340
Yeah, maybe.
00:37:55.340 --> 00:37:56.320
It's hard to know.
00:37:56.320 --> 00:37:58.660
What's the story of this one?
00:37:58.660 --> 00:38:14.120
So there's a handful of things on here that really I was attracted to because I just love this idea that data science and just kind of data analysis in general is becoming so much more mainstream.
00:38:14.120 --> 00:38:21.420
And so this post was about the Game of Thrones.
00:38:21.920 --> 00:38:34.300
And you may know, this is actually a shame that my co-host Chris isn't on the show right now because Chris produced a data set from A Song of Ice and Fire, the books, not necessarily the HBO TV show, Game of Thrones.
00:38:34.300 --> 00:38:39.000
Obviously, the two are related, although I hear the TV shows kind of going off the rails and moving away from the books.
00:38:39.000 --> 00:38:46.460
Anyway, so there's a lot of dying in this TV show, Game of Thrones, and the books, A Song of Ice and Fire.
00:38:46.460 --> 00:38:52.480
And my co-host made a data set of all of the battles where he tallied the number of deaths.
00:38:52.480 --> 00:38:57.080
And obviously, there's some estimation in here because the books don't go into detail all of the time.
00:38:57.080 --> 00:38:59.480
But so actually, hold on.
00:38:59.480 --> 00:39:00.120
Let me take a step back.
00:39:00.120 --> 00:39:06.140
So what I should probably tell everybody is if you're not familiar with Game of Thrones and the books, the series of books are called A Song of Ice and Fire.
00:39:06.300 --> 00:39:10.000
If you're not familiar with them, it's basically kind of like medieval fantasy type stuff.
00:39:10.780 --> 00:39:18.600
So there's, you know, dragons and there's houses and, you know, the houses fight with each other over land and there's different families that are at war with one another.
00:39:18.600 --> 00:39:20.340
Kind of the whole thing, right?
00:39:20.340 --> 00:39:23.600
And so, of course, because they fight all the time, there's a lot of dying.
00:39:24.320 --> 00:39:33.340
And if you were of a more data statsy persuasion, you might want to know, like, quantitatively, which house is the best.
00:39:33.340 --> 00:39:35.200
Like, who wins the most battles?
00:39:35.200 --> 00:39:36.540
Who kills the most soldiers?
00:39:36.540 --> 00:39:37.700
Who has the biggest army?
00:39:37.700 --> 00:39:40.820
And these are the sorts of questions that Chris's data set answers.
00:39:40.820 --> 00:39:43.060
So you should definitely go find that.
00:39:43.060 --> 00:39:45.640
I think it's just Game of Thrones battles on GitHub.
00:39:45.640 --> 00:39:47.180
If you Google for it, I bet you'll find it.
00:39:47.540 --> 00:39:56.660
But there's other work, other very interesting work that's being done about the likelihood that you'll die if you're a character in the books or the TV shows.
00:39:56.660 --> 00:39:59.820
And so I just loved it, right?
00:39:59.820 --> 00:40:01.920
Because, I mean, it's a fictional world.
00:40:01.920 --> 00:40:02.680
It's made up.
00:40:02.680 --> 00:40:06.640
The likelihood that you'll die is whether or not the author decides that you should die.
00:40:06.640 --> 00:40:09.060
Like, that's really what's going on here.
00:40:09.060 --> 00:40:21.000
But based on the sort of constructs of the world that the author created, this data scientist went through and he used – there's a whole Wikipedia clone called a wiki of ice and fire.
00:40:21.000 --> 00:40:21.920
Haha, snort.
00:40:21.920 --> 00:40:28.600
That catalogs and documents the deaths of every major character, like every major death in the book.
00:40:28.600 --> 00:40:36.320
And so he went through and basically calculated the likelihood that any given character will survive based on their characteristics.
00:40:36.320 --> 00:40:38.500
So are they from a particular family?
00:40:38.700 --> 00:40:39.580
Are they highborn?
00:40:39.580 --> 00:40:40.380
Are they lowborn?
00:40:40.380 --> 00:40:41.060
Are they a man?
00:40:41.060 --> 00:40:41.660
Are they a woman?
00:40:41.660 --> 00:40:42.440
How old are they, right?
00:40:42.440 --> 00:40:45.720
All of the things that you might say are like a feature of a character.
00:40:45.720 --> 00:40:50.720
And then once you understand the features of a character, you can calculate the likelihood that they'll die.
00:40:50.720 --> 00:40:52.480
Anyway, it's fascinating.
00:40:52.480 --> 00:40:55.700
I'm sure, you know, we'll link to the blog post so you can go check it out.
00:40:55.700 --> 00:41:00.040
But it pans out kind of like you'd expect.
00:41:00.040 --> 00:41:04.100
You have a much higher chance of dying if you're not of the noble class.
00:41:04.100 --> 00:41:06.160
I think men die more often than women.
00:41:06.160 --> 00:41:08.040
And so on and so on.
00:41:08.080 --> 00:41:08.720
You should go check it out.
00:41:08.720 --> 00:41:18.860
But really, the reason to bring it up is just to say doing data analysis about characters in a fictional universe, I think is we're at some kind of peak.
00:41:18.860 --> 00:41:22.340
We're at like peak data science awesomeness when that's a possibility.
00:41:23.420 --> 00:41:24.100
Yeah, that's really cool.
00:41:24.100 --> 00:41:30.000
And you could also run this algorithm on your favorite character and know whether or not you should get attached to them, right?
00:41:30.000 --> 00:41:31.500
Yeah.
00:41:31.500 --> 00:41:34.980
I mean, I think to be fair with these stories, the answer is probably always no.
00:41:34.980 --> 00:41:37.820
That's right.
00:41:38.780 --> 00:41:39.100
Yeah.
00:41:39.100 --> 00:41:40.580
I mean, readers of the books will know.
00:41:40.580 --> 00:41:42.420
I'm not, I'm not, this is not a spoiler or anything.
00:41:42.420 --> 00:41:43.420
I'm not giving anything away.
00:41:43.420 --> 00:41:50.200
If you like a character and you hope for them, if you have any hope that the character will survive, that's it.
00:41:50.340 --> 00:41:52.900
I think that's probably the best indicator that they're going to die.
00:41:52.900 --> 00:41:53.820
It's the kiss of death.
00:41:53.820 --> 00:41:54.040
Yeah.
00:41:54.040 --> 00:41:58.020
So speaking of dying, we're all getting older.
00:41:58.020 --> 00:42:04.180
And Microsoft has decided that machines can know really well how old we are.
00:42:04.180 --> 00:42:11.740
So if you go to how-old.net, Microsoft is using machine learning to guess how old you are.
00:42:11.740 --> 00:42:12.920
I know.
00:42:12.920 --> 00:42:15.260
Isn't that, how old did it get?
00:42:15.260 --> 00:42:15.900
Do you want to say?
00:42:15.900 --> 00:42:17.420
You don't have to reveal if you don't want to.
00:42:17.580 --> 00:42:18.600
I don't mind.
00:42:18.600 --> 00:42:24.340
So I've always kind of looked a little younger than I actually am.
00:42:24.340 --> 00:42:25.280
People are always like, what?
00:42:25.280 --> 00:42:26.100
You have kids?
00:42:26.100 --> 00:42:26.420
What?
00:42:26.420 --> 00:42:29.680
So my whole life has been kind of like this.
00:42:29.680 --> 00:42:35.420
Like it was a problem in high school because it's not cool to look younger than everybody else when there's only a four-year stretch.
00:42:35.420 --> 00:42:38.600
But when I get older, it looks better.
00:42:38.600 --> 00:42:44.220
So I uploaded a picture of myself, the one I have on my main website.
00:42:44.220 --> 00:42:45.660
It's the one you probably see on Skype right now.
00:42:46.020 --> 00:42:47.420
And it actually said I was 47.
00:42:47.420 --> 00:42:48.080
I'm like, what?
00:42:48.080 --> 00:42:49.540
I'm only 42.
00:42:49.540 --> 00:42:50.460
This is crazy.
00:42:50.460 --> 00:42:52.020
But I have my glasses on, right?
00:42:52.020 --> 00:42:54.260
And so I'm like, I'll try one without my glasses.
00:42:54.260 --> 00:42:54.940
Uploaded it.
00:42:54.940 --> 00:42:55.680
It said I'm 42.
00:42:55.680 --> 00:42:59.040
They hit it straight, right on exact.
00:42:59.040 --> 00:43:00.300
Whoa.
00:43:00.300 --> 00:43:00.980
Whoa.
00:43:00.980 --> 00:43:03.400
What does that say about the algorithm, though?
00:43:03.400 --> 00:43:05.820
That you got aged by five years just because of your glasses?
00:43:05.820 --> 00:43:07.040
Yeah.
00:43:07.600 --> 00:43:10.700
I think the glasses, well, they may trend upwards.
00:43:10.700 --> 00:43:14.660
I would wreck, if you don't want to be old, take your glasses off.
00:43:14.660 --> 00:43:17.060
That's all I got to say.
00:43:17.060 --> 00:43:20.500
How accurate was it for you?
00:43:20.500 --> 00:43:20.780
Yeah.
00:43:20.780 --> 00:43:21.660
Was it in the ballpark?
00:43:22.760 --> 00:43:25.060
I was similarly offended, actually.
00:43:25.060 --> 00:43:34.180
I uploaded a photo of myself and it guessed that I was in my early 40s, which I'm not.
00:43:34.180 --> 00:43:35.240
Which is fine.
00:43:35.240 --> 00:43:36.240
Early 40s are great.
00:43:37.120 --> 00:43:38.080
I'm happy for you.
00:43:38.080 --> 00:43:41.280
But you shouldn't be in them when you're not.
00:43:41.280 --> 00:43:43.080
I don't.
00:43:43.080 --> 00:43:47.980
Well, when you're in your, like, I'm in my early 30s and I was like, wait a second.
00:43:47.980 --> 00:43:48.480
Wait a second.
00:43:48.480 --> 00:43:48.680
Okay.
00:43:48.680 --> 00:43:49.460
So I thought the same thing.
00:43:49.460 --> 00:43:55.900
I was like, maybe there's something about my appearance in the photo that is, you know, cause for concern.
00:43:55.900 --> 00:43:56.260
Right?
00:43:56.260 --> 00:43:57.120
I was like, okay.
00:43:57.120 --> 00:43:59.920
Did it like search for ties or like a collar?
00:43:59.920 --> 00:44:01.680
Maybe throw on like a hoodie?
00:44:01.680 --> 00:44:03.840
Exactly.
00:44:03.840 --> 00:44:04.780
Like put on a hoodie.
00:44:04.780 --> 00:44:06.680
I found a photo of myself without a beard.
00:44:06.840 --> 00:44:07.880
I don't, this is like super vain.
00:44:07.880 --> 00:44:09.220
I was like, wait, I can't live with this.
00:44:09.220 --> 00:44:11.500
So I found a photo, an old photo of myself without a beard.
00:44:11.500 --> 00:44:17.160
I put it into the system and it guessed that I was like 37, which is also older than I am.
00:44:17.160 --> 00:44:17.520
It's better though.
00:44:17.520 --> 00:44:18.440
And so I've had to accept.
00:44:18.440 --> 00:44:23.000
It's moving in the right direction, but I've had to accept that either, either.
00:44:23.000 --> 00:44:30.260
It's just like everybody got offended because like the, if, like if you were Microsoft and you were training this algorithm, like wouldn't you want it to skew young?
00:44:30.260 --> 00:44:31.600
Like, let's be honest.
00:44:31.600 --> 00:44:32.060
Yeah.
00:44:32.060 --> 00:44:33.420
Okay.
00:44:33.420 --> 00:44:34.960
But let's say, okay, so it skews old.
00:44:36.060 --> 00:44:38.520
Or, or I'm just old looking.
00:44:38.520 --> 00:44:40.260
Like maybe I'm aged beyond my years.
00:44:40.260 --> 00:44:42.660
I have a kid that, you know, this, the kids do this to you.
00:44:42.660 --> 00:44:46.260
I bet, I bet it would have guessed that I was like 25 until the day she was born.
00:44:46.260 --> 00:44:47.280
And then it was like 40.
00:44:47.280 --> 00:44:50.320
One week later, you aged like 10 years like that.
00:44:51.900 --> 00:44:53.060
Yeah, exactly.
00:44:53.060 --> 00:44:53.100
Exactly.
00:44:53.100 --> 00:44:58.980
So number nine on our list here is actually a little bit of a sad story.
00:44:59.680 --> 00:45:15.740
There was this really cool study about how, how much a reasonable argument with some, not argument as fight, but like a logical argument or a discussion around a political position may or may not change someone's opinion.
00:45:15.740 --> 00:45:25.340
And so this, this study was done trying to change people's opinion, open, open them up being more favorable towards same sex marriage.
00:45:25.440 --> 00:45:39.160
And they found in this study that if they went around and they had people canvas the location, go and knock on the door and talk to people after talking to them, they could actually make them more open to this idea.
00:45:39.440 --> 00:45:50.000
Which this flew in the face of a lot of political science, which is if you argue with somebody from an opposite perspective on something political, they typically dig in and like, ha, I'm way against you, man.
00:45:50.000 --> 00:45:50.540
Right.
00:45:50.540 --> 00:45:53.720
It like hardens them against your argument rather than brings them over.
00:45:53.720 --> 00:45:56.680
So this was a big deal until it was a fraud.
00:45:56.680 --> 00:45:58.100
Yeah.
00:45:58.100 --> 00:45:59.460
Yeah, exactly.
00:45:59.460 --> 00:46:09.160
And, and, and I think everybody was, I mean, it was one of those things where it was like, okay, because I think part of the, one of the big parts of the study was that, or, you know, purportedly.
00:46:09.160 --> 00:46:10.080
Turned out not to be true.
00:46:10.080 --> 00:46:23.480
Was that if the person who was having a discussion with you about your views on same sex marriage was themselves gay, then that would increase the likelihood that you would, your opinion would change.
00:46:23.480 --> 00:46:29.840
So in a good way, like, or toward, you would be more favorable of same sex marriage if you were talking to a gay person.
00:46:30.120 --> 00:46:40.560
Basically saying that, like, if you, this idea that the reason that most of us might hold a prejudice are simply because we're not exposed to people of a certain group.
00:46:40.560 --> 00:46:46.040
And as soon as we are, our prejudices start to melt away, which is kind of a nice idea.
00:46:46.040 --> 00:46:46.440
Right.
00:46:46.760 --> 00:46:48.680
And so everybody was really excited about this survey.
00:46:48.680 --> 00:46:53.740
But it, well, I mean, like we're talking about, it turns out that it was just made up.
00:46:53.740 --> 00:47:00.740
And what's also interesting is the way in which some additional researchers discovered the flaw.
00:47:01.220 --> 00:47:05.820
They were just, as often happens, they were going to do an extension of the survey.
00:47:05.820 --> 00:47:10.500
They were going to build on the research that had already been published, which is pretty common.
00:47:11.140 --> 00:47:20.800
But then when they looked into it, they discovered that the responses in the survey were pretty normal, like pretty consistent, which is actually like they were too consistent.
00:47:20.800 --> 00:47:21.380
Right.
00:47:21.380 --> 00:47:22.760
The pattern was too clean.
00:47:22.760 --> 00:47:35.060
And what they actually found was that there were like irregularities in the data and the irregularities that they found in the data looked like the sort of thing that a human would do if they were trying to be random.
00:47:35.060 --> 00:47:36.400
Like, isn't that amazing?
00:47:36.520 --> 00:47:42.900
Like, so like actual randomness is hard to produce and like human beings just aren't good at producing things randomly because that's just not what we do.
00:47:42.900 --> 00:47:44.640
Like we go like, oh, I picked a three last time.
00:47:44.640 --> 00:47:45.880
So I have to pick an eight this time.
00:47:45.880 --> 00:47:47.020
And I've already picked an eight.
00:47:47.020 --> 00:47:49.760
So randomness says that I wouldn't pick an eight two times in a row.
00:47:49.760 --> 00:47:50.540
I should pick a 12.
00:47:50.540 --> 00:47:52.600
And that's actually not how randomness works.
00:47:52.600 --> 00:47:55.460
So it's like that's super intentional.
00:47:55.460 --> 00:47:57.400
Like you're layering your own bias on top of it.
00:47:57.400 --> 00:48:00.700
And so what they saw was like very uniform noise in the data set.
00:48:00.700 --> 00:48:02.180
And they were like, wait a second.
00:48:02.180 --> 00:48:04.160
This actually looks like you just made it up.
00:48:04.160 --> 00:48:04.940
And it's true.
00:48:05.000 --> 00:48:08.640
Like I guess he took the results from a previous study and then tried to apply them to this one.
00:48:08.640 --> 00:48:15.800
Like and it basically applied the previous study's results in a new context and then sprinkled some noise on top in a way that he felt would look random.
00:48:15.800 --> 00:48:25.580
And these other researchers basically had to say this like huge study, this like seminal study that had been published in, you know, I forget which, you know, nature maybe or whatever.
00:48:25.580 --> 00:48:26.220
Like one of them.
00:48:26.220 --> 00:48:33.620
And it was covered in a ton of like high profile news outlets too, right?
00:48:33.620 --> 00:48:36.340
New York Times and places like that.
00:48:36.340 --> 00:48:39.200
So it got a lot of traction until it crashed.
00:48:39.200 --> 00:48:40.300
Yeah.
00:48:40.300 --> 00:48:40.960
Yeah, absolutely.
00:48:41.220 --> 00:48:50.500
And so, I mean, on so many levels, it's like it's really – I mean, on the one hand, you look at this and you go, oh, what a horrible fraud this person perpetuated on, you know, the public.
00:48:50.500 --> 00:48:53.440
And on the other, you go, well, I guess, you know, the process kind of works.
00:48:53.440 --> 00:48:55.480
I mean, it shouldn't have been published in the first place.
00:48:55.480 --> 00:48:56.740
Peer review should have caught that.
00:48:56.940 --> 00:49:02.260
But ultimately, the academic community discovered the fraud and out of it, which I think was good.
00:49:02.260 --> 00:49:04.900
But it was – so it's interesting in two ways.
00:49:04.900 --> 00:49:08.460
It's just interesting how research, the kind of the mechanics of the research industry work.
00:49:08.460 --> 00:49:22.720
And then on the other hand, it was interesting because it was this really – the thing that – this kind of novel idea about randomness that's not normally part of mainstream discussion that everybody had to start talking about in order to explain why the story worked out the way that it did.
00:49:22.720 --> 00:49:23.280
Yeah.
00:49:23.280 --> 00:49:24.840
How did they catch him making fake random?
00:49:24.840 --> 00:49:25.660
I don't understand.
00:49:25.660 --> 00:49:26.420
What does that mean?
00:49:26.420 --> 00:49:27.720
Yeah.
00:49:27.720 --> 00:49:28.420
That's really cool.
00:49:28.420 --> 00:49:28.760
Yeah, yeah, exactly.
00:49:28.760 --> 00:49:29.460
What does that mean?
00:49:29.460 --> 00:49:31.120
Like, yeah, it was cool.
00:49:31.120 --> 00:49:32.220
That's great.
00:49:32.220 --> 00:49:38.240
And on this note, you know, I was seeing this story in a really positive light.
00:49:38.240 --> 00:49:43.980
Like, hey, maybe we could sit down and talk to each other and we could, like, help evolve each other's opinions one way or another.
00:49:43.980 --> 00:49:45.660
And we could kind of come to an understanding.
00:49:45.660 --> 00:49:49.640
But it turns out that, like, no, probably we can't.
00:49:49.640 --> 00:49:53.460
There's a really interesting book that I think is related to, like, data science.
00:49:54.200 --> 00:50:02.140
It just has an insane amount of data analysis in it called The Big Sort, how the clustering of like-minded America is tearing us apart.
00:50:02.140 --> 00:50:10.080
And it really just goes to, like, 20, 40 years of data of, like, people's opinions and working together.
00:50:10.080 --> 00:50:16.520
And it sort of would not support the study that we can just talk about things and agree more.
00:50:16.520 --> 00:50:17.960
But check it out.
00:50:18.860 --> 00:50:19.640
That's interesting.
00:50:19.640 --> 00:50:20.920
I'll have to go check out the book.
00:50:20.920 --> 00:50:21.920
Yeah, it's an amazing book.
00:50:21.920 --> 00:50:22.800
One of my favorites.
00:50:22.800 --> 00:50:34.940
So, number 10, which I'm going to nominate to be my favorite of this year, is that Python is at an all-time high as a programming language.
00:50:35.680 --> 00:50:43.200
So, the TOB index is one of the more respected sort of how popular is my technology and the technology fight indexes.
00:50:43.200 --> 00:50:48.900
And Python is now number four of all programming languages.
00:50:48.900 --> 00:50:52.320
And it jumped from eighth to fourth in one year.
00:50:52.320 --> 00:50:55.940
So, it's one of the very few with, like, a double up arrow.
00:50:55.940 --> 00:50:57.540
This thing is super growing fast.
00:50:57.620 --> 00:51:09.380
So, for a language that was created in the late 80s, came out early 90s, and it's been around for a long time, this massive jump, you know, I think it's interesting to ask, like, where is it coming from?
00:51:09.380 --> 00:51:11.040
It's coming from academics somewhat.
00:51:11.040 --> 00:51:17.400
Like, this Python is now the most popular language for first-year college students studying computer science.
00:51:17.400 --> 00:51:20.740
But I think it has a lot to do with data science as well.
00:51:22.000 --> 00:51:25.340
Yeah, I mean, it's hard to see how it couldn't be, actually.
00:51:25.340 --> 00:51:37.600
Because I think Python's always been great as kind of along the same lines as PHP, which I can only assume for all the software engineers who listen to this podcast because they like Python so much, I've pretty much just been sacrilegious.
00:51:37.600 --> 00:51:38.840
So, don't get me wrong, you guys.
00:51:38.840 --> 00:51:39.900
I hate PHP.
00:51:39.900 --> 00:51:40.780
Those guys suck.
00:51:40.780 --> 00:51:54.100
But also, Python is a pretty good general-purpose programming language, as is PHP, for building web applications and all of the, you know, all of the sorts of products that we've seen being released on the internet in the past, whatever, 10 years.
00:51:54.100 --> 00:51:57.160
This huge boom in web-based products.
00:51:57.160 --> 00:51:58.960
And Python is great for that.
00:51:58.960 --> 00:52:02.180
It's great for writing software that gets released on the internet.
00:52:02.180 --> 00:52:13.880
But at the same time, it's also overtaken another statistical programming language called R and become, I would say, the de facto language for doing data science and analysis.
00:52:13.880 --> 00:52:16.520
And which is really cool because now it's a powerhouse.
00:52:16.520 --> 00:52:20.860
Now you can do two things in the same language that used to be totally separate from one another.
00:52:20.860 --> 00:52:28.420
So, we talked about before, there's statistical computing, which is basically like, I already know how to do the stats, but I need to script it.
00:52:28.420 --> 00:52:32.680
And what language will help me do that, you know, almost like an Excel power user.
00:52:32.680 --> 00:52:40.240
And then that goes all the way down through kind of, you know, complicated machine learning and artificial intelligence and neural networks and all that stuff.
00:52:40.240 --> 00:52:49.520
And so, the fact that you can couple that with how do I respond to an HTTP request and hit a database and return some content, like, those two worlds used to be different.
00:52:49.520 --> 00:52:55.480
But now we can build smarter and smarter applications that are seamlessly integrated with one another thanks to Python.
00:52:55.480 --> 00:52:57.280
So, I mean, it's no surprise to me.
00:52:57.280 --> 00:52:57.740
It's awesome.
00:52:58.260 --> 00:53:00.880
Yeah, I think there must be a huge boost coming from that direction.
00:53:00.880 --> 00:53:03.820
And generally, people are really jumping into it.
00:53:03.820 --> 00:53:10.540
But, you know, thanks to data science for making our language look even more popular and awesome like it should be, right?
00:53:10.540 --> 00:53:12.540
I know.
00:53:12.540 --> 00:53:16.940
The only languages I think above it on the list are, like, really lame ones that nobody wants to write in.
00:53:16.940 --> 00:53:18.440
They're probably, like, forced to by their boss.
00:53:18.440 --> 00:53:19.160
Like, Java.
00:53:19.160 --> 00:53:20.300
I mean, come on.
00:53:20.300 --> 00:53:20.720
Yeah.
00:53:20.720 --> 00:53:21.600
Let's be real.
00:53:21.600 --> 00:53:22.160
Oddly, yeah.
00:53:22.160 --> 00:53:25.180
A little tear just formed in my eye.
00:53:26.040 --> 00:53:29.080
No, but seriously, like, Java literally is number one.
00:53:29.080 --> 00:53:30.620
And C is up there as well, I believe.
00:53:30.620 --> 00:53:32.480
It's pretty interesting.
00:53:32.480 --> 00:53:36.960
But it's, you know, C and Java are fighting for first place.
00:53:36.960 --> 00:53:38.000
Then it's C++.
00:53:38.000 --> 00:53:39.040
And then it's Python.
00:53:39.040 --> 00:53:39.780
Right?
00:53:39.940 --> 00:53:41.340
So, that's beautiful.
00:53:41.340 --> 00:53:42.300
Yeah.
00:53:42.300 --> 00:53:46.240
I mean, honestly, kind of any dynamic language being as popular as that.
00:53:46.240 --> 00:53:52.500
I think there are a lot of old school software engineers who are crying in their coffee at the release of this report.
00:53:52.500 --> 00:53:53.540
You know, like all of that.
00:53:53.540 --> 00:53:56.900
That's a toy language that will never be useful for anything in production.
00:53:56.900 --> 00:54:00.060
And, yeah, now here we are.
00:54:00.060 --> 00:54:04.500
Python's running some pretty massive, massive products.
00:54:04.500 --> 00:54:06.260
So, it's very cool.
00:54:06.260 --> 00:54:09.020
As I'm sure all your listeners know, they listen to your podcast every week.
00:54:09.020 --> 00:54:12.840
Yeah, I'm sure a lot of them are building some pretty amazing stuff out there.
00:54:12.840 --> 00:54:14.280
But, yeah.
00:54:14.280 --> 00:54:18.580
I think this is great news for everybody who wants to get into Python.
00:54:18.900 --> 00:54:25.780
It seems like the job possibilities, job prospects in regular programming as well as data science are just going up.
00:54:25.780 --> 00:54:31.240
So, if I were betting on my career, I would consider Python one of the top choices.
00:54:31.240 --> 00:54:32.820
Yeah, absolutely.
00:54:32.820 --> 00:54:36.580
And it makes it so much easier to start to explore new concepts.
00:54:36.580 --> 00:54:49.700
So, when you want to start, when you're ready to start bringing in kind of machine learning or any kind of statistical, you know, data processing into your applications, it's complicated.
00:54:49.700 --> 00:54:54.840
But it's a lot less complicated when you're already familiar with the language that you're writing in.
00:54:54.840 --> 00:54:58.380
So, I think it really gives you a leg up when you're trying to make that transition from one to the other.
00:54:58.380 --> 00:54:59.300
Or vice versa.
00:54:59.300 --> 00:55:18.160
If you've been, you know, for the partially derivative listeners, if you've been sort of scripting a lot in Python to try out, to do some research or to work on some analysis, and you want to start building applications to make the things that you're, make the products that you're building accessible to the world, there's a whole ecosystem waiting for you.
00:55:18.160 --> 00:55:19.660
And the water is warm, my friends.
00:55:19.660 --> 00:55:21.160
The water is warm and shallow.
00:55:21.160 --> 00:55:22.460
Wade right in.
00:55:22.460 --> 00:55:24.960
Yeah, exactly.
00:55:24.960 --> 00:55:26.560
So, yeah, it's fun.
00:55:26.560 --> 00:55:28.340
And it's just, it's a fun language to program in.
00:55:28.340 --> 00:55:29.120
Let's be honest.
00:55:29.120 --> 00:55:30.000
Yeah, absolutely.
00:55:30.000 --> 00:55:30.440
Love it.
00:55:30.440 --> 00:55:35.600
So, Jonathan, that's our top 10 list for the big news in data science this year.
00:55:35.600 --> 00:55:40.460
So, I got to ask you, what is your resolution you're going to break this year?
00:55:40.460 --> 00:55:42.100
You're going to give up on in two weeks.
00:55:42.100 --> 00:55:44.940
Have you already decided what you're going to not hold up?
00:55:44.940 --> 00:55:47.760
I already decided what I'm not going to do.
00:55:47.760 --> 00:55:48.240
That's right.
00:55:48.240 --> 00:55:49.420
Let's see.
00:55:49.420 --> 00:55:56.240
The other people on my engineering team would probably appreciate it if my New Year's resolution was to write more unit tests.
00:55:56.240 --> 00:56:02.560
Or better comment my code.
00:56:04.060 --> 00:56:06.840
And I'm almost certain that neither of those things will happen.
00:56:06.840 --> 00:56:10.040
So, let's call, I'll make, I'll do a dual resolution this year.
00:56:10.040 --> 00:56:12.760
Unit tests and documentation.
00:56:12.760 --> 00:56:16.340
That's a pretty safe one to throw out there, I suspect.
00:56:16.880 --> 00:56:21.760
I think I'm going to make mine to actually put proper comments on my get check-ins.
00:56:21.760 --> 00:56:22.720
Yeah.
00:56:22.720 --> 00:56:24.420
Those could be some improvements.
00:56:24.420 --> 00:56:24.720
Yeah.
00:56:24.720 --> 00:56:28.540
Kind of get in a hurry and they're not so good sometimes.
00:56:28.540 --> 00:56:29.960
Yeah.
00:56:29.960 --> 00:56:30.200
Totally.
00:56:30.320 --> 00:56:31.480
Like fixing stupid bug.
00:56:31.480 --> 00:56:32.020
Yeah.
00:56:32.020 --> 00:56:36.600
Like that's, I think, that's probably not as helpful as it could be to my fellow programmers.
00:56:36.600 --> 00:56:42.840
It does express your emotional feel about the code in the check-in, but it doesn't really help them figure out what it meant.
00:56:42.840 --> 00:56:45.500
Yeah.
00:56:45.500 --> 00:56:46.240
That's fair.
00:56:46.240 --> 00:56:46.780
That's fair.
00:56:46.780 --> 00:56:47.600
That's a good resolution.
00:56:47.600 --> 00:56:49.880
But you don't think you'll stick to it?
00:56:49.880 --> 00:56:51.100
I mean, it's a hard habit to break.
00:56:51.100 --> 00:56:54.980
I'm going to stick to it until I get to a super big hurry and there's some production bug to fix.
00:56:54.980 --> 00:56:56.980
And I'm going to like probably skip it.
00:56:57.820 --> 00:56:58.980
Not that I think that's a good idea.
00:56:58.980 --> 00:57:00.100
I'm not recommending it.
00:57:00.100 --> 00:57:02.680
I'm just telling you, these are the resolutions you make and you break.
00:57:02.680 --> 00:57:04.200
Yeah.
00:57:04.200 --> 00:57:04.960
I love it.
00:57:04.960 --> 00:57:06.260
Like this is the recommendation.
00:57:06.260 --> 00:57:12.400
So all of you out there who are learning Python and software programming, engineering for the first time, here's what to do.
00:57:12.400 --> 00:57:18.440
Talk a big game and then ultimately just write a bunch of spaghetti code that nobody can read.
00:57:18.440 --> 00:57:20.500
That's how the pros do it, you guys.
00:57:20.500 --> 00:57:21.820
That's how the pros do it.
00:57:21.820 --> 00:57:26.240
On that lovely note, I think we should probably call it a show.
00:57:26.840 --> 00:57:28.260
Jonathan, this has been really fun.
00:57:28.260 --> 00:57:31.560
Thanks for teaming up to put together an end of the year show for us.
00:57:31.560 --> 00:57:32.900
Absolutely.
00:57:32.900 --> 00:57:33.740
This has been a blast.
00:57:33.740 --> 00:57:34.800
Thank you so much.
00:57:34.800 --> 00:57:37.900
Thank you so much for suggesting it and having me on the show.
00:57:37.900 --> 00:57:38.800
This has been a blast.
00:57:38.800 --> 00:57:39.240
You bet.
00:57:39.240 --> 00:57:39.760
Thanks.
00:57:39.760 --> 00:57:40.000
Bye.
00:57:40.000 --> 00:57:40.020
Bye.
00:57:40.020 --> 00:57:44.120
This has been another episode of Talk Python To Me.
00:57:44.120 --> 00:57:48.520
It's a joint episode with Partially Derivative and Jonathan Morgan.
00:57:48.520 --> 00:57:51.020
And it has been sponsored by Hired and DigitalOcean.
00:57:51.020 --> 00:57:53.020
Thank you guys for supporting the show.
00:57:53.020 --> 00:57:55.560
Hired wants to help you find your next big thing.
00:57:55.820 --> 00:58:00.820
Visit Hired.com slash Talk Python To Me to get five or more offers with salary and equity
00:58:00.820 --> 00:58:04.640
presented right up front and a special listener signing bonus of $4,000.
00:58:04.640 --> 00:58:10.600
DigitalOcean is amazing hosting blended with simplicity and crazy affordability.
00:58:10.600 --> 00:58:16.300
Create an account and within 60 seconds, you can have a Linux server with a 30 gig SSD at
00:58:16.300 --> 00:58:16.860
your command.
00:58:17.200 --> 00:58:18.700
Seriously, I do this all the time.
00:58:18.700 --> 00:58:20.180
Remember the discount code.
00:58:20.180 --> 00:58:22.420
It's Talk Python, all caps, no spaces.
00:58:22.420 --> 00:58:28.140
You can find the links from today's show at talkpython.fm/episode slash show slash
00:58:28.140 --> 00:58:28.480
40.
00:58:28.480 --> 00:58:30.460
And be sure to subscribe to the show.
00:58:30.460 --> 00:58:33.200
Open your favorite podcatcher and search for Python.
00:58:33.200 --> 00:58:34.940
We should be right at the top.
00:58:34.940 --> 00:58:38.840
You can also find the iTunes and direct RSS feeds in the footer of the website.
00:58:39.200 --> 00:58:43.740
Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.
00:58:43.740 --> 00:58:46.480
You can hear the entire song on talkpython.fm.
00:58:46.480 --> 00:58:48.280
Just look for music in the nav bar.
00:58:48.280 --> 00:58:50.200
This is your host, Michael Kennedy.
00:58:50.640 --> 00:58:53.840
I really appreciate you taking the time to listen and share this with your friends.
00:58:53.840 --> 00:58:56.080
Smix, take us out of here.
00:58:56.080 --> 00:58:56.080
I'm out of here.
00:58:56.080 --> 00:59:17.240
I'm out of here.
00:59:17.240 --> 00:59:17.740
Bye.
00:59:17.740 --> 00:59:17.980
you
00:59:17.980 --> 00:59:18.480
you