145: Unsupervised Machine Learning

WEBVTT
0:00:15.522 --> 0:00:21.782
Welcome to another, as we've termed.
0:00:22.390 --> 0:00:50.600
It, duo episode. That's just Jason and myself. No special guest this time. It's because we have a lot to say. We have it all pent up. So expect a high energy episode today. That's right. We'll start off by talking about email. What's more high energy? No, no, spoiler alert. I do tell this to people in my meetings. I'm like, if you're in an afternoon meeting, I'm typically more ramped up. And like, you know, kind of, you'd think you'd be tired by the end of the day, but I get kind of like riled up about stuff throughout the day.
0:00:51.080 --> 0:01:00.342
Yeah, I wonder if maybe we're more common. Because I feel like, isn't that more natural? You wake up, your brain still hasn't really like kicked in yet. So,
0:01:00.595 --> 0:02:18.840
Maybe, I don't know. And then I just crashed at the end of the day. But that's another problem for another time. So as Jason foreshadowed, we're going to talk a bit about email. So this is actually a topic Jason and I were talking about. So we decided to talk about it on air, which is if you are not in a large organization, maybe this is a foreign thing to you. But most folks, I would say, I've never looked at it, actually. But I would assume a lot of people work in big companies. And in big companies, you get a lot of email—a lot of email, not junk mail, not spam. Although I guess you could. But at my company, they're pretty good about filtering that out. Just like emails from random teams, random automated announcements, automated meeting notices. Everything just gets pushed to email. And it's important to stay organized using either rules in your client or on the server or just making sure you put stuff into various folders or flagging stuff. I think everyone has a different scheme. I got criticized actually by my kids because I have like over a thousand unread messages. Oh, see, I can't handle that. Oh, no. Oh, you're going to criticize me too? But I have a method. Like it is, I'm going to say like methodical. Like I do have a method to my madness. Like it is organized for me. I can't not treat incoming email.
0:02:19.220 --> 0:02:31.160
Actually, wait, wait, hang on. So wait, do you have a thousand unread because you like you might somehow know the last one you looked at, you know, the subject line? So you actually read them all.
0:02:31.240 --> 0:02:32.901
They're just marked. They're all.
0:02:32.901 --> 0:02:34.700
Like handled. Okay, okay, okay.
0:02:35.180 --> 0:03:17.160
So it's not that they're not looked at. It's just like sometimes, like you said, I kind of move the—I'll move the meter for is this on me? This is bad habit. I would not recommend it. Like you said, I'll move the high watermark forward as like what I've kind of read through by looking at summaries. And most of me don't need to do anything, but I don't necessarily take the extra few milliseconds, seconds, whatever, to kind of put those in an appropriate folder or get them out of my inbox. So then every so often I've just piled through and via searching, you know, call all the junk that needs to get marked as read. So it's just sort of like another folder for me. Like there's the unread and read folder, and they sit in the same logical folder in my inbox. I would not recommend this approach. You should definitely be more organized than me.
0:03:17.160 --> 0:03:59.820
So I'll tell you what I do for unread is I have the—well, at least on desktop, I have like the double-paned Gmail where like you still see the list of emails while you can see, you know, an email you currently have clicked on. And then I have it set up to where whenever I like hit the up arrow, it marks it as read. You know, like as soon as it gets to an email, it marks as read. So I could just go up, up, up, up, up, up, up on the upper arrow and, you know, and I'll end up with everything read at the end, but I won't actually look at ones that are like marketing and stuff like that. I do. I am pretty good about taking the time to unsubscribe to as many things as possible.
0:04:00.940 --> 0:04:09.380
Does that work like in general? I guess at corporate mail that should work pretty well, right? Like unsubscribing from something should make it stop emailing you. Yeah.
0:04:10.040 --> 0:04:36.039
I mean, even publicly, it's—it's pretty good. I can only think of a handful of times where I actually had to block somebody because like blocks on marketing, you know, distribution, cause they weren't respecting it. Most of the time it's just like a one-click unsubscribe. But I think the most important thing you can do is to have folders. Folders are a total lifesaver. I'm not too sure what's the difference between the server-side rules and the client-side rules. That's one thing I wasn't ever totally sure about.
0:04:36.040 --> 0:05:23.820
So I think in some cases, so like you mentioned Gmail, I think it's a bit different because the most common ways of accessing Gmail are either web apps or Progressive Web Apps. Like they're on your phone, but they're still a web app. And so they're like the state of the kind of set. But if you're using like, you know what, I guess that's POP or IMAP or whatever, you could have multiple different clients. And so if the clients are filtering, they're kind of notifying the server and moving rules. And you could have two clients with two different sets of rules. So if you access your emails from two different places and the clients weren't synced, you could end up with sort of like not good stuff. I think in Gmail, as an example, when you write a rule, it's just always a server-side, right? Like it always in the background when the mail comes in, not when you fetch the mail; the server side stuff gets executed.
0:05:24.520 --> 0:05:34.900
Oh, that makes sense. So like you have a client-side rule, but your desktop's like not on. And so now like none of those rules are taking effect. So you're looking at email on your phone and it's unfiltered. Yep.
0:05:35.240 --> 0:06:07.855
Something like that. Yeah. I think most places have moved to server-side. Although I will say I feel we at my company don't use Gmail. And in fact, I actually don't like Gmail. I preferred they killed it. This is a rant another time. Inbox. I lived with the Inbox way of doing it. I was very organized when Inbox was a thing. Inbox is nice. They killed it. And now it's Gmail. And now I'm unorganized again. So I blame bad tools. And I know there are other ones out there. People will send us links. I know there are lots of other people and folks doing things that are clever. I just, yeah, I don't know.
0:06:07.960 --> 0:06:47.300
You know, I don't know if at some point I stopped getting ads. There was a period of time where I was getting, you know, Gmail ads. So like you go to look at your email and then the first email would actually be fake. It wouldn't be a real email. That was super, super annoying. They got rid of that maybe about a year ago or so. And so that made Gmail kind of like, there's not that many downsides to Gmail. I did like Inbox. What was so magical about Inbox? I also remember loving Inbox, being really sad when they shut it down. But now it's been so long, I don't remember what was good about it.
0:06:47.300 --> 0:07:05.140
They've morphed Gmail over time to copy some of the UI things. But I think it was just like the way of handling the UI and like what was shown in the ordering of like how you went through messages and stuff, which is different. I think the complexity on their side is around like needing to support both the old way and the new way. And so.
0:07:05.340 --> 0:07:17.480
Well, one of the things I think they added was the swiping, right? So like in Inbox and then later in Gmail, you could swipe right to archive, swipe left to delete, or I don't know, maybe I have it backwards. But basically you swipe and it goes away.
0:07:18.320 --> 0:07:22.004
You just know. It's just one of those things you could just do. Yeah. All
0:07:23.640 --> 0:07:26.740
Right. Well, we get on to our news of the show.
0:07:27.740 --> 0:08:49.440
Yeah. My news of the show is simplifying lines with the Douglas Puker. Did I get that right? Puker? Puker algorithm? Anyways. I know what this is, but I don't know how it's said. I have no idea how to pronounce that person's name. I'm sure I butchered that person's last name. Anyways, this is really cool and I found a particularly cool article where they kind of walk you graphically through how to do this. But I was always really interested in the game Worms. Do you ever play this game? Yes. Yeah. So Worms was this, you know, pixel-based physics thing where, you know, there's also Scorched Earth that was kind of similar, but much more simplistic. Yeah. Yeah. And Scorched Earth, like things couldn't float, like the ground always settled. And so it was somewhat of an easier problem. But, you know, in Worms, there would be these floating islands and you could walk on them and you could bounce grenades off of them and stuff. But anytime there's any kind of explosion or anything, you know, it would chip away at these islands and it would start to take away some of the pixels. And so I was always fascinated with this, like, how do you bounce a grenade off of this pixel? Like, you know, this 2D array of this binary array of pixels. And so that kind of led me to this algorithm, which is really cool.
0:08:51.525 --> 0:10:51.380
So basically, you know, you could imagine kind of tracing the outline of one of these islands and then just, you know, anytime there's an explosion or something, you know, retracing that part of it. And, you know, maybe an explosion caused an island to split into multiple islands, and that's fine. You could detect that. But then you have this problem of, okay, now I've traced this island and I have this like, you know, the outline, right? The shell of this island. How do I like put that into a physics engine? Right. And, you know, even like if you were to put every point in a physics engine, the angles would be all really jagged and stuff. Right. And so looking at some game development sites kind of took me to this, which basically this algorithm is this really fast way of saying, okay, I have this. It could be a line. It could be a polygon. It could really be anything. But I have this, you know, line that's made up of a ton of points. And I want to keep sort of the structure, you know, the essence of that line, but get rid of, you know, a ton of the points. So it's a simple, unsupervised way to do that, which I thought was really cool. And the gist of it is you essentially like go from the start to the finish. I guess this wouldn't work if it's a loop. So there has to be other ways to do that. Oh, I think it actually it still works. We have to adapt a little bit. But you go from the start to the finish and you kind of draw a fat line, you know, like imagine like a rectangle that's oriented, right? Like a fat line from the start to the finish. And then you see like, okay, are all the points just in that line, in that in that rectangle, in that zone? If they are, you could just delete all of them. And now you have just a start point, a finish point and a straight line. But if there are some points that fall outside of that zone, then that means, you know, if you were to delete that point, you'd kind of be changing like the essence of this.
0:10:51.380 --> 0:11:25.780
So maybe it kind of like is more of like a parabola or something. Right. So if you find a point that's outside of the zone, then you kind of do this subdivide type thing or it's like a divide-and-conquer approach where you say, okay, let's go to that point and create a rectangle from the start to that point. And then a second rectangle from that point to the end. And now with these two rectangles, did I cover everything? And so you kind of keep going in this way. And yeah, it also looks like really mesmerizing. So if you ever wanted to make your own Worms game, this is how you would do it.
0:11:26.600 --> 0:11:44.860
Yeah, I think there's a few ways of doing this like line simplification. This one uses like you're kind of saying this fat line as like lateral stuff. I think there's some that use sort of circular distance as well, depending on what you want to do. But yeah, you're trying to like preserve the shape, but also simplify the number of points.
0:11:45.840 --> 0:11:47.680
Yeah, exactly. Cool.
0:11:48.160 --> 0:12:18.160
My news article is 'How to Pick a Starter Project.' Now, ironically, or I guess this takes the stance initially of telling you how to pick a starter project for someone you want to get rid of. And by illustration of all the things that you do to get rid of someone, highlighting how to not pick a starter project. And it's kind of a funny thing, but I guess nothing in here is sort of earth-shattering.
0:12:19.965 --> 0:13:57.839
But remember that starter projects really are important for onboarding new folks, like those first things that you have them do and making sure that it works well. It also gets into a big deal with choosing mentors and that kind of stuff. And I just wanted to highlight because I think this is a really poorly thought-through thing, like how to onboard new people to teams. Some companies will have like a company-wide onboarding process, but I think still the importance of within a team, making sure new folks get up to speed and how that's like a joint thing across the manager, but also the members on the team. And so here it's highlighting things like 'Pick a gigantic project that has vague requirements and goes cross-team and cross-discipline and is tracked at a very high frequency by upper management.' These are all ways to inspire someone to quit on the spot and to not get brought up to speed. I think oftentimes, even under poor circumstances, people do integrate into the team eventually. But I think often those formative first few months are critical to kind of having a really positive experience and making sure that the team knows that they should be taking time away from their other tasks to onboard a new person. And I think whenever you change jobs, it's always difficult in the beginning. I can look at these comments and think back to some of the times I've switched teams or changed companies and had starter projects. It's more or less sort of, I guess, were intending to push me away. And maybe I was too dumb to realize that I should have changed jobs. No, just kidding.
0:13:58.459 --> 0:14:02.520
Your starter project is to delete all the code that your boss wrote.
0:14:02.520 --> 0:15:29.780
I did have a starter project that was, you're new to the team. The team has been divisive about coding style. So why don't you come in and start enforcing coding guidelines? Oh, no. You gotta be kidding. Is that real? Nope. Yep. That's absolutely true. Anyways, it is important. And I think as well, if you say, 'Well, okay, I'm not a manager or whatever. I'm not assigning starter projects.' And I think it's important to know that this is a thing and a thing that people don't always do well. So if you're new to a team and you're not getting good work, be communicative to your boss or your co-workers. Like, 'Hey, this seems kind of vague. I'm happy to work on it. But is there other stuff that I could be doing as well?' Be proactive in being an advocate for yourself and trying to, assuming that they're not doing it on purpose. Like, trying to get work that you think is cleaner. I would say, like, one of the more common unintentional mistakes is giving someone something you didn't realize that was a lot harder than it actually seemed. So you give someone a project. You think it's really easy. And then they really, really struggle. This has happened to me before. And then it turns out, once you sort of start reviewing what you're doing, people are like, 'Oh, I did not realize this was that complicated.' I just thought you were kind of slow. It's like, well, people really should be assigning work that almost they would just sit down and do in an afternoon. Like, it really should be too easy. Better for them to finish too fast than to get bogged down.
0:15:30.400 --> 0:17:29.800
Yeah. The other mistake I see a lot is where if there's a junior engineer, someone will—a manager might say, 'Oh, I really wish I had time to work on Project X,' which is some new project that the team, you know, no one on the team has time to do. Project X. But if we did it, it's like this big win. And then somebody joins the company. They say, 'Oh, this person has, you know, 100% free time. They just joined. We're going to put them on Project X.' And it ends up being this really risky thing. And from your perspective, you feel like, as a manager, you might feel like, 'Well, I'm giving them opportunity.' Yeah, this person has a chance to hit a home run and there's no downside because they just joined and, you know, they can pivot and, you know, they're not going to get evaluated right away or anything.' This is actually a terrible idea, right? So the reality is, people are under kind of the most pressure to perform in the beginning. And so, you know, this advice here in the article about, you know, put somebody on the main bread and butter project. Like I see people mess that up so many times. But yeah, as Patrick said, you want, especially for junior folks, but even for senior folks, you want them to be exposed to, you know, the core essence of the company. Even if your long-term plan is for them to try and invent something new. Yeah. Cool. So my article is 'Tic-Tac-Toe in a Single Call to Printf.' These code golf things, I love these things. I've never tried it. I don't think I would enjoy actually building this. It's more of like an art installation than, you know, coding exercise. But it is so freaking cool. So, you know, as you might imagine, there's a ton of pandefiance to make this a reality. There's a certain way in Printf you can actually capture input.
0:17:29.800 --> 0:18:24.972
But I did not know that. I didn't know it either. Yeah. It's—I'm trying to find it here now. I mean, there's a whole document here on how they actually did it. Anyways, there's a way you can capture input with Printf. And they're hijacking that. So basically this prints a Tic-Tac-Toe board, lets you type in input and play games against yourself. The entire thing is done in one Printf call. There is a while loop, while Printf. I guess that's because every time you enter a key or something, right? But really freaking cool. And they actually took the time to make it like actually visually artistic. So the entire program is spaced in a way where there's a like ASCII art percent n in the program. So it's a—oh, they're actually, actually—I take it back. There is a scanf. Yeah.
0:18:24.972 --> 0:18:28.280
I see that. In the argument to Printf, there's a scanf.
0:18:28.580 --> 0:18:41.037
Oh, okay. So actually. No, but that's still crazy. I didn't know you could. It does not seem like a good idea. Oh, the scanf is inside the Printf? Yes. Oh, my gosh. My mind was just blown. Oh,
0:18:41.037 --> 0:18:59.560
No. Oh, that's not that bad. It's just in the argument string. So like comma, scanf, open print. So it's just a function call that's taking place. And then the result going in as an argument. Okay, okay, okay. This is not as like devious, cynical, like as—yeah, okay. This is so crazy, but I got it now.
0:19:00.360 --> 0:19:21.120
Okay, okay. Got it. Yeah, I got it too. Yeah, so, okay. So it's a call to Printf, but that Printf has as an argument, a call to Scanf. Yeah, so there's two functions, but the whole thing is freaking cool. I just—I love seeing stuff like this. So definitely check it out. It will give you a chuckle and you can read through how it's implemented, which is very clever.
0:19:21.120 --> 0:19:37.020
It will give you a chuckle if you're either very against C or very deep in admiration for your C programming language. I think that those two folks may like everyone else. Jason's description may be sufficient for you to realize you could save yourself the click.
0:19:37.020 --> 0:19:49.800
Well, you could copy-paste this into your terminal and GCC it and then play Tic-Tac-Toe. It'd be a fun exercise. They even have instructions on how to do that if you're new to the program.
0:19:49.800 --> 0:20:14.340
If there were only a web browser where you could just type a website and play web games that don't involve incredibly obfuscated C code. Okay, this, yeah, this is pretty cool. I would say the name almost is also a good competitor for obfuscated. So they did a good job naming it to seem, to seem really sinister. It also claims printf is Turing complete, which doesn't surprise me, but sounds horrible.
0:20:15.200 --> 0:20:16.800
Yeah. Oh my gosh. That's wild.
0:20:18.440 --> 0:20:26.421
That's probably like a complexity measure of your language, which is like how many features of your language are themselves Turing complete? Oh, interesting.
0:20:27.700 --> 0:21:14.800
Yeah. I wonder, you know, I wonder what, so there's been historically have been languages where I've had a hard time working with them. I feel like that's pretty common. So one of them is Scala. Another one is Haskell. Those are two languages where I don't have strong opinions. I mean, I actually enjoyed Scala, but I found that the programs that, you know, when you started to scale up the team size, Scala and Haskell both became really difficult for me, at least to understand. And so, yeah, I wonder if there's somehow a connection, like, I wonder if you could somehow take all of this anecdotal evidence and regress it to say, okay, you know, yeah, somehow some complexity metric of the language causes it to balloon in this way. Interesting.
0:21:14.800 --> 0:21:18.480
So why is Python so bad as a programming language then?
0:21:20.879 --> 0:21:23.940
What is the zeitgeist on Python?
0:21:24.540 --> 0:21:33.027
Oh, no, no, no, no. We're going to get them up to. We got to keep making progress. Okay. I don't know. I don't know. We also throw JavaScript in here too. Um,
0:21:33.027 --> 0:21:33.899
Oh my gosh.
0:21:34.680 --> 0:22:42.980
Okay. Mine is completely unrelated. It's not an obfuscated C code, although it probably does have C code on it. And that is the Artemis one project. By the time you listen to this, hopefully it successfully launched—the time of recording, they've sort of attempted the first launch, which got scrubbed. But I want to say this had been a big deal for a long time for the United States government trying to send a rocket effectively back to the Moon. This first one unmanned, eventually, you know, a manned moon base. Contrary to the last time that the United States of the moon, this one's actually being done much more internationally. Some of the sections of the rocket are actually done by the European Space Agency in collaboration with NASA. So I feel like this is a bit different in that way, leaving all the politics aside of the last one. But I want to say, I feel like this came a bit out of the blue. Like it is a big deal, I think. And I think that it had been going on for so long, so much of a run, but people kind of forgot about it. And then it was sort of like, oh, it's on the launch pad getting ready to go. And so that's pretty exciting.
0:22:43.760 --> 0:22:58.680
So, yeah, I mean, again, politics aside, but I thought that back in like 2010 or something, they totally defunded NASA. So I was like really surprised to see this. So I really don't know how, like what, what really transpired there.
0:22:59.160 --> 0:24:32.720
Yeah. I don't know about defunding NASA, but there's been a history of telling various government agencies in the United States like what to do, but then not giving the funding. So it's like a two-step process. First you give them like a new mandate and then you're supposed to go in, in the budget sort of like budget for that new mandate. And so there can be like, 'Hey, you need to go do X.' And then we've got to make the budget. We don't give you money to do X. So are we really telling you to do X or not? It's a bit ambiguous. And so I think that's true of NASA, but true of other government agencies, but I'm not sure on the particulars, but no, it hadn't been defunded. It had been sort of in some ways, people will say uncancellable because it was involving so many states, so many companies, so much international cooperation. There's just all this stuff in it. It just sometimes, you know, estimation for timelines. We are not that software engineer is the only one who get them wrong. But other companies are getting wrong too, but no, I mean, leaving that aside, I think it's exciting. Like people quibble over the expense of it, but I think that giving that sort of optimism and hope of going to space and putting people onto the Moon and the immense amount of technological research that falls out as part of this, I think it's an exciting thing. I won't justify that it's worth the cost. I don't know. I kind of, it's hard to say. Yeah. It's exciting. And like for, you know, having myself children who are elementary age, you know, their teacher turned it on. They get excited about things about science. I think like from that stuff, it's really hard to measure just how big of an impact these kinds of things have.
0:24:32.980 --> 0:25:30.580
Yeah. I was talking to somebody about something related to this. I was talking about how, you know, if you look at like the Sistine Chapel and like, you know, the pyramids, like how, you know, we did these things that were kind of like really powerful. And then also like, like wholly unimportant at the same time, like, like, uh, like the pyramids. So the pyramids are extremely cool as a tourist attraction. And I'm sure Giza like generates a lot of, you know, tourist revenue from it. But like, at the time, like, it was just maybe like a thing to do, or it's like a, like a religious endeavor. I don't actually know the history of the pyramids, but but I feel like, you know, humanity is full of these sort of endeavors where it's like not totally clear what to do, but you do feel like this is sort of like a milestone in humanity, whether it like turns into something economical or not. Right.
0:25:30.580 --> 0:26:30.760
Like it's always hard, right? I think these things are complicated, but from a celebration of what humanity can do for along this technology, you know, pushing the envelope, doing these new things. I think it's incredible. Also, there's been a theme of various space and rocket related things throughout the history of the podcast. But I mean, I think I said it, I think even for our predictions for this year, which probably ended up needing to slide out or whatever, but the just the amount of new rocket hardware coming online and people doing things, we haven't had as many of these duo episodes, but there's a number of other sort of interesting rockets coming to fruition. It's really cool that soon our ability to access orbit and access things like the Moon are going to be just so different than they were before. And I'm really excited to see what we're not anticipating about such a transformation. What do you think about the space slingshot? Oh, is it Spin Launch? I think that's the name, right?
0:26:31.000 --> 0:26:31.919
Oh yeah. I don't know.
0:26:32.560 --> 0:28:31.442
You're talking about the one that spins in the centrifuge that's in a vacuum and then launches. I mean, I think it's one of those people have done stuff similar before in research and giant, you know, artillery guns, basically that would shoot rockets out the rocket equation, which is, oh, this is not my area of expertise, but basically the fact that you need a huge amount of rocket fuel to lift your rocket, but you add rocket fuel and rocket fuel is heavy. So therefore you need more rocket fuel to lift the rocket fuel to lift your rocket. And you end up with this sort of like cascade of stuff. And the Earth is in a pretty heavy gravity well, right? Like it's, the atmosphere is dense. The Earth is massive. So it's pretty hard to get to orbit. And so if you can get just a little percentage of Earth, you know, is diameter away or Earth's radius up and sort of lessen the effects of gravity, lessen the effects of the atmosphere and just sort of get through those really quick by energy held in your launch device, not in your rocket. You simplify a ton of things. Of course, doing that has its own caveats with acceleration and stuff, but I actually Spin Launch seems to know what they're doing. That is not necessarily always enough, but they built a something at a scale people didn't really think was possible. I feel there's a lot of armchair quarterbacking, so I'm hopeful that it'll work because the idea is you have this giant circle that stands up on its side. It gets almost all the way to a vacuum and just very, very little air left in it. And then they rotate this huge arm in the middle with a rocket on one end basically. And then at the right time, they unleash the rocket and it's, you know, shoots out like a sling, you know, shoots up in the atmosphere and it gets through all that hard, dense part of the air gets to a pretty high altitude and then lights its rockets. And so it can be a much cheaper, smaller, more compact, and you can launch a lot of times because you just load a new one up and you do the same thing again. And so,
0:28:32.202 --> 0:28:37.760
I mean, if it works for small satellites, it would be a huge unlock.
0:28:37.960 --> 0:28:47.960
Yeah. I mean, it looks really freaking cool. I mean, Patrick did an amazing job describing it, but you have to watch the video. It's just really, really cool to watch.
0:28:48.440 --> 0:28:55.240
Fingers crossed. I hope it works. And also good luck Artemis. And if you already know what happens in the future, well, yeah.
0:28:55.580 --> 0:29:00.439
Wait, actually, so we should predict, do we predict Artemis will launch in the next month?
0:29:01.459 --> 0:29:06.019
Okay. Well, we're recording at the very end of August. You're saying by the end of September. Yeah.
0:29:06.120 --> 0:29:14.520
So, I think this show will go out in October, but anyways, so let's say September. Will this will the rocket launch in September?
0:29:14.860 --> 0:29:17.258
Well, the rocket will have had launched in September.
0:29:17.258 --> 0:29:19.380
Yeah, that's right. Nice job.
0:29:19.700 --> 0:29:21.900
I'm going to say yes, just 'cause I want it to be true.
0:29:22.780 --> 0:29:32.620
All right. Yeah, I feel like it will too. I mean, they're so close, right? I mean, I think it was just some kind of a liquid leak or something. I mean, they could easily fix that.
0:29:33.760 --> 0:29:36.740
Flex Seal. Just slap it on there and it's good to go.
0:29:40.580 --> 0:29:43.330
Now you know why Jason and I are not rocket scientists. All
0:29:43.330 --> 0:30:33.920
Right. Yeah, exactly. Oh my gosh. It's amazing. Flex Seal. I bought Flex Seal tape the other day. It actually is pretty awesome. I used it to fix a hole in our pool. Like, you know, the pool has that thing, which like goes on the bottom of the pool and cleans the bottom, you know, call it? Yeah. Basically a pool vacuum type. Well, 'pool vacuum' means something else. That's like a thing that you manually do. Anyways, so I used Flex Seal to fix a hole in that. It actually worked pretty good. I was impressed. I think there was this—there's this one I found online that I also ended up using for some other project that actually it gets hot when you stretch it and it like the tape chemically bonds to itself or whatever. It's like wild. I mean, they have amazing, crazy tapes now on Amazon.
0:30:34.400 --> 0:30:50.155
We can talk about the tape. That's like electrically conductive sort of through the thin part, but not across the tape. So along the long part, it's not conductive, but like, so you can if you can pass through the tape, but not along the tape electricity. Whoa,
0:30:50.543 --> 0:30:52.095
That's wild.
0:30:53.000 --> 0:30:53.860
Okay. Anyways.
0:30:54.080 --> 0:30:54.540
All right.
0:30:56.060 --> 0:31:29.600
Oh, actually one more shameless plug. So, there's a gentleman that I know who started a company called Bitrip, and Bitrip is basically—I mean, I'm probably going to totally butcher this, but imagine just like a roll of tape with QR codes. And so the idea is if you're out in the field, like you're an electrician, you know, like a public utilities worker or something, you could just like slap this on anything. And then you scan the QR code and put some data into some app. And then someone behind you can like scan the same tape like a year later.
0:31:29.919 --> 0:31:40.105
I see this. So it's like every rip is like a unique code? Is it so is it like a non-repeating pattern? Like what do they call that? Oh man. You
0:31:40.105 --> 0:31:45.977
Know, I don't know the details. I always thought it was just like tape with QR codes on it, but I haven't actually seen the product. Oh,
0:31:45.977 --> 0:31:46.200
Okay.
0:31:46.380 --> 0:31:47.440
Let me see. Bitrip.
0:31:48.180 --> 0:31:52.200
Yeah. It looks a little bit like QR codes, but they're not exactly QR codes.
0:31:52.660 --> 0:31:57.960
Oh yeah. You're right. It's almost like a barcode or something, but it has a unique fingerprint.
0:31:58.640 --> 0:32:03.038
Now I'm curious about it. You brought this up, man. Now we got to know what the secret is. We
0:32:03.038 --> 0:32:16.217
need to get the Bitrip guy on the show. Oh, 600 GPS tracking tags. Embed photos, documents, audio. Oh yeah. It's definitely not a QR code. It's some kind of. It's
0:32:16.217 --> 0:32:23.744
like a Penrose tiling. If I just had to take a naive guess, I think it's a Penrose tiling, which is like a non-repeating infinite series. I
0:32:23.912 --> 0:32:25.500
think you're right. Yep. Yep.
0:32:26.220 --> 0:32:30.540
But okay. Oh, we probably messed up his pattern. We probably shouldn't be here like reverse engineering tape on the.
0:32:30.679 --> 0:34:30.340
Yeah. So, how many of you think that in the month of September we'll get sued by the Bitrip guy? All right. It's time for book of the show. My book of the show is The Meditations by Marcus Aurelius. I remember first hearing about Stoicism a long, long time ago and thinking to myself, oh, that's pretty much me. I live like a really simple, relatively simple life, always trying to simplify things. And it kind of really resonated with me. So years and years later, I decided to read this book. Marcus Aurelius is—I don't wouldn't say he's the founder of Stoicism, but he's the person who really kind of popularized it. And I guess you know, it was a bit of what's the word? Like, I think I overhyped it too much in my mind. You know, it's kind of like, actually Patrick, you brought this up before the show how it's like another one of these books is The Art of War by Sun Tzu, where everyone talks about these books all the time. And you think, oh, this is going to be something that's going to totally blow my mind. And what I actually found was that so many people have already talked about this book that I already kind of knew what was going to happen. So it really—it's almost like kind of watching the movie of Jurassic Park after you've read the book, or maybe Harry Potter or whatever, any of these. So it's kind of like, you know, it was a little bit underwhelming because I'd already kind of known the material, but I felt like it was still a good book. Halfway through it. If you don't know what Stoicism is, or if it sounds interesting, if you're interested in how to lead a simple life, like what that means metaphysically and everything, check it out. It's a good book. It's also, you know, obviously really dated. I think Marcus Aurelius was what, like a Greek emperor, I think, or Roman emperor. I think Roman emperor. Yeah, that's either right. A Roman emperor. Roman emperor. There you go. So, so yeah, I think it's going to be a very dated book, but it's a little
0:34:30.340 --> 0:34:46.800
difficult to read, but I think it's nice. And you know, it could be sort of something that you do while you're in the car and just kind of have it in the background. And you know, there's probably a lot more contemporary books on Stoicism and other kinds of philosophies. I highly recommend.
0:34:47.820 --> 0:35:30.279
I think understanding some of those philosophies and approaches, even if you're not saying, 'Hey, I'm actively seeking to model my life or to do this as a key tenant,' I think still are useful for helping see how other people think, to just have new ideas and to question if there's some nugget of value there for some part of your life, rather than necessarily reading a self-help book where you're saying, 'I'm going to adopt this as everything about who I am and make it my core identity.' I feel sometimes there's—I don't want to say like a pressure, but in my head at least like a thought like, 'Oh, if I read this, I'm kind of wanting to adopt it as,' and I don't think that's accurate. I think like Jason's pointing out, you can read it and still learn a lot from it.
0:35:30.960 --> 0:37:28.979
Yeah. I think, you know, I was really fascinated when we were working on YouTube. I was really fascinated by what made things go viral, like what actually makes things go viral. And one thing that I learned in that process is that a lot of things that you think are organic or actually not organic, right? So there's actually a lot of viral videos where you look at that and say, 'Oh man, you know, perfect timing,' but it's actually highly scripted. So it's very hard to tell what's real, what's fake. But either way, I think that what I noticed was a lot of viral content on really anything, any type of media, it sort of taps into this kind of latent—I don't know how you describe it—just like latent, like shared common culture, like the sort of hive mind of humanity, or maybe hive mind of if you want to be more local, like hive mind to your country or your region or whatever. It's like it does it directly say, 'Like, hey, we're going to talk about Stoicism today,' but it just—it sort of taps into a lot of that sort of latent energy. Like Harry Potter is the example that keeps coming up in my mind. How if you actually look almost all of those stories, like at one point they fight a snake, and that snake, like Harry Potter fighting the snake, is like you can see the parallel to this other ancient story that people read for hundreds and hundreds of years. And so it's like you have this evolutionary footprint, and really popular content sort of likes piggybacks on that footprint. So yeah, reading a lot of these canonical books will give you kind of an understanding into that, which would help you if you ever wanted to write or produce some content.
0:37:29.282 --> 0:37:35.100
Like write a book or, or, or, uh, or anything like that, that, that, uh, wants to tap into that, that same energy. Yeah.
0:37:37.060 --> 0:38:36.968
Well, as tradition holds, Jason gave us a very high brow, very thoughtful book. This may be a bit late, but we've not done as many of these recently. So, it was all the fad during, sort of like the COVID at home stuff to kind of bake sourdough. I see that. I see that meme. Mine is, mine is a book about bread baking: flour, water, salt, yeast by a gentleman named Ken Forkish. And there are a lot of books about bread making and artisan bread. And, I think this one, for whatever reason, just reading it and the stories, it really kind of like made me excited to try the recipes, to do them, to take the approach. I just thought it was a very thoughtful way of thinking about bread baking. There we go. I can, and using simple ingredients—I guess they call it sort of like a lean bread, which is there's no, there's no fat in it, right? Just flour, water, salt, and yeast. They of course have, is
0:38:36.968 --> 0:38:39.420
That literal? Like, do they literally put fat in bread?
0:38:40.180 --> 0:39:56.720
Yeah. So an enriched bread. So like, if you—I mean, like a challah bread or something like an egg roll, right? Or like a yeast, you put egg or butter or oil, like a lot of pizza doughs have oil in them. Wow. Oh, okay. That makes sense. Yeah. Pizza dough. Yeah. And so, those would be like enriched in some way with some kind of fat. And so they're like, and a lot of bread we eat has a little bit of that. And so kind of going back to this very basic sort of loose shaped, you know, round bread gained popularity. I know on the West Coast of America, the sort of like Portland and San Francisco, these places have like kind of developed a renaissance of this style bread making. But if you're interested in bread making, which most of you probably aren't, that's fine. This, I would encourage you to check out this book. I really liked this book. I baked a lot of the recipes in here and had a good time. You know, I don't get super into it. Probably should do it more, but it does take quite a while. It's an endeavor, but it feels good at the end to really eat it. And there is something hugely different about eating a loaf of bread—you know, smelling it, cooling it, eating it that you made versus going to the store and buying one. And so I think, I think it's something that people should try at least once.
0:39:57.160 --> 0:40:14.260
Yeah. This is super cool. So, I was looking up the author to see if Ken Forkish is a real name or not. It just seems like too good to be true to write a book about cooking and he has 'Fork' in it. But so far, everything I look up, says it's not a pseudonym. There's actually a real guy named Ken Forkish.
0:40:14.440 --> 0:40:17.860
I didn't even think about that until you said it. Yeah. I think he runs a restaurant in Portland.
0:40:18.100 --> 0:40:31.229
That's right. Yeah. You got, he runs a bakery in Portland and he actually, before opening that bakery in 2001, he worked in Silicon Valley as a tech worker for 20 years. Oh,
0:40:31.229 --> 0:40:33.700
Maybe this is why it resonated. I didn't know this.
0:40:34.280 --> 0:42:32.580
Yeah. So that's wild. So 20 years. So that means he went to Silicon Valley in 1981. That was probably when it was literally Silicon, you know, like making chips and everything. Then yes, worked there 20 years, opened a bakery. Good for him. Very cool. All right. So time for tool of the show. My tool of the show is this app called Pythagorea. I guess sticking with the Greco-Roman theme that I have going here. So Pythagorea is this game that where you basically have to solve geometric puzzles. And so that sounds like it would be really boring, like doing math homework as a game or something, but they actually do a great job of making it really engaging. One of the things that they do really nicely is, the game is played on this 16-point grid—or maybe it's more than that. It's a 36-point grid. So there's a six by six grid of dots. And so, the first level is very simple. There's a dot on the left side, dot on the right side. They're like, find the dot in the middle. You tap the middle of the screen, you move on. And then, it kind of tells you, 'Hey, you know, you have two dots, make like an isosceles triangle.' And so you can drag lines between the dots to make triangles, but you're restricted by these dots and the fact that you can only draw a line either from dot to dot, or you can make new dots where two lines intersect. That's where it starts to get really complicated. So for example, there was one puzzle where you kind of needed a point that was sort of in the middle of four points. So you needed a point where there wasn't one. And so what you have to do to solve that is you can just make a little X, right? So imagine your mind like four points, you know, in a square shape, right? And if you draw the two diagonals, now you have an X, right?
0:42:32.640 --> 0:44:32.420
And so then the middle of that X, you can actually tap that and now make a fifth point in the middle. And so you can now, like when you get to the harder levels, now it really opens it up because you can really make a point anywhere, as long as you can figure out how to get two lines to intersect at that place, right? So, the hard puzzles start getting really hard where it's like, 'Okay, I need to go to like seven sixteenths of the way between these two points.' And so like, what lines can I draw to make that happen? And so it's actually—it's really fun. I mean, it's one of these things that's very hard to explain through audio. But I highly recommend you check it out. The other thing is it's completely free. It's a donation game. So I went and gave them, I think it's like a dollar or whatever they're asking for, but you can play the entire game cover to cover, totally free, no ads, nothing like that. And I found it really stimulating. The other thing is, as soon as you get it right, it kind of dings and you know you got it right. You're not really guessing. And so some of the levels, I would stare at it, stare at it, stare at it, stare at it. And I'd kind of figure out, 'Okay, here's sort of the trick.' And then, you get that trick, you solve the puzzle. It's very satisfying. I felt like they did a good job with the pacing. A game like this, it's very easy to make a level that's extremely difficult, and then you just can't move on and it's really frustrating. They did a good job of ramping up the difficulty. And one of the other things they did to help with the pacing is the game is broken down into chapters, but the chapters aren't in increasing complexity. They're just different phenomena—geometric phenomena. And you can actually play all the chapters asynchronously. So if you get stuck in Chapter Two, you just go to Chapter Three. I felt like that was also really clever game design, and yeah, definitely check it out. Totally free. So there's nothing to lose.
0:44:33.500 --> 0:44:40.120
Awesome. That's really cool. So now you're going around with like a compass and ruler and making dodecahedrons and like showing all your friends.
0:44:40.540 --> 0:44:48.370
Everything looks like a geometric problem. It's like, 'Okay, you know, this door won't close.' Let me pull out Pythagoria. You know? That was
0:44:49.500 --> 0:46:23.714
Awesome. Mine is—we might have even had this as a tool to show before, but that is Google Keep. I feel a lot of people may have heard of this before, but some may have not. It's a way of doing note-taking, but I think the power that I recently realized about Google Keep is having sticking with it and using it and jotting what amounts to kind of like Post-it notes in the app or on the web, or sending links as a way to doing bookmarks, and putting pictures in and just sort of gathering a lot of unorganized data. And then just being able to search it, to be able to go to dates, to be able to do categorization, but even just leaving it messy and then putting stuff there over time and building it up. Something happened to me where I was like, 'I think I had written this down one time,' or I was cooking some dinner and I was like, 'Oh, I think I last time debated what temperature to put this at.' And like, well, let me go see. Oh yeah, sure enough, I took a note here because it felt like something I'd want to remember. So just putting these little shots of a note or a picture, recipe, or a link, and being able to find those things later, anytime where I try to figure out something that I knew and I don't, or couldn't find it—I always try to make sure to go put it in there the second time because if I needed it, you know, twice, probably going to need it again. And so building that up over time, I think there are some open-source or different alternatives and other platforms that people use. So this is my tool—it's Google Keep, which Google allows you for free comes with all the traditional Google cons, I guess. But that is a pro, but then, you know, are
0:46:23.714 --> 0:46:24.680
There ads or no ads?
0:46:25.299 --> 0:47:00.980
I don't think I've seen ads, but people in general have a love-hate relationship with Google these days, which I completely understand. So don't put any private information there that you're not willing to share with Google. But using a tool like this—I guess would be my shout out—which is something where you can just very low overhead, not super structured, just sort of put your information in and allow it to kind of accumulate. We talked about the importance of email organization. If you're super organized, maybe you don't need this. I already confessed in the beginning that it's something I need to do better at. But, you know, I think here, this is a way for me to not lose those little ideas.
0:47:01.680 --> 0:47:09.759
Yeah. Do you rely on search then to retrieve the notes? Okay. So the notes aren't like hierarchical or anything?
0:47:10.299 --> 0:47:33.000
No, I've seen stuff and always been intrigued about doing that. I feel like the hesitation for me for hierarchical or linked notes and stuff would be really cool, except that I just know that I'm not going to put—I'm going to worry more about where in the hierarchy goes. And then therefore I'm not going to put stuff in if I don't think it'll fit in the hierarchy. That makes sense. So here it's like, I need to just record it.
0:47:33.779 --> 0:47:44.759
Yep. Yep. Totally makes sense. I wonder if maybe we could automatically generate the hierarchy. Oh, what do you mean? That'd be pretty cool. Like maybe with some unsupervised learning.
0:47:48.480 --> 0:47:50.451
Speaking of which, if only our
0:47:50.451 --> 0:47:51.580
episode today was about.
0:47:52.680 --> 0:47:55.899
How did that happen? Cluster my notes.
0:47:55.900 --> 0:49:01.820
ClusterMyNotes.com. That needs to be a—Oh, did I tell you a bit of a side topic and then we'll jump in on unsupervised learning. I made a website called visual-if.com. The idea is the UI is very clunky, but it's an interactive fiction game that runs Dolly as you're playing the game. So, you know, you type in like, you get past the intro screen and it's like, look up, look down movies. Like one of these interactive fiction is like Zork or Adventure, right? But anytime there's a room description or anything, Dolly is running in the background and you get like some crazy art installation of whatever that is. And it really actually makes the games really fun. If you've played interactive fiction before, you know, if you've played a particular game before you could play it again and just see all these really trippy pictures with it, see if it matches up with what you were envisioning when you were playing at the first time.
0:49:02.560 --> 0:49:11.080
I—I'm not sure if I went to the right spot or not. It's either really well done to be confusing or I'm on some other random person's website.
0:49:11.700 --> 0:49:14.980
So it's Photopia is the game that starts when you.
0:49:15.420 --> 0:49:16.780
Okay. Yeah, yeah, yeah, yeah, yeah.
0:49:17.040 --> 0:49:39.500
Yes. It says, 'Would you like instructions?' Yes. The UI sucks. I'll admit it. So it says, 'Would you like instructions?' And you actually have to click on that and then type no or yes. Oh, okay. Okay. And then it says, 'Will you read me a story?' You have to actually click again. And then now you're in the game. I need some way to like, maybe I should make like a little intro screen before the game. Kind of tell people what they're in for.
0:49:40.180 --> 0:49:46.180
Nice. This is cool. I mean, these pictures are like kind of creepy. I'm not going to lie. The ones I'm getting. Yeah.
0:49:47.020 --> 0:51:44.460
Yeah. It's like, there's a person skiing or something. At least that's what I see. But yeah, it was pretty fun. I might try to double down on that if I have time. So I found out people didn't really understand how to use that product. Like I just asked people; they were like, 'I don't really understand what I'm supposed to be clicking on.' But you know, as you kind of work on any type of product, you definitely can show it to your friends. You could try it out yourself, but you will eventually want to show it to strangers, right? And as we talked about with Kevin in the marketing episode, right? Eventually you're going to give your either app or website or whatever you're building to the public. And you're not going to be able to really look over their shoulder and find out what they were thinking about it. So, you know, definitely we talked about marketing and surveys and all of that. There's a whole human element too, but another thing you want to do is, is you want to kind of gain insights from data. So ideally, imagine if you're making this app, for example, this visual IF. I didn't implement this, but you could imagine, you know, I could track where people are moving their mouse or what they're clicking on, or how long they're spending on that site, right? And that could all go into some kind of report that would give me information. And then I could even go for a step further and A/B test. I could try a new version of the site to see if it improves, right? The challenge is, you know, all of this data that you're going to get is going to be highly unstructured, right? So imagine, it's going to start with these print logs that maybe you did when you were doing development. So, you know, a person clicked on a page or printed the page ID or printed—you know, a person moved the mouse to the bottom of the page, right.
0:51:45.259 --> 0:52:33.260
And you have to somehow turn that into something that you can look at, you know, some type of graph that you could look at and say, 'Oh, here's some thing that I can do to make things better.' And so that's at a high level, like one of the main things that we want to do with unsupervised learning is take a lot of raw information. And like, one of the biggest examples that is used repeatedly is Wikipedia. You know, take all of Wikipedia and can you just learn something from reading? You're having a computer read all of Wikipedia. So that's basically what unsupervised machine learning is all about. And yeah, I think Patrick, you brought this show topic up to the table. I think it's a great show topic and something that I am really passionate about.
0:52:33.260 --> 0:54:30.500
So, yeah, I mean, I think like Jason was saying this, taking your data and sort of helping to structure it. But I think, and I made the joke about clustering earlier, I feel that sometimes Jason is a machine learning practitioner, right? Like that's his trade. I'm not. So in general, I try not to do it, not because I don't know what it is or can't do it, but just because it comes with certain expectations. And so when I set out to do work, it's like the engagement model I have with the data that I have in front of me and with the task at hand is a bit different than Jason. Jason might have as a machine learning practitioner. At the same time, I think this area in parts has a lot of overlap between—and I'm just making, I don't know if there's probably a better word—but sort of machine learning practitioners and sort of other folks. And so I think there are things like, for example, like clustering where there may be features of your data or heck, there might even already be numbers that you have along multiple dimensions, just even two or three dimensions where if you clustered them together and looked at them, that would already be quite helpful. And you may say, 'Well, that's not machine learning. That's not—you know, there's no Neural Network. There's no whatever.' But I think that's okay. I think things where you're fitting a regression to your data and trying to say, 'You know, hey, look, there's numbers here and I'm fitting a line to it,' so I can think about what the next number would be or between two numbers or I'll pass the last data point I have.' I think those are kinds of things where you are overlapping in a lot of this and learning about those things and thinking about them. And those is another tool in the toolbox. We talk about that all the time. And so I'm pretty excited to talk about some of these things because I think a lot of them have value even to people who wouldn't call themselves machine learning practitioners, but anyone who has data, which tends out to be a lot of people—most people have some.
0:54:30.500 --> 0:54:38.640
amount of data that are trying to work with and ways of organizing, cutting that data down, smoothing the data, thinking about the data, all those kinds of things.
0:54:39.600 --> 0:56:38.120
Yeah. Right. So, I mean, imagine like you have a bunch of data around people who visit your website, right? And so you want to set up clusters. You want to set up kind of cohorts. For example, you might learn that there's somehow that there's a lot of identity around age. So it's like the people visiting your website are either really young or really old for whatever reason. And so maybe they're coming for two different reasons. You have to figure that part out. But, you know, clustering will kind of looking at the centroids. The centroids are the centers of the clusters that you've developed and can give you a lot of information. So, for example, you might have a whole bunch of different features and then you might do clustering, which basically says for each of the data points—and a data point would be sort of like a set of features. So maybe a data point is a person who's come to play your game and a set of features about them. Like how do you describe this person? Right? And so then after you've done, you're finished with a clustering algorithm. Now, each person, they're going to be assigned to one cluster. If it's a hard clustering, if it's a soft clustering, then they could be assigned to like some mixture of clusters, right? But that's neither here nor there. So now you can look at this—at sort of these clusters that people are assigned to and you could find the center of them. In other words, given this group of people who are all assigned to Cluster A, what is the center point there? So what is like the person who would be most aligned with Cluster A, like this hypothetical person who just perfectly lands in the middle of the cluster? And you say, 'OK, this is sort of an archetype. Like this is—there is something unique about this group of people.' You know, you can also do this with like faces if you're trying to face recognition or even if you're trying to object recognition.
0:56:38.120 --> 0:57:02.320
You can even do clustering on images and say like, 'OK, I have this huge bank of images and they're falling into one of several categories.' So I think, you know, clustering has been used for tons of different things. You can also then do machine learning on top of the clusters. What have you used clusters for in your work?
0:57:03.040 --> 0:58:45.100
Yeah, I think like one of the things that came up recently is we noticed that which—I guess just thinking about your data is like we were doing some processing and sometimes certain configurations of input were causing the data to take a lot longer than other configurations. Sorry, speaking vaguely, but whatever. And so we were trying to kind of understand if there's a difference in the features in one. And so this what you alluded to, it's exactly right, which is, 'Hey, we have a whole bunch of measurements like the size of the input data.' The like, let's just say it was text, right? Like how many characters are there? How many lines are there? You know, how many punctuation marks would there be? Right. Things which we could kind of look and say, 'Hey, is there something about this that's making the processing take longer or not?' And once we sort of started plotting it out and saying, 'OK, hang on, let's look at what the clusters are of these like sort of easy inputs and hard inputs.' And like what? What? Oh, OK. Well, look, these hard inputs all have a lot of extra punctuation in them. And then, you know, realizing that the processing we were doing was going to cost a lot more when that happened, right? But we didn't—it was vague enough that it's a bit difficult to sort of know that in advance and just look at your code and say, 'Hey, actually, I see here we do all this extra work with punctuation.' It was the sort of second-order effect that was causing it. And so by doing this clustering and just looking at the results and saying, 'Oh, look, these things are different than those things,' allowed us to kind of say, 'Hey, up front, let's check for that and handle them specially or maybe decompose them further or do something special.' And so that's what we use it for.
0:58:45.440 --> 1:00:11.880
Yeah, that makes sense. I mean, one area where you see a lot of clustering is around log, you know, log ingest and log reading. So imagine, you have a website. The website has a MySQL database. It's got servers, back-end servers. It's got the JavaScript on the client. And all these things are generating logs, right? Your server, your database is generating all these logs like, 'Oh, you know, I'm getting full up,' or 'Oh, like the utilization is too high,' or 'I've actually, to be honest, have you ever looked at a database log like a MySQL log?' No, I've never done either. I just assume it works. It shows that we're terrible DBAs. But, you know, it's generating a ton of logs. And, you know, if your database like goes down or all of a sudden it takes—I do have some actually. Like there's a lot of popular websites that are only run by like 10 people. You know what I mean? Like or I think Craigslist is famous for having just extremely small staff for such a popular website. But, you know, eventually like you'll run into this where your site just doesn't load and you're going to have to go step through all of these. So you look at the client, say, 'OK, the client's fine.' You go to the server logs and it just says, 'You know, it's just waiting on the database access,' or 'Wait database results.' And you go to the database. It's like, 'Yeah, utilization is 100 percent,' and you have to end up doing something, right.
1:00:13.860 --> 1:02:08.900
So you're getting all these logs. A lot of it is code that you haven't written, right? Because they're logs from programs that you're using. And you need some way to say, 'OK, can I separate the signal from the noise?' Right? Like is there like is this log actually interesting? And actually a lot of these systems like Sentry and Bugsnag and these other systems use clustering. So what they'll do is they'll take every log line and they'll do what's called an embedding, which we can get to later. But but they'll basically take every log line and turn it into a point in some space. So imagine some cube, but it's like a hypercube. It's like, you know, a 200-dimensional cube or something. Actually, we talked—I've got it in my mind. Yeah, I've got the 200-dimensional cube. I'm picturing it in my mind. That's how like Magnus Carlsen plays chess or whatever. Yeah. But we talked to Edo Liberty about embeddings on that show. And so, yeah, you have this big cube. And you've figured out a way to take these lines of log and put them in this cube, right? So if I have a log line that's printing every second, that's like, 'You know, things are good.' You know, it's like 1901. Things are good. 1902. Things are good. That's going to look the same, right? It's always going to say 'Things are good,' and then some kind of date, right? And so since it's so similar—and even the date, the number, the timestamp is also kind of similar—those will likely end up close together in this space, right? Once you've done this embedding, and so you can throw all these logs into that space and then do clustering. And chances are the 'things are good' message will get its own cluster if there's so many of them. And then you could just throw them all away, right.
1:02:08.900 --> 1:02:40.339
You could also do things like say, okay, this line of log isn't even really near any of the clusters. So it must be something pretty unique, pretty special. So maybe this one I should send an alert or something like that. And that's called outlier detection. And that's also a really hard problem, but there's a bunch of great libraries. There's PiOD, there's a bunch of great libraries for outlier detection.
1:02:41.920 --> 1:02:45.560
And they're all kind of related to clustering.
1:02:46.680 --> 1:02:50.100
So what are some other examples of unsupervised learning?
1:02:51.440 --> 1:04:43.580
Yeah, I think a lot of these like words, like unsupervised learning, reinforcement learning—a lot of them have become kind of really nebulous, right? As all of these fields have overflowed, right? But now there's the hot thing is sort of self-supervised learning. And the idea with that is it's still unsupervised in the sense that you don't have a human. Actually, we should probably talk about that. So, supervised learning is typically where you have a human in the loop. So imagine if I'm playing chess and I train some model to mimic my moves. If I move the pawn, then I tell some model, hey, when you see this board, I want you to move the same pawn to the same place. And so it's supervised. I'm a supervisor, right? And it is just trying to mimic this, right? Now, unsupervised would be a little different. Unsupervised would be where, for example, you might train an autoencoder. So you might say, here's a picture of a chessboard. I want this algorithm to put, you know, embed that picture. So find a function that takes this picture of this chessboard and puts it and creates a point for that picture somewhere in this space. And then I want another function that takes that point and creates the picture again, right? And so you're going from the picture to the point back to the original picture. And so when you do this and you train this model, it ends up having to represent, you know, the essence of that picture in that point.
1:04:43.580 --> 1:06:24.660
So, for example, let's say all of my pictures have the same chessboard and it's on a black table, right? And it's the same camera setup. It's like a tripod. So it's very reliable, very stationary. It's all pictures of this black table with this chessboard on it. So it can just recover the black table without needing any extra information, right? Because every single point that we draw in that cube, when you go back to draw the picture, you're going to need that black table. And so that's where like this really powerful compressive ability comes in. So actually, the points now don't need to differentiate based on the table. They're all going to have the table in it. And so, if two points are close together, then that means that the two images they generate must be similar, even given the similarities that there are broadly. Like they must be even hyperlocal. They must be similar. Otherwise, those two points will get pulled apart. And so the way the autoencoder works is, you know, you generate the chessboard. It's not going to look exactly right. So you have some error and then you say, okay, you know, this pixel is like too dark or you drew a pawn here and you really shouldn't have it—it's empty. And so you, given that, the right answer, you just tell the model, hey, here's the right answer. Adjust yourself. And it will figure out how to use that volume, that embedding volume in the best way or in a good way to be able to generate all of those pictures, not just one of them. Does that make sense?
1:06:25.420 --> 1:07:23.440
Yeah, I think so. I mean, that was pretty deep. But here, I guess when we were talking about like clustering, you don't necessarily mention there, like the right answer. There's no way to necessarily feed that. So you might as a human tune something or do something like the number of clusters. But there may be algorithms you do that, but there's really no necessarily right answer. When you're talking about it's still unsupervised, but this sort of autoencoding, you're kind of giving a problem, constricting the amount of information that can be shared between sort of like the left half and the right half. And then trying to say like, you need to simplify down to a representation and then reconstitute that representation back to the original and then look and compare the two. So you have a well-defined metric for saying like, hey, how well did you do at your task? And so you algorithmically are supervising it, but as a human, you're not sort of like at each interval, sort of like labeling something or giving a behavior to emulate.
1:07:24.060 --> 1:08:01.080
Yeah, that's right. And the reason why all of this kind of comes together is, you know, clustering. You know, imagine you're looking at a group of people, like you're in a helicopter and you're looking down at a stadium full of people or something, or you're looking at a football game or something, right? There are these little dots, like they're maybe the size of ants or something running around on this football field. So when you cluster, you're going to be using sort of the geometry of the field, right? So if somebody is twice as far away, then that really has a big impact on whether they're going to be in that cluster or not with these other group of people, right?
1:08:02.880 --> 1:09:50.160
And so for all of clustering, you need to have a space that's pretty uniform. So like, for example, let's say you fed a bunch of features into some clustering and one of your features is person's age in milliseconds. And the other feature is person's height in meters, right? Well, one is enormous, right? Your age in milliseconds, it's a huge number. And so the clustering algorithm will totally ignore the other feature because your height in meters is, you know, I don't know, from, you know, I guess 0.5 to 3 or something. You know, it's such a small range. Actually, I guess there's nobody nine feet tall. But anyways, so your height in—your height in—I'm trying to figure out the tallest person in the world is what, eight foot? Anyway, so, you know, your range is tiny. It's like three units, right? But your age in milliseconds is enormous. And so the clustering algorithm will just cluster ages until you, you know, not pay attention to the other one. And so, if you're trying to cluster images or text or some of these things, you quickly run into this problem where, you know, the thing isn't geometric. And so the clustering can't really take into account different dimensions in a way that's fair. And so the nice thing about this autoencoding is, you know, the way that the loss kind of propagates backwards from the correct chessboard to that latent space to the input chessboard. So the way that those dots move and the way that things kind of shake up ends up creating like really nice spaces where all the dimensions have relatively the same importance. This is
1:09:50.650 --> 1:10:22.640
interesting. This is interesting. So, yeah, so you're training both halves, but you may be taking—I guess you were calling it like the latent space in the middle, the encoded thing—and doing some using it as input to other parts of your system or sort of like clustering in that sort of more well-formed space so that you can say things about it, even if you never end up reconstituting. Like there's really no reason for you to get back to the original chessboard. Like you had it as input, like you could just use it. You didn't really need that part, but it helps you to get that middle part that you could then use to do clustering on.
1:10:23.360 --> 1:12:14.900
Yeah, exactly. So now let's imagine we have a bunch of people who go to your website or we can even stick with the football analogy. We have a bunch of football players, right? And we have a bunch of statistics about them. And these statistics are all over the place. Some of them are important. Some of them aren't important. The units are all different. And so if we just feed these players into some clustering algorithm, then it's going to have a really hard time. Like maybe basketball. I know, I know more about basketball. So, you know, basketball, these people score a lot of points, but their height, you know, in feet, let's say, and decimal feet is going to be relatively small. So some might score 20, 30 points, but they're only like seven feet tall. And so the, so you have different scales there as well. Right? You know, you have a number of minutes played. And so all of these have different and even slight differences in units can really matter. You could do some type of, let's say, contrastive learning, which is a self-supervised approach. So you might say, here's a list of players who I felt played very similarly. So I could come and say, okay, you know, Shaquille O'Neal and Dikembe Mutombo. They're both centers, really tall, strong people who just can are strong enough that they can just push their way through and dunk the basketball. Right? Those people are very similar. So I'm going to pull these two people together. So, whatever their features are, they're going to—we're going to create sort of a point for these two people based on their features. And then we're going to say, these two points need to be closer together. Then I'm going to take Shaquille O'Neal and like Anfernee Hardaway.
1:12:15.480 --> 1:14:03.780
And so Anfernee Hardaway is like a three-point shooter, like small person, like for basketball standpoint, small person who goes and shoots shots from far away. So these people clearly are far apart. You might even actually just use the positions, right? They, you know, the—you might say, okay, all the centers should be close together. And then take two people who are from two different positions. They should be far apart. And so in this way, you're not it's not supervised learning because you're not saying, okay, you know, Shaquille should be right here or, you know, Shaquille should make like this many points or something. You're basically saying, you know, these people should be closer together. These pairs should be close together. These pairs should be far apart. And it's contrastive learning. It's self-supervised if you can automate all of that without a human in the loop. You know, basketball is a weird example because at some point a human did decide you should be a center, right? So maybe not the best example, but you could even imagine like doing contrastive learning on images. So you could say, here's a bunch of images that are on the same website. And because just by virtue of them being on the same website, they should be pulled together. And then here's two images from two different websites. They should be pushed apart. And so if you do this and you have a low learning rate, because that's going to be a weak signal, right? But if you do this and you have a ton of images and you've scraped a lot of the internet, you'll end up with an embedding that's really powerful. And so contrastive learning and autoencoding, where you feed in the same thing that you're trying to predict, are two ways of generating like really nice spaces that then you can do clustering and other things with. This is awesome.
1:14:04.821 --> 1:14:26.160
So we were sort of giving examples and sort of saying the algorithmic approach, I guess, to doing this. What are some applications of what people do with—I mean, we talked about outlier detection for logs. I think that was a good one. We were talking about it's like classifying things, right? What are some other examples of applications of this process?
1:14:26.160 --> 1:15:28.622
Yeah, I mean, you know, all of the language processing is now pretty much done in this way. So, for example, you know, it used to be that if you wanted to train a model, let's say to translate French to English, you would have to pay people to manually translate tons and tons of things, like literally millions of sentences. And then you would train—you would have some translation that just goes from French to English, right? It's extremely expensive, right? So now what they do is, they will do what's called a word embedding. So basically, there's a whole bunch of different ways to do this. One way would be a self-supervised approach where you say, given all the words up to this word. Like, what was it called? Like the Brown, what is that one that's like you see all the time?
1:15:28.920 --> 1:15:49.099
The quick brown fox jumps over the lazy dog. Yeah, that's it. Okay. So you say, like I didn't know this. The reason that's a sentence is because for handwriting, it uses—it's like the shortest sentence that uses all the letters of the alphabet, or a very short sentence which uses all the letters of the alphabet. What? So it was a penmanship test. I never knew that. I never knew that. I learned that like a week ago.
1:15:49.760 --> 1:16:27.380
What? Shut the front door. Wait a minute. And then I was like sitting there counting them all. Yeah, no, no. Yep. They're all there. Wow. Oh my gosh. My mind is totally blown. I feel like that. Have you seen that video where the guy pretends to do a magic trick and he takes the straw and he has his friend put the straw behind the other guy's back and it blows his mind? Anyways, yeah. That's me right now. So, wow, that's freaking awesome. Okay. So, so is what the quick brown fox or no, the quick—I thought the dog was brown. The quick brown fox jumps over the lazy dog. I thought the dog was brown. Anyways. So let's say the quick brown fox. Any word ring probably works.
1:16:28.260 --> 1:18:27.900
So, so, you know, this algorithm will learn—you give it 'the quick brown,' and then it has to produce 'fox.' Right? Now, if you look at that in isolation, that's like almost impossible, but you give it a ton of these sentences, and in every sentence you say, okay, here's the first word. Predict the second one. Okay, here's the first two words. Predict the third one. Predict the fourth one. Right? And you give it all of Wikipedia or something. Right? And so, you know, yeah, I mean, for some of these subjects and objects, it's going to be really difficult. Like 'the quick brown,' it could be anything. But you're going to also see a lot of correlation. So you'll notice like whenever you see of, like maybe you see the afterwards, very common. And so you'll actually learn a lot of structure from doing that, from what they call a forward model. Right? And the awesome thing is it's effectively free. This is another thing because of Moore's Law and because computers have become so cheap and so efficient, you know, it's really the people time that's the killer for a lot of these things. Like if you can eliminate the time that a human has to do something, you're in really good shape. And so, with these forward models, you just download Wikipedia. I mean, you could do this on your laptop right now and predict the next word. You don't have to pay any humans to rate any sentences or anything. So now you have this embedding, which says, given a part of a sentence, I have some point in some space based on what words coming next. And so, you know, sentence fragments where the next word is going to be 'fox' will all kind of be close together. Right? But it turns out now, if you do that same translation problem, but you work with that embedded space instead of with whole sentences, you need—and it's been a while since I saw this, but I think it's like one one thousandth of the data or something like that.
1:18:27.900 --> 1:19:32.240
I mean, it's extraordinary. I mean, the difference is unbelievable. And so, you know, what used to take millions and millions of sentences, now after like 10,000 sentences, you're done. And there are even models now where they've embedded—they've actually done this jointly with different languages. And so they embed like literally every language into the same space. And then all they have to do is train the second half of it, which does the translation part. So, yeah, so all of Natural Language Processing completely redone with self-supervised learning; like it's massively changed that field. And I think even with image processing, you're starting to see a lot of interesting things. The image equivalent of this is where you basically cut a piece of an image out and you say, reconstruct that missing piece of the image. It's like, remember the Magic Eraser, that Adobe Photoshop thing? That was like really popular like 10 years ago.
1:19:32.460 --> 1:19:33.740
Content aware. Yeah.
1:19:34.160 --> 1:20:14.029
Yeah. And so you could actually, yeah, you could erase a person and it'll fill the fence in behind them. Right? So imagine, you know, you cut the person out, but your intent in this case isn't to literally cut them out of the picture. It's to see like how good your reconstruction can be and if—and you immediately know what the algorithm did right and didn't do right. So in this case, you actually want it to generate. So you wouldn't do something really difficult, like cut out an entire person. You would, you'd randomly cut out squares. And some of the time it would be impossible because you cut out a whole car or something, but most of the time you'll cut off parts of things and you'll be able to. Yeah.
1:20:14.040 --> 1:20:19.380
Like if you cut out one person's eye, you just copy their other eye or whatever. Yeah. Or something. Yeah.
1:20:19.960 --> 1:21:50.640
Exactly. And so same kind of thing. So you have this model that reconstructs things by putting them into this big cube. And then you can just do more and more things on that cube. So if you look at DALL-E from OpenAI, this is what they did. So they trained a model on Wikipedia to predict the next word of the sentence, like I just described. And then they trained another model on images. And then they created another thing which said, 'I have captions for images.' So I have like a picture of the quick brown fox jumping over the lazy dog, and then I have that caption. And those two points should be close together. And then I'm going to take captions that don't belong with their picture—mismatched caption-picture pairs. And those points should be far apart. And I'm going to take those input embeddings and now train another, what's called a joint embedding that tries to unify or push apart those pairs. And that's how DALL-E works. So, so then, you know, when you go to OpenAI and you say, 'astronaut eating ice cream on the moon,' it's taking those three models—the language model. I guess it doesn't need the image model anymore, but it's taking the language model, the image model, and this joint model. It's using all three of them to generate that picture of that astronaut on the moon.
1:21:50.640 --> 1:21:59.025
And the question is, is the Artemis capsule floating around the Moon in the background? We should
1:21:59.025 --> 1:22:02.600
just type 'Artemis' into DALL-E and see if it's on the Moon or not.
1:22:04.200 --> 1:22:11.240
I think like, but they have exclusions for a lot of proper names and nouns and stuff. So I don't know. I don't know how that works.
1:22:11.500 --> 1:22:29.540
Really? Yeah. I use it a little bit. I found it to be, you know, really captivating. There's something powerful about that. I've always been a really big fan of Dolly, but other than this visual interactive fiction, I haven't found a practical use for it.
1:22:31.080 --> 1:22:51.640
Well, it feels good to do a duo episode. I know it's been a while, but going through the habit of the first few episodes—well, many, many episodes of you and I doing this together. It feels, it feels good to do it again. Do our tools of the show, book of the show, news. And then this discussion about machine learning was a really good time.
1:22:51.980 --> 1:24:38.760
Yeah, definitely. I think these are all very accessible, approachable things. You can use SageMaker or other tools. You can train on all of Wikipedia without having to download it to your desktop if you don't want to. So I think I saw training that model I just talked about run you like 30 bucks or something, which is the price of going to the movies. So it's not nothing. But it's also pretty amazing that for 30 bucks, you could train a model on the entire Wikipedia corpus and it'll come out correct and everything. So, you know, it's a lot of fun; amazing times we're living in. And I guess as a final thing, anything you build, you're going to need to collect some type of metric to understand the people who are using your product. And so this is a really good area for folks to brush up on. All right. With that note, yeah, it's really awesome doing a dual episode. Looking forward to seeing this one come out. Looking forward to seeing if we're right or not about our prediction. And really looking forward to your emails. We've been getting a ton of really great emails. So appreciate everybody out there. We do read them. Even if Patrick has them marked as unread, he has looked at the subject. Yeah. I actually think both of us literally read every email that we get on Programming Throwdown. So we really appreciate your support and supporting us on Patreon and Audible. So, so thanks so much. Definitely subscribe if you're not subscribed to the show using whatever podcast catcher; we should be on all of them at this point. If we're not, let us know. And we will catch y'all in two weeks.
1:24:42.740 --> 1:24:52.719
Music by Eric Farnedollar.
1:24:54.519 --> 1:25:12.559
Programming Throwdown is distributed under a Creative Commons Attribution ShareAlike 2.0 license. You're free to share, copy, distribute, transmit the work, to remix, adapt the work. But you must provide an attribution to Patrick and I. And ShareAlike in kind.