Episode 839 transcript

FLOSS-839

Jonathan: Hey folks. This week we talk with Benny Vasquez about Alma Linux. What's new in version 10, how it's different from RL, why electing the board might be a challenge this time, or even more of a challenge next time and a whole lot more if you don't want to miss it. So stay tuned. This is Floss Weekly, episode 839, recorded Tuesday, July the first.

I wanna get paid twice. It's time for Floss Weekly. That's the show about Free Libre and open source software. I'm your host, Jonathan Bennett, and today we've got something that, well, I have already learned some things that are gonna help me out in my professional slash private life because I've got an old server over in that corner of this room inside the cabinet.

It is just old enough. That I can't run. Well, I didn't think I could run a spin of Enterprise Linux 10. And I have discovered in the pre-show that I can. And that's something we're gonna talk about because we are chatting with Benny Vasquez of Alma Linux. Welcome, welcome, Benny.

benny: Thank you so much. I'm happy to be here.

Thanks for having me.

Jonathan: It is great to have you back. You know, I didn't go and look at what show number you were on last. I bet I can find it real quick because my Google FU is strong. And you

benny: probably don't have many Benny's,

Jonathan: probably not, especially not Benny's with a lowercase B at the beginning.

benny: Exactly.

Jonathan: That's gonna be your thing.

benny: It's, it's, it started as a weird. Silly branding thing because I was working in marketing at a, a web hosting company. Mm-hmm. And we were talking about like personal brands, but honestly before personal brand was a thing. This was what? 2004? 2005. And now anytime I see my name with a capital BI just like, I just get.

Jonathan: It was 764 episode 7 64. Nice. Year and a half ago. So things have, things have happened. I don't know. We'd say changed. Yeah. A few things have changed. Yeah. But lots of things have happened in LINACs since then. I. Before we like fully dive into that, I actually, I was thinking about this last night and I how hands on are you with like the technical stuff?

Do you run Linux somewhere? Do you have a server running All my Linux,

benny: so yeah, I do actually have my, my, my pre-life, I guess I like the first whatever I used to work at this company.

Web hosting control panel. Mm-hmm. And after I left there, I needed a place to host my website. So I set up a CPA server with, to like everyone did. Like

Jonathan: everybody did. Yes. Yeah. Yes. And then

benny: I obviously after I, after sent to. The end of sent was announced and I needed something to move to. I obviously picked on the Linux, and so yeah, I, I do in fact have a, a server running Linux.

I also have a laptop running on the Linux. I'm not using it right now because I'm, I have been using Mac for 15 years at this point, so I, I have not adjusted my workflow.

Jonathan: You know, there's this, there's this project you can put on that Mac.

benny: There sure is. It's a, actually, so Lin doesn't support the M fours yet.

Jonathan: Mm-hmm.

benny: But I'm sure you've heard about OA Linux, right?

Jonathan: Oh, yes.

benny: So cool.

Jonathan: Yes. That is, that is an impressive engineering effort.

benny: Yeah.

Jonathan: And I was, I was delighted to see that they are continuing on and still making good progress even after Hector Martin. Kind of burned out honestly, that that's what happened.

Yeah. I mean, that's what

benny: it looked like to me.

Jonathan: Mm-hmm. Yeah.

benny: As, as a person who has burned out, thankfully, not publicly, but I've definitely hit that burnout. Yes. That's exactly what that looked like to me. Yes.

Jonathan: Yeah. And so you can run on the, the earlier, like the M1 through M three, you can put Almo Linux on it.

Mm-hmm. Not the M three. Okay. The one of the M two. Okay. Yeah, still that's still, that's fun. That's a lot of fun. Yeah. There's, there's some really, and that's a good segue actually 'cause there's some really interesting architectures that you can put Homo Linux on now. So there's arm builds of course, but pretty sure there's, there's risk five builds as well.

benny: So we're very, very close.

Jonathan: There's

benny: preview build, right? We have both. We, we've been active in the work that the Fedora Project has been doing.

Mm-hmm.

So we've been helping there and we're trying to make sure that like. Things flow the right way.

Right.

So it started in Fedora and we've been working there and we've been, we have a cluster and have started doing builds of Linux.

It is very slow, as you might imagine.

Yes.

Especially compared to what we're used to with, with all of the other builds that we have. But we have a. Bunch of hardware that's been donated to the Elmo project and it's, it, it's actually in Iraq in our infrastructure leads basement right now because of course, because open source, right?

Because of course

Jonathan: it is

benny: that how this goes, but we've been yeah, we're working on it. We're. We were hoping to bring it out with 10. Mm-hmm. But it's just not ready yet.

Jonathan: Oh, yeah. I'm not surprised. It'll,

benny: it'll be there.

Jonathan: I, I've commented several times, so I've, I've purchased like risk five development boards.

In fact, here's the, here, here's one of them, the star five, you know. Nice. This, this is the, it's the JH 71 10 and like it's great, it's got dual ethernet on it, so you can put open WRT and run it as a router. It's got HDMI out it's got, you know, the raspberry PI 40 pin header on it too. Mm-hmm. Which is sort of interesting that that's become a standard, but it totally has.

benny: Yeah.

Jonathan: It's not a great experience to actually use.

benny: No,

Jonathan: but it is, it is just enough. Let's see, how did I put it? It is just enough for developers to get on it and compile things and so it like, so very, very important bootstrap step for risk five. Exactly. But not ready for prime time yet.

benny: No. And if you think about it, like we have never in, in my experience at least, I have never seen a new architecture mm-hmm.

Get adopted at the speed with which risk five is being adopted.

Yeah.

Like kudos to the Risk five project and all of the work that they're doing to.

Really cool.

Jonathan: Yeah, because most of the other architectures that Linux supports, like predate Linux, like Arm has been around for longer than the Linux colonel has. Mm-hmm. X 86 has you could sort of say that we saw the X 86 64. Architecture come along, I think after the kernel was started and probably things like titanium maybe, but of course nobody supports that anymore.

But yeah, it's, it is, it's, it's, it's very, very new and yeah, there are some definite growing pains from how fast that support has come online. How fast the, the architecture's been built for that matter too.

benny: Yeah. Well, and especially because there are, like I understanding right? Like minute differences between the different boards.

Mm-hmm.

That mean that, okay, we support risk five, but we only support this like, producer of it. Like,

yes.

That's, that's one of the other problems we're running into is how, how much can we do and how limited do we want to be? If we do release it.

Jonathan: Yeah. It's the, it's the fragmentation problem.

benny: Yeah.

Jonathan: On the micro architecture level, which is, which is it?

Although we sort of have that now with X 86 64 and the whole V two and V three and V four scheme, which mm-hmm. Don't map perfectly to reality. Like, that's the FI guess that's a problem with any time you try to put things into categories. The categories are useful, but. They're just categories. They don't map perfectly to reality ever.

Homo Linux is doing something about the X 86 64 version levels as well. Yeah. Can you tell us about that?

benny: Yeah, so the extremely boring story, short, I guess. With with Red Hat 10, they changed the minimum version of the chips that, that they support to X 86, 64 V three.

Mm-hmm.

Which is. And they don't, they're doing that so that they can take advantage of the speed increases and the features that come with that chip set.

Sure. Fully understand why they did it, but it is a huge, it leaves behind a huge amount of hardware that's still perfectly valid, still perfectly usable. Like one of the, one of the arguments is that. The hardware isn't efficient or isn't it like isn't good for the environment. And I'm like, sure. But just trashing it isn't good for the environment either.

Like replacing it just to replace it isn't good.

Jonathan: There's, there's a lot of, there's a lot of those weird arguments that get made that, so the, the term that I kind of like is greenwashing, where it's like, oh, we're being environmentally friendly. It's like, yeah. You're not really though. Yeah. Anyway, that's an entirely different subject, but

benny: like if you feel, if you feel compelled for that reason, great.

Upgrade your stuff, that's fine. But there, there's huge amounts of especially adoption for the Onlin community where replacing hardware that quickly, just because Red Hat decided to drop support is not an option. Mm-hmm. Right. They asked us to support V two and we added a build. And then as part of that conversation, it was like, we all know that the, the enterprise ecosystem hugely relies on Apple, the extra packages for Enterprise Linux project from Feda, and they follow Red Hat.

So the builds that they're doing. We'll now only support V three and that means that Al's V two release is not as useful as it could be. And so we worked with the Apple project and, and said, Hey, we're we wanna rebuild your packages for V two, we're not gonna change anything, any bugs or anything that we find, we'll send upstream.

We'll make it so that. Bug reports are very clear. Like if somebody comes to you with a, a problem and it's with one of our builds, it'll be very clear in like the data that they give you. Mm-hmm. So you can redirect them to us. Is that cool, man? And they were, they were supportive of it. So what we are doing is just rebuilding Apple.

For V two. So the people that wanna use Linux with the V two support can still take advantage of all of the work that the Apple Project does.

Jonathan: Yeah, yeah, that's great. It's, it's amazing how many things you gotta do with a r or RL derivative and the instruction stall install Apple and there's, there is another, I don't remember the name of it.

There was one other sort of semi-official third party package, but Apple is, is the, the extra packages for Enterprise Linux is the big one. Yep. So with, with these things that Almo Linux does, and we'll get into more of them, but where, who, who pays for this? Who pays for the rebuild of Apple for.

X 86, 64 V two. Where does that money come from?

benny: That is always a good question. That's always a question. So there's two.

Okay. So we have 35 sponsors.

Okay.

That give us support in a variety of ways. Mm-hmm. Just like any open source project, some of them give us money, some of them give us people, some of them give us like credits. Sure. For example, both Microsoft and Azure. No, both Azure, Microsoft, Azure, and a Ws.

That what I was trying to say.

Things

right. Give us credits on the, the cloud platforms to do builds and, and host, you know, all of the different pieces that go into making analytics go. Mm-hmm. But we also have two. Platinum level sponsors that give us money and people,

Hmm.

So the, there's Cloud Linux and there's CyberTrust Japan.

And those two, between those two, especially around like a build time, we typically have around 40 like active engineers. On a regular basis, it's much closer to like 10 to 20, but in terms of like the effort, that's all done by those people that are paid to contribute to ly. Mm-hmm. So it's not like I'm, it's not like I'm writing a check to somebody, but it's definitely the, the foundation finding the, the resources to do it and then the work being done.

Jonathan: Right. Is, is Almo Linux itself a nonprofit?

benny: Yes. Yeah. That was one of the big things for us.

Jonathan: Mm-hmm.

benny: When the whole shift with Santo Us came out, we said, we, we don't want this to be possible. We don't want this to be. Anything that can be bought or sold.

Mm-hmm.

We want it to outlive us in such a way that as long as the community needs it to be around, it'll be, so we set up a 5 0 1, six in the us.

We're hoping to get our C3 status, but that requires a diversity of in funding that we just have not hit yet. And then. That is the board is all elected. We have that the, we're actually having another election this year. Mm-hmm. That will bring up, I think it's five of the seats are getting reelected this year.

And the, the foundation is staffed and managed by volunteers, but a number of those people like me are sponsored to work full time for the project. I two years ago, holy crap, it's been two years. I was at Microsoft and I got laid off.

Hmm.

As part of the big, the first big wave of layoffs in 2023.

And while I was looking for a job, the whole shift in Red Hat source code location thing happened and it became really apparent that. Really needed somebody doing kind of community outreach and marketing sort of stuff full-time. And I convinced the the folks at Cloud Linux to pay me to do this job full-time.

So I've been, I, I've gotten to do open source leadership and open source, you know, advocacy full time for. It's, it's pretty awesome.

Jonathan: I, I've, I've not seen any angry mobs with torches and pitchforks coming for Almo Linux, other, other projects. Sure. But Almo Lins not so much. You guys have stayed out of the news for, for that particular reason.

So

benny: I'm trying, man. I'm sure

Jonathan: that

benny: hilariously this is, this is my favorite sticker.

Jonathan: I, I like that so much. And it happens for various reasons, but it, it drives me nuts when I see open source projects and, and all of the drama around the like mm-hmm. Needless drama.

benny: The needless drama is the problem, right?

Yeah. Like, there's going to be, we're all humans, we're gonna. Like hard conversations. That part is, frankly, it makes us better when we have hard conversations because if if, if I am the only, if, thank God, the people in my project have strong opinions and the willingness to bring them to me and say, we're doing this wrong.

We need to fix it. Because if they weren't, then it would just be the Benny Show, and that doesn't make any sense. Like, I don't have enough expertise.

Jonathan: Yeah, absolutely. It makes me happy

benny: that we're doing that.

Jonathan: Absolutely. I, I think, I think the thing I see that drives me the craziest is a, a, an open source project does something that is mildly controversial, which again, that happens sometimes hard decisions have to be made right.

And. It shows up on NICs or a YouTuber covers it. It's like, well, okay, that was gonna happen. Yep. And then people start giving feedback like, we don't like this. Mm-hmm. And some small percentage of those people that give feedback are not very nice about it. Mm-hmm. Okay. This happens all the time. And then you've got a community manager.

This is the part where really it goes sideways, in my opinion, watching this in several projects. The community manager. Freaks out and just starts banning everybody that has negative feedback. It's like, no, that is not the right solution here. No, you need to walk away from your phone. Turn your phone on silent.

Sit it on the table. Just walk away for 24 hours. Yeah, let people know that I'm gonna go take a break, walk away, and then come back and deal with it. Don't just ban everybody.

benny: No. Like, and it's, it's one of the hardest things to teach and learn is how to weather that kind of storm. Mm-hmm. I've been through a couple of situations where like a community around what I was doing or what I was the face of at the time were very, very upset.

Mm-hmm. And you have to be able to navigate the line between you know, I'm. I hear you and I want to hear your feedback, and I want to engage with you. And now you're doing personal attacks, like that's, that's not okay. Mm-hmm. We can have a conversation about how mad you are. I'm a hundred percent into that.

Mm-hmm. But as soon as you cross this line until like making it personal, then it's not okay. But knowing how to deescalate, knowing how to. Engage with people that are that angry or justifiably or not, is, it's a hard skill to learn.

Jonathan: Mm-hmm. I, I think not taking it personally is a big part of it, and that that is also different, especially when you're in the midst of it and it's your hundredth message that you've had to read for the day.

It is very difficult to have that detachment. It's really hard.

benny: Yeah. It's really hard, especially because when you, if you. If you have attached your personal value to the thing that they're attacking

mm-hmm.

Then it's really hard not to take it personally. Like if, if somehow we did end up with a, a mob of pitchforks, right now I have put so much of myself into analytics.

I think that I would have a really hard time. Mm-hmm. But it is. It's a lot easier when it's a commercial thing and you didn't make the decisions. You just say, I, I, I didn't decide any of this. You can be as mad as you want because I'm mad with you.

Jonathan: That's on them. Exactly. Yeah. Fun stuff. Yeah. So you guys are about to have an a an A board election.

And this kind of fits interestingly into this because there's hmm. There are projects that are having problems filling their board slots. I've, I've seen some of this. Is that something that all Linnux has had to face or has the lack of drama helped with that?

benny: Actually, we, we have had to face that.

Mm-hmm. And it's funny because I would've expected the lack of drama to.

While we have people who are passionately in support of Almo Linux, they are not excited about joining a board. Mm-hmm. Right. It's, it's a thing that all open source projects face, especially around trying to balance between making sure that the people that are on the board are invested in some.

That, that the people who care about the project for whatever reason, get to engage in a way that feels comfortable for them. Mm-hmm. It's so hard because you wanna have, you know, as, as a person who is responsible for the foundation, I want to make sure that the people who are involved in the project know what you know, what conflicts.

It are and how to man navigate that side of kind of situation. Mm-hmm. And know how to, you know, approach the, I'm trying to figure out how to politely say this. There, we're going to have disagreements, but we need to go forward united. Mm-hmm. So there's like coming together as a group to, to have.

Disagreements in a productive way. Mm-hmm. And then go forward as a united front is important. Mm-hmm. I never wanna tell anybody they're not allowed to have an opinion, but that's always rough. Right? When, if you've got, if you end up with a person who doesn't have the right motivators, shall we say?

Yeah. Involved in.

Especially because as the, you know, the, the people who are very angry with Red Hat kind of calm down and get quieter. We're not seeing the, the influx of people saying, I wanna give here because I feel really strongly that I hate red. Well, I want you here mostly because you love lytic. Right? It's a slight difference.

Jonathan: Yes, yes.

benny: It's hard to, it's hard to bring that those people in or even really identify them, especially because being on a board feels like a big thing.

Mm-hmm.

Even though it's really just like coming to a meeting, talking to, talking to everyone, and making sure that the project has, you know, financial stability and is continuing to serve its community.

Like, that's, that's all you gotta do.

Jonathan: Yeah.

benny: But it feels like a big thing.

Jonathan: How many, how many board members are there or are there slated to be,

benny: that's a thing that I should know off my head. I think it's nine. I think we have nine slots and I think we have eight people currently occupying those.

Jonathan: Okay.

Are, are they all community selected or is this, does all lytics have a situation? I know some, some organizations do where, like if you're a platinum sponsor, you get to put one of the board members on the board. Is that, is that something that Linux does?

benny: So, no, it is all community elected, but there's a, there's, there's an asterisk the platinum sponsors and all the sponsor members get slightly heavier votes, like weighted votes.

Mm. Okay.

So they, the way that it happens is like you have, I think they have. It's some number of votes. I don't wanna misquote it because then people will accuse me of misrepresenting. But I, there is some number of votes that if I am a, a silver sponsor member, then I have, as an individual member, I have one.

Mm-hmm. I think silver sponsor members have 10. Okay. So they can, you know, spread those across multiple people if they wanna, or they can throw them all behind one person. Each level gets different numbers of votes. Mm-hmm. So the platinum sponsor members do get a pretty heavy influence and because of that can pretty easily put a person pretty far ahead if they care about it.

But we also have 400 individual members. So even if that platinum sponsor gets say 40 votes, that it's still gonna be balanced out by the individual members. They're not gonna be able to put somebody there that is bad for the project.

Jonathan: Yeah, yeah.

benny: You know what I mean?

Jonathan: Mm-hmm. Yeah. Certainly makes sense.

Yeah. And so when you guys do the election, how many of those seats go back up for, for reelection? Do they all go at the same time?

benny: They do not. Thankfully.

Jonathan: Yeah, that's at first it, that seems like it'd be a little wild.

benny: Yeah. It would've been that. But we have the, the, the first election was in October of 2023.

Mm-hmm.

And we since then have had two additional elections. Is it two or maybe just one? Man, I should know all this.

Members of the board right now will not be coming up for reelection because they were just elected. I think they still have one more mm-hmm. Year in term, so they won't, they won't be up for reelection, but there are six of us. That will be,

Jonathan: yeah. Very cool. Yeah. Do, do you, do you have to get elected too?

benny: Mm-hmm. Yeah. Interesting. I'm, I'm the chair. But the chair is chosen by the board.

Jonathan: Right.

benny: So we're all elected as directors. Mm-hmm. And then among the directors, we get, we, we pick a, a chair.

Jonathan: Yeah. Very interesting. It'd be, it'll be fun to see who, who makes it and who doesn't. You said that there were some, there were some challenge in finding enough people to run.

Are, are there some shoe winds. Is there, is there anyone that's running that's really controversial?

benny: Well, so we don't, haven't done the, the nominations yet. Okay. The election will happen in September, I think.

Okay. Is what we

decided. And the nominations will happen sometime in August. Okay. A couple months.

Yeah. So we've got some time. The, the big thing, we thankfully haven't had anybody that.

We in talking with the directors that are up for reelection, everybody has off, has decided that they will be rerun rerunning. Mm-hmm. Which is nice. But that also means that all of us are gonna have to take a term off in three more years because we're term limited, we can only have only serve two back to back terms.

So I'm still. Working hard to make sure that the people who are excited about the project and invested in the project have the ability to step in as we have to cycle out. Because the project needs leadership. Mm-hmm. Every open source project needs leadership and needs invested leaders. I just wanna make sure that it's not, I never wanna set up a project that's gonna be dependent on the.

Right.

If for whatever reason I win the lotto or get really sick or like whatever, whatever the thing is, right. The bus factor. If I'm, if I have to step away mm-hmm. I don't want the project to fail just because I'm not here. I haven't done a good job if, if that ends up happening.

Jonathan: Yeah. Yeah. Absolutely. Alright, so we we've just celebrated some mind milestones for Almo Linux. We've got 9.6 and 10 coming out Yeah. At about the same time. Mm-hmm. Must have been a, a kind of a, a wild month or two. Trying to get all of that ready.

benny: Yeah, it always is. Yeah. It always, as we so Red Hat. Has historic done either a week or two apart.

Every time they do a release this time there was something up with the, the way that they were releasing.

Mm-hmm.

And they released 10 at Red Hat Summit. But it, it was like, they didn't do any of the announcements. There was no, there were no release notes. There were no like nothing, but they announced it.

Event. And we were like, I what to do. Like we're ready to go, but should I should, should we go? Should we not go? I and so. They, and they did nine, six the same day.

Jonathan: Oh, fun.

benny: Yeah. So we ended up doing nine six first because we felt really confident about that. Mm-hmm. And then we finally got ahold of somebody inside Red Hat who said, oh it's just honestly a clerical problem.

It's fine. We were like, okay, cool. So then we did.

Jonathan: That was, that was kind of something I was wondering about, like do you have inside cont is there a back channel between Alma Linux and Red Hat? Sounds like there is now.

benny: I mean, yeah. We, we've always had connections inside Red Hat where we've been in the community for a really long time.

Right. The official back channel is from me to a couple the leaders over there, but mm-hmm. There's all kinds of, you know, inside baseball, we've been front Jack who's on our board, used to work at Red Hat for years and years. Yeah, for sure. We've got connections and friends there.

Jonathan: Yeah. And so what what's the, what's the big headlines with Almo Linux 10?

Like, so, okay, before I even ask that, what was kitten?

benny: So I'm gonna answer that in two ways. The first one is when Red Hat shifted where they were storing their source code. In 2023, we opted to shift the upstream that we were using to send to a stream.

Mm-hmm.

That meant that we're still building alongside Red Hat.

We're still matching the versions. We're still releasing at the same cadences and code. Once we started building sent to us or sent to us Almo Linux 10 we, and like we knew it was coming, we said, you know, we kind of wanna get started setting up everything to do with Almo Linux 10 ahead of time, and we're kind of gonna have this build of Linux that is 10, but like six months early because Cento a Stream 10 was out.

So we knew that we could just like pull all of that data and start building it and using that as our upstream has been really, really useful. And then we had some people come to us and say, Hey, we wanna do, do you have a look at what Lin 10 is gonna be yet? Because we wanna start adding support for, for Lin 10 in our software.

We wanna start you know, people that are building based on Lin 10 or Linux. They wanted to get an early look at what Lin 10 was gonna be, and we said, yeah, I mean, kind of we, we, we've been building it, but we don't, it's not like a release. And so then we decided it was gonna be a release. So the people that wanted to use it to start working on those things had access to it.

And we, when we were naming it we came up with all of the, the traditional names, right? The, the dev and all of the silly, the ones that everybody would kind of. But I also put on the list kitten because all of our all of our releases have a code name that is a color and a cat.

And

so I was like, it's, it's the small, it's the small cat that's gonna grow.

The kitten gonna grow into the cat. And at first everybody hated it. And then once we had knocked off all of the other ideas that were on the list, for one reason or another, we ended up with kitten. So kitten acts as our upstream for what we're building for 10. And it gave everybody early access to what analytics 10 would be.

Jonathan: Yeah. Yeah, it, it was it was pretty neat to see it out there. Unfortunately when I went to grab it and try to run it on something, it, it was a, a, a weird arm board and it didn't work. The boards were, the builds were not quite there yet. And unfortunately I quickly ran outta time on that project to try to actually get it to compile, because yeah, repeatedly compiling the Linux kernel is just.

It's on my bingo car, but I, I didn't, I was not able to check it off this time.

benny: I get it. Trust me.

Jonathan: It's just, it's just a sign. I need a faster desktop to be able to do the builds faster.

benny: Yeah.

Jonathan: I'd be able to get so much more done.

benny: The, the problem, the problem is that you are limited by your technology.

Exactly. And that means that you have to improve the technology.

Jonathan: Of course. That is exactly what that means. So. Alma Linux in itself. Like, what's the, what's the difference between Alma Linux and Upstream Red Hat? And I, I know that, you know, it was based on CentOS and CentOS was basically a bit for bit rebuild.

Mm-hmm. Alma Linux is less so now. What are, what are sort of the, the, the big differences that people should be aware of?

benny: So what we have done is kind of take. Separate from Red Hat. Mm-hmm. Like alongside, still compatible, still matching software versions and all of that kind of stuff, like I said. But we have a lot more freedom because we don't have to match Red Hat one for one.

And we can still do things that our community needs without having to, to follow Red Hat. Exactly. And we are super taking advantage of that with 10. With obviously that V two bill that we talked about, but we also did things like support because Yeah. Unsupported. So and, you know, people needed, we had multiple requests constantly, people asking for, so we did that.

We enabled frame pointers and actually you, you mentioned s. They just did a bunch of benchmarking of Lin 10 versus Rocky and rl, and one of the biggest complaints or I guess, concerns around us enabling frame pointers was that they, it would slow everything down, but this, the, the benchmarking that.

Yeah,

so that's pretty nice. We also added a check tech preview of support for IBM Power. It is very, very close to actually ready. Mm-hmm. But there were a couple things still wrong that we didn't wanna, we didn't wanna say it was supported with, like, knowing that there were bugs. Right. So that'll hit full, full release in, or full support in.

Jonathan: Yeah. Very cool. Yeah. So what, what does that bring the platform list up to now for all Ma Linux X 86 64? Of course. Arm, arm risk five is not there quite yet. Officially

benny: not quite yet, no.

Jonathan: Okay.

benny: And then we've got S3 90 and Power pc. Mm-hmm. Look at me coming up with the fullest.

Jonathan: Yeah. Nice. S3.

What's now the S3 90? Is that that That's an IBM thing, right? That's one of their, yeah. Big, big iron machines.

benny: Yeah. Those are all the, the like massive, there's a word that I'm not coming up with that is not supercomputer, but that's what I'm gonna call it.

Jonathan: Yeah, I know what you mean. Yeah, I can't think of it either.

It's the opposite of a workstation.

benny: Exactly.

Jonathan: Big, big machine.

benny: Yeah, we're too old.

Jonathan: Oh my goodness. It's bad when I have tip of the tongue syndrome at the same time as the guest does.

benny: Yeah. Right. At least normally you can, you can count on one of us to come up with the word.

Jonathan: Yeah. Anyway and, and then so one thing that I've never fully understood is the point release.

Red Hat. Mm-hmm. And Almo Linux does it as well. Mm-hmm. It's because I'm used to, I'm used to running other districts. I run fedora on my main machine and there is no idea of point releases there. Yep. What, what is the deal with the point release?

benny: So it's kind of a, there's gonna be a lot of people mad at me when I say this right.

But it is you know, there, there are some software version upgrades. Updates. Updates. There are some like minor feature additions, that kind of stuff, but it's mostly, from what I can tell, eh we, we, look, we did more work. It's worth it from the Red Hat perspective to keep paying for this operating system because it security.

So like if you're

mm-hmm. When I say

specific versions, like the version isn't gonna much the, you know, the base level. Software versions aren't gonna change much. Ah.

So

as an enterprise, I can count on stability there, and I know that my, my code is still gonna work that as it's running on this. But there are some feature updates, there are some minor like I said, minor version updates that will bring in more, more functionality.

Mm-hmm.

Jonathan: So someone can say, I need to stay at 9.5 and not go to the 9.6 update because I need this exact version of Pearl.

benny: Kind of, they won't. There's this concept of long-term support in Red Hat. If you go with Red Hat specifically, they do like extended support. For the minor versions we don't, but the, the at red.

All of our support partners would help if you needed to stay on a minor version of Linux. They'll do, especially Cloud Linux. They do like security updates for extended support for minor versions.

Yeah,

there's one version. My brain is fi. I, I just got back from travel and my brain is fighting me today.

Jonathan: Travel fog is real, right.

benny: I think it was nine three. Somebody's gonna correct me if I'm wrong, but there was a a version of Linux that got FIP certified. Ah. And so obviously, like we're not, we're not shipping security releases for that version version, but Cloud Linux will. So if you need. FIP certification along with security updates, which I assume you, you do need security updates, then you want, you wanna go talk to them.

Jonathan: Certification not too useful without them.

benny: Right. You've got nine six will also be fip certifi, so. Oh, very. Yeah. Yeah. If you need it, you can go get support from Cloud Linux long term and still have Linux.

Jonathan: Mm-hmm.

benny: And FIT certification.

Jonathan: Yeah. Yeah. That's that's, that's really cool to see. Now you guys have taken the opportunity that you're not bit for, bit compatible with Red Hat to do some other interesting things.

I've seen like push bug fixes and even security fixes out first. Yes. But very fascinated to see this.

benny: It's, again, we're serving our community. Right. The, the first time we did it. Was directly after Red Hat had changed where had forced us to change where we were pulling from. And there was this massive security flaw that the fix was tested and released upstream.

We all knew that it could be patched. Red Hat was dragging their feet for some reason.

Yeah.

And we were like, we, you know what? We're not doing it. We're just going so pulled.

It was the first time that we had deviated from Red Hat and we were terrified. Everything came through fine, and we've continued to do that. So if there's a bug that gets especially, you know, a massive security flaw that gets patched and there's like a coordinated release, we get to participate in that because it's, we, we don't have to wait for Red Hat.

But the cool thing is if they, once Red Hat does. Thing we revert to what they've done.

Mm-hmm.

So we're not deviating heavily or like exponentially over time. It's just in the short term we've got this patch out. It's meaning, meaning that ly systems are not vulnerable anymore. But then we're still staying in line with Red Hat, without without having to wait.

Jonathan: Right. Right. I, I re I remember, I'm trying to remember exactly what it was, but it was a fair, it was a fairly big security thing and it was very odd that Red Hat was just dragging their feet on it. Yeah. So I'm gonna, I'm gonna put on my conspiracy theorist hat for just a second and say, and no,

benny: I love it.

Jonathan: I am aware that I'm putting this hat on, but I will still go ahead and make the comment. Back when Microsoft used to do this all the time, my thought then was. They've gotten a letter from the NSA that says, we are aware of this bug. Do not patch it until this date. And it would surprise me if Red Hat was doing that.

And at the same time, it would not surprise me because if you get one of those national security letters, you're not allowed to tell anybody about it.

benny: Oh yeah, for sure. As, as the person that used to get the, the, like, the knock on the door from the b.

Jonathan: That sort of thing does happen.

benny: Okay. Yeah, it does happen. It for sure happens. It absolutely

Jonathan: does happen. I don't, I don't, I don't know for sure that Microsoft has ever delayed, or any vendor for that matter, has delayed pushing patches out. But I would guess that it has happened at least once to someone

benny: it has to have.

Right. Like that's, that's too much of a. It's too much of a tantalizing option mm-hmm. For them not to have ever been able to take, take advantage of it.

Jonathan: Right. Right. And, and the, the, the place where that really gets iffy is you start thinking then, well, so then do, do these special services places like the NSA, do they have people on payroll that work inside of.

Companies that tip them off,

benny: I wanna get paid twice.

Can you imagine?

Jonathan: Oh, I mean, it's, it's, it's a huge ethical concern, but yeah, that'd be fun.

benny: Yeah, you're right. I like, I'll get there eventually, but I'm Capricorn through and through. I.

Oh man, that sounds amazing.

Jonathan: It'd be a pretty sweet gig. Alright, so that is, that is my conspiracy hat thinking. I love it. I'll take, I'll take that hat back off for now. Yeah.

benny: I mean, for that matter. No, nevermind. I was, I was gonna push it further and say if, if they wanted to do that, then they could just go to the project and say, assuming it's US based project.

In some way. Well,

Jonathan: well, so it's interesting. It's interesting you say that. And, and so we were, we were having fun with that, but that has happened. Oh yeah. Not, well, that's, that was the SSH thing and lib. That was someone impersonating a project developer. Yeah. Got to the point of being a trusted developer that could make releases and then pushed extra code into the release.

Mm-hmm. And you know, a, a Microsoft employee. It's funny, a Microsoft employee found that. Yeah. Because suddenly his server was taking a few extra seconds to SSH into it. He's like, what's going on? And the guy, I love that. Oh yeah. It's, it's a great story. Everything about it. It's a great story.

benny: Such a.

That such a good example of what can happen in open source. Mm-hmm. When we neglect our developers, like if we actually pay attention to people and give them the support that they need, and whether that's financial or just work, then they can't be taken advantage of like this. Mm-hmm. And it's also made us like super hypervigilant at, there's always people coming in.

If they're acting odd. We all immediately key into it. Like this one feels weird.

Jonathan: Yeah. So, you know, Debian, for the longest time, I think they probably still do, Debian had this like archaic really weird policies, like to become a Debian developer or package maintainer, I forget the exact term they used.

You have to meet up with an existing developer in person to get your key signed. And I was like, oh man, that onboarding pro process is terrible. And now I look back at that and I'm like. They knew what was coming.

benny: This point gets access to build and release. Mm-hmm. Extremely vetted.

Jonathan: Mm-hmm.

benny: Like extremely vetted.

Jonathan: Yeah. Well, I mean, you, you have to, I mean, so the, the, the Lib Z thing was just sort of the, for those that weren't paying attention and didn't realize what was going on, that was sort of the, the, the eye-opening movement. But I mean, you think about how many computers around the world are running Almo Linux.

benny: Yeah.

Jonathan: Or run. So we, we talked to is it. I wanna say Joshua Steinberg, that might not be the right name, but the developer behind Lib Curl I think his name is Joshua. Let's see if I remember that right.

benny: Tell me the answer. Computer.

Jonathan: Yes. Yes. Daniel Stenberg. Okay. I was half right. 50%. 50 50. Batten.

50 50 on that one. Yeah. I'll take, it'll take it. The bad thing is I was pretty sure Joshua was right and I wasn't sure about the last name. But anyway, so he, we, we interviewed him and he's like. Lib curl is running on a billion computers around the world at least.

benny: Yeah.

Jonathan: And the, when we asked him about it, I'm like, how do you handle that pressure knowing that it's running on that?

So then you think about it and it's like, well, okay, so what is, if you were to try to take this and put it in a dollar amount, how much money would it be worth to run arbitrary code as a part of that lib curl? Install base.

benny: Exactly.

Jonathan: And that is a very large number.

benny: Yeah.

Jonathan: And yeah, so I, the same thing has to be true of Almo Linux.

Mm-hmm. You know, I don't know that it's in. It is not in as many places as Lib curl is. No. If only because every copy of Almo Linux has lib curl as a part of it. Exactly. You know, the, the, the set one set completely contains the other. But still there are a bunch of servers with a bunch a lot of internet access for people that want to DDoS with a lot of.

Information that might be secure or, you know private. And so, you know, if you were to try to put a dollar amount on that, like how much would it be worth for someone to inject something malicious into Alma Linux?

benny: It's terrifying. And, and you have to think about it, like it's not just the number of installs, but where it's installed.

Mm-hmm.

I, I know that we have around 1.6 million devices calling home just to our mirror system. So those are only the ones that are calling home for updates to the mirror system that we manage, we control.

Mm-hmm.

Anybody that's using their own mirror, which is a ton of people

mm-hmm.

Are, they're not, I don't.

Based on conversations at conferences, we're looking at around 3 million. But that's still, like, that's, it's a lot of devices, but it's also, you know, places like the US Army.

Sure.

Chevron and. Like I, I know of a number of banks that I won't name because that might not be great, but sure. Like there, the places matter just as much as the number of devices, right?

Because if you can get root access to those devices and they're, you know, behind something because they'll, their mirrors are gonna sink to our mirrors and then you'll get access to the devices that aren't even using our, our stuff that then you can start sniffing traffic, you can start doing some scary stuff.

Jonathan: Yeah. Yeah. Very, very interesting. Those are the things

benny: that keep me up at night.

Jonathan: I'm sorry. It's okay. I'm glad somebody is worried about it, but I'm sorry. It has to be you.

benny: Yeah, I get it. I appreciate that.

Jonathan: All right let's see. We talked about elections, we talked about processor types. What what haven't we talked about yet?

What, what, what else is going on at Alma Linux that people will wanna know about?

benny: Shameless plug. Sure. For anybody that is in or will be in v.

Hosting an Alma Linux day. So we do these events called Alma Linux days. They're almost, they, they are all driven by somebody in the community wanting it, it to happen. Mm-hmm. We've had three so far. This will be our fourth. The, we've had two in Tokyo and one in Germany. The This one is centered on the VFX community because it's adjacent to an event called siggraph.

Oh.

We have

a huge amount of adoption in the VFX community that I honestly didn't know about. It's one of my favorite things to go to an event and have somebody walk up to me and say, oh, we love you guys. We use you. And I had no idea because it's just one more place that I. Serving a need and it makes me happy.

Mm-hmm.

But,

We went to SIGGRAPH for the first time last year and the response was insane. It was awesome. This company that, I'm gonna show my ignorance here, there's a, a company called ILM or Industrial Light Magic. Yes. I. Did not know who it was. And they walked up to me. There was some, one of their employees walked up to me at, at one of the evening events and said, Hey, I'm from, I lm and we use you guys.

We just actually replaced you or used you on all of our desktops and we're moving to you on our render clusters and yada yada yada. And I was like, oh, that's great.

I was at that moment wearing earrings that are the millennium fuck. So like, like I am certain he expected a very different reaction. And then the next day, Tristan is like one of the big guys that has been helping me in the V FX industry, like serve the.

I was, I was like, do you know this company ILM, like, and he was like, I'm sorry, what? I was like, oh, I, whoops. Okay.

Jonathan: Okay. In fairness, I don't know that I would've immediately put together that ILM is industrial light and magic. I definitely, I definitely know who industrial light and magic is. Right. But putting it together from the acronym that, that is pretty understandable.

benny: I didn't know either of them.

Jonathan: Mm-hmm.

benny: I am, I was completely ignorant. I have been so far removed from anything VFX or like movie production or anything like that, that I had no idea. But anyway, back to on the Linux Day in Vancouver is centered on the V FX community because that's who, that's the event that we're next to, and that's what we're.

With this one.

Mm-hmm.

We've done the web hosting industry in Germany, in Tokyo. It continues to be about educating the, the open source community about Lin and open source in general. Mm-hmm. Like the software community about open source in general. And it's really fun for me because it is one of those.

Like one of those moments where you get to really connect with people who directly benefit from the work that you're doing. It's very fun. So August 9th, free, free registration. If you are close to Vancouver, come hang out.

Jonathan: Yeah. I'm curious, so this, this brings something to mind. You mentioned educating folks in, in Japan about this.

Is there any uptake of almo Linux in, in mainland China and the long soon architecture? Is that what they call it?

benny: I don't know. That's a good question.

Jonathan: And you know the,

benny: I know if I look at our like mirror heat map mm-hmm. There are certainly almo installs in China. I wonder, I don't know anything about that architecture, so I'd have to look.

Jonathan: I I think it's the, the long soon, is that what they call it? I'm trying to remember now. It's, it's something like that. Yeah, long soon. Name of the journal purpose, MIPS based computer architecture. Yeah, I'm just, and of course, so yeah, all Linux is based in the US isn't it? It's where it's a, yeah. 8 0 1 C six you said?

Three. Oh

benny: yeah. 5 0 1. C six

Jonathan: five. Oh, you're not, it's okay. You're not wifi.

benny: Exactly. Yeah.

Jonathan: Too many numbers. Yeah.

benny: We're incorporated in the US but Right.

Jonathan: So like trying to even try to reach out to Chinese companies could be challenging, especially these days.

benny: Yeah. There are a number of countries right now where finding the appropriate.

Type of contacts

Jonathan: mm-hmm.

benny: Is a challenge.

Jonathan: Yes. There's like, there's, there's the there's list of questions you have to ask. All right. Have you ever been approached by this person? Have you ever worked with one of these organizations? Have you ever done this? Okay. We can have a conversation.

benny: Okay. Now we can talk.

Okay. Yes. Cool.

Jonathan: Yes. I, I'm going through some of that too. Yeah, I bet. Yeah. It's fun. It's it, no, it's not fun. I, I, I wish we didn't have to do it, but there's a lot of things about the world that I wish that is not the way it is. Good. All right. Interesting. Okay, so we've talked, we've talked about a lot of stuff.

It's been a lot of fun. Yeah. I'll ask once again, is there anything that we haven't covered that we need to? I don't think so. Need to let folks know about. Alright. Well then I've gotta ask you a couple of final questions.

benny: Yes.

Jonathan: And that is,

benny: are these one that I answered before too?

Jonathan: Probably.

benny: I hope I have the same answers.

Jonathan: It's fine if they change people's answers change all the time. Do you have a favorite text editor and or scripting language? I.

benny: So text editor is I, okay. The reason I'm giving you this big sigh is because it's a text editor that I found out recently is no longer actively supported, so I'm gonna have to change to something else.

But it's called Text Meet.

Ah.

I have used it for a decade and I'm really sad. Like I had a feature request and so I went to email the address that was on the website and it bounced. And I was like, oh, that's not a good sign. And I did some digging and it seems like it hasn't been supported for a while.

I'm surprised it continues to work, but that's also a testament to the software that was built. Yeah. Like so now I'm gonna have to change, but that's been my favorite for a number of years.

Jonathan: Yeah. And then scripting, scripting language.

benny: Oh, bash forever.

Jonathan: Bash,

benny: you know, I'm a sis, admitted heart. Yeah. So like Bash forever,

Jonathan: we we, we interviewed the guy that wrote Bash and I'm not gonna go to come up with his name.

We interviewed him though, and at the end of it I was, I was doing it and at the end of it, I, you know, we go to ask that question and I'm like, I don't know for sure that bash counts as his scripting language. And Oh, I offended him. He was like, of course it does.

So ever, ever since then, start fights? Yeah. I, I, I try to avoid the drama, but man, sometimes it just comes naturally. So Bash is a perfectly acceptable answer.

benny: Fantastic.

Jonathan: I have it on good authority. You have it on good authority? Yes. Alright. Vinny, it has been delightful. Thank you so much for coming and hanging out with us and we will, we will have to do it again maybe about the time Almo ly 11 drops.

We'll come back and we'll do it again. I like it. The world will be very different by then, but that's fine. I'm sure it'll give us lots to talk about. I'm certain. All right. This has been this has been Benny Benny Vasquez talking about Alma Lennox and all the stuff going on there. Been a lot of fun. We don't have a guest lined up yet for next week, so if you have a project that you want us to talk about, let us know.

You can email us at. floss@hackaday.com, I believe is the email address. If that doesn't work, come tag me on Twitter or you know, the various places where I hang out, you could find me. There's also the security column goes live every Friday morning on Hackaday. You can go and check that out. And then we do have the other show Untitled, the Untitled Linux Show over at twit twit TV slash uls.

And we get up to all kinds of fun shenanigans there as well. You don't wanna miss it. Appreciate everybody that watches and listens. Those get a. Get us live and on the download and we will see you next time.