CHRIS:
Welcome to the Elixir Outlaws, the hallway track of the Elixir community.
AMOS:
Hey James!
JAMES:
How's it going?
AMOS:
Good. James looks more serious than the rest of us. That set up he's got.
ANNA:
I know, right? That is an awesome set up.
CHRIS:
Is my mic working?
AMOS:
Yes, it is now.
CHRIS:
Okay, Cool. I was talking to Amos for like 10 minutes and he was refusing to acknowledge me.
AMOS:
Well, I try not to acknowledge you in public. I don't want people to know I know you.
CHRIS:
It's bad for all of us.
AMOS:
Oh, James has a Kansas city shirt on.
JAMES:
I do have a Kansas City shirt on. Yeah. I can hear you, Anna but you're a little quiet.
ANNA:
All right. Thank you. Is that better?
JAMES:
I would still say a little quiet.
AMOS:
Yep. That's that every time you use that mic, you have to get really close to it.
ANNA:
Testing... That better?
AMOS:
That's a lot better.
CHRIS:
This, this whole setup process is just part of the show at this point.
JAMES:
Is this the show?
AMOS:
Yes. This is the show.
JAMES:
Is it too soon to make that joke?
ANNA:
No, that was perfect timing.
ANNA:
How's it going, everybody?
CHRIS:
They're going well, Anna, how are things going for you Amos? This is Chris speaking.
AMOS:
Chris keeps giving me books. Or suggesting books. So, um, I'm not getting anything done. And one of the books that he suggested to me, he was getting things done, which is just put off more stuff.
JAMES:
Amazing.
AMOS:
And, uh, the philosophy of design is another one, um, which has been, uh, a great transition out of Pivotus for awhile because you, you know how that one's going.
JAMES:
Amos is going to need a support group for that book.
AMOS:
I am. Yeah.
CHRIS:
Hey Amos. This is Chris again. I'm making sure that people understand who we all are, but you want to, uh, do you want to introduce our guests, which is a show first for having a guest on.
JAMES:
Have you not had guests before?
CHRIS:
We've had guests. We Just never introduced them.
ANNA:
It's going to be a show first. Perfect. Good job Chris.
CHRIS:
Yeah. Sorry. No, no, no, actually like being courteous to our guests and like introducing who they are, as opposed to just assuming or like everybody's clued in.
AMOS:
Uh, so, so today, uh, on the show we have James Edward Gray, the second. James, uh, has written a book. It was pretty prolific in the Ruby community and is doing elixir. And, uh, is involved in a conference coming up that we might be involved in. Maybe, uh, James, you wanna talk a little bit about yourself and that kind of thing
JAMES:
You guys are going to the conference?
AMOS:
Well, it depends on if Chris's wife has baby or not.
CHRIS:
That's true.
AMOS:
I might be watching his children while he's going to a hospital.
JAMES:
Holy cow. Wow.
CHRIS:
That's true. Yeah.
JAMES:
That's exciting.
CHRIS:
That's a non there's a non-zero chance that that might happen.
JAMES:
Yeah, Amos. That's who I am, I was really active in the Ruby community for many years. And if you listen to podcasts, I did the Ruby Rouges podcast for like three years. Um, and now I spend most of my time in the Elixir community doing Elixir-y things. And I recently had dinner with Chris and Amos. Anna Wasn't there.
AMOS:
We missed you, Anna. We had a great dinner and a great time there at Stragne Loop. Learned quite a few things. And then, um, the Erlang ICFP was, was pretty great. James, you, you made it to, I think just the first talk?
Speaker 2:
For that first tier, the morning, then I had to drive back.
AMOS:
It's a long drive.
JAMES:
Yeah, for sure.
AMOS:
What do you guys have on your mind today, I know that James sent out a few things that he wanted to talk about.
JAMES:
I brought questions.
AMOS:
I'm sure Chris will answer them for you.
CHRIS:
Oh yeah, yeah.
ANNA:
Chris is ready. He's wired, he's had coffee.
CHRIS:
Um, um, this is probably one too many coffees. If we're being honest.
JAMES:
Perfect for a podcast.
ANNA:
It's not the first time that's happened. I feel like we hear the one too many coffees situation often.
CHRIS:
It's because of the podcast. Like I intentionally do that. It's unhealthy for sure. But it's part of it.
AMOS:
When you can't get a good aerobic workout for your heart, just drink a lot of coffee.
ANNA:
There we go. It replaces the need for a good aerobic workout. Right. That's how, that's how that works.
JAMES:
All right. So can I ask questions?
CHRIS:
Yes, please. Yeah.
New Speaker:
Okay. I am going to give this talk. At Gig City Elixir and it's about using less code to solve our problems. Um, but along the way I touch on, I wanted to touch on several questions. So I'll set this up with a few questions. Let's say, I assumed that we would all on this podcast, agree that if I needed some functionality that exists in a heavy dependency, it would be like a big dependency with lots of things, but I only need a little bit of the functionality so I can either bring in this big dependency or I can just write, let's say like 20 lines of code. I assume we would all agree that I should just write the 20 lines of code. Does that seem fair?
AMOS:
Yes.
ANNA:
Yes.
CHRIS:
Yes.
JAMES:
So now let's go. The other way. I also assume we would all agree. I'll, I'll try to use a Chris example here to do raft consensus
AMOS:
That we may be going down a dark path.
CHRIS:
No, no, no. This is good. Let's do this.
JAMES:
This could take a turn for the wrose. If I had to get raft consensus, then in that case, I should probably go ahead and use the library or framework or whatever, because it's too error prone, and it's just too likely that I'm going to screw something important. Right.
AMOS:
And hopefully some free maintenance along the way that like, I don't know enough about raft. So, um, I'm hoping somebody else will take care of that part for me.
JAMES:
Exactly. Right. Okay. So here's, here's the actual question. This is like a continuum, right? Uh, on one end we resolved the problem ourselves and on the other end, lean on a dependency for it, the maintenance, for solving the problems we don't understand or, or stuff like that. And my question is most of the time, like where should we be aiming to land on this continuum? Like, do we want to be like right in the middle? Do we want to be a little closer to one side the other side? You know what I'm saying?
CHRIS:
Who, who wants to go? Cause I'm ready. I've got an answer. I don't want to be. I pushed back on these stereotypes that you all have created for me. Okay?
AMOS:
You created these yourself. What are you talking about?
CHRIS:
So in my mind, the thing that you're driving towards, if I was going to like, um, kind of like shape or frame the problem in a, in a specific way, and you can tell me if I'm off base on this framing, bringing in a dependency is a point of coupling. And we know that we could sort of take it as read. Coupling is bad. Like most of the best practices that exists in software development are in a lot of ways to like help reduce coupling because coupling keeps us from being able to move forward and change things and all that kind of stuff. So any dependency you bring in is a point of coupling. So to me, in my mind, the power of the abstraction, the power of the dependency, which is an ephemeral word that I don't have a good definition for at the moment, but like the power of it has to outweigh the cost of the, of that point of coupling. Is that a, would that be like a fair framing of, of, of, of the question some degree?
JAMES:
Oh yeah. I think so.
ANNA:
And how do you define when it does outweigh?
JAMES:
Yeah. That's right. Good question.
CHRIS:
Yeah. I think that's going to be, it's tough. I think that's going to be very dependent on where your project is in its like life and the context of the people that you have on your team working on it and yeah.
ANNA:
Can you say more about that? Depends on when, where the project is in its life.
CHRIS:
I don't want to - I battle internally with us all the time, because I've definitely worked with, I mean, as you have, right. And I'm like, uh, at C5, like you see plenty of early stage companies that are like, I need something I've got three months to like get a thing out into the wild. And I don't even know if this idea is good or not yet. And I just need something out there to even just prove that like I could make a business out of this.
JAMES:
Otherwise we run out of runway. Right?
CHRIS:
Yeah. Otherwise we literally don't have a business. And as a consultant, like, I feel like your job is to enable that like your job is to do that is to allow them to figure those things out. And if that means like cutting corners, bringing in extra, I mean, I think this is like, you see this in the proliferation of like Ruby gems, right. Not to like, I'm not, not to bag on Ruby or whatever. Like you see this in proliferation of lots of libraries and lots of ecosystems, like libraries that do things that arguably you should be doing but that you can defer to somebody else. Hey. Yeah. I mean, Left Pad is like the funny example. Right. But like, there's plenty of things that I actually think that, you know, you don't need a gem for whatever, but maybe you just do it because that'll like, you know, that cuts a couple of days off of your timeline. And if all you've got is days, like if you can measure your, your runway in days, those days matter.
Speaker 3:
To me, one thing that matters is what is your current domain knowledge of the thing that you might be pulling in? Do you have the time to put into it and the domain knowledge to put into it, or is it going to be a huge learning thing? So, because Chris asked me reading philosophy of software design, the first thing I thought of was he puts in a, uh, uh, complexity, um, function in here. And, uh, I think that it kind of comes down to the complexity of the problem, which, uh, he puts in here as, um, the complexity of each part that you're pulling in, uh, times the fraction of time that a developer is going to spend on working on that part. So, you know, if you're going to, it might be super easy, but let's say it's going to take four weeks to do. Maybe you don't have that kind of time or it's going to be really, really complex, but take a little bit less time. So I think that you have this complexity versus time thing that you have to think about whenever you're deciding to pull something in, and maybe your time is better spent just reviewing the code of a dependency that you might pull in to see if you think that it's worth pulling in or not
JAMES:
By going through the dependency and seeing how much of its paradigm it's going to impose on your code basically.
AMOS:
Right? Yep. And then can you protect yourself from that future change too? Like if the interface isn't that very big, would it be easy for me to wrap it inside of my own interface? So I can hide that a little bit, that, that coupling into one place so that maybe I can switch it out later for my own. It depends on like how littered in the code it's going to be, how prolific in the code are you depending on this thing.
ANA:
And also, I mean, I think time plays a big factor in that complexity component, right? Because how much of this talk about complexity depends actually very, in a very real way. On how much time you have to press point about getting something out for like, if you have to get it out within a few weeks, will there be more complexity because you have less time or is there a way that you can mitigate that? Right.
CHRIS:
I think a big part of it is, um, it's also the, the, and this, this comes down more to, uh, experience in just, you know, doing this a lot and kind of understanding, you know, getting a feel for these sorts of things. But I think that there's, you know, the, the time part of it is also like, what is the time spent? What, what not even like, what is the time you're going to spend upfront, but like how many touch points are going to be in the rest of the code? How much, how many, how often are you going to look at thing? Because in his equation where he's saying, you know, the total complexity of a system has all the individual pieces times the number of time you spend working on it, it's not even just upfront, it's like over the course of the thing. So if you can hide a thing that's as good as removing that complexity, like you don't deal with it anymore, um, from a cognitive burden standpoint. Yep.
JAMES:
So the interesting thing to me, it sounds like you're re almost, so I agree back up front, you know, when you have to move fast, the dependencies can be really valuable. So then when the app enters like the mid to late life stage, do we then go back and remove those dependencies in favor of replacing them with their own.
AMOS:
I knew that question was coming. Its a really good question.
ANNA:
Has anybody actually had an experience of that happening?
AMOS:
Yes.
JAMES:
But I wouldn't say it's common
CHRIS:
And it wasn't fun.
AMOS:
It was a very painful process.
ANNA:
What led you all to decide to do that then? Other than a dependency maybe is no longer supported or...?
AMOS:
Yeah. Yeah. Um, so that's what I going to say first. It was unmaintained dependency was the first example that I can think of. And another one that, um, was that it, it, we had certain performance characteristics that we had to meet that the dependency wasn't meeting, because it was more general than actually what we needed. We needed something. Maybe I would call it like a very surgical strike. I just needed to pull the appendix out. I that, which doesn't mean I had to cut the entire chest open. And that's a terrible analogy.
ANNA:
No, I mean, but essentially by removing the dependency, even though you needed it, initially you are making, you are lessening the complexity of your system, right. Because you are only essentially reintroducing the piece of complexity that you need.
AMOS:
But the, the big thing that made it hard to pull out was we did not wrap our dependency. So we had calls to this everywhere, and I'm not sure, I'm not sure how much that would have changed if we did wrap it because when you wrap it and you're looking at one interface and then you try to change it to another interface, unless you did a lot of upfront thinking, which when you're in that hurry runway, I'm running out of runway phase - you often, aren't doing as much thinking about what are the repercussions of the interface itself. And so when you do wrap it, a lot of times, it's really a facade of the exact same interface, which is not helpful.
CHRIS:
We had an, um, a project once at a C5 where, um, along, you know, previously, some other people had added, um, I'm trying to hear what the gem is called, whatever. It doesn't matter. I don't want to malign like their gem or whatever, but they had brought in like a gem to do, um, uh, authorization type stuff. And so they had all these rules and they had all these. And it was like all these classes about like how they did authorization and all those sorts of things. And there came a point where we realized the business wanted these really rich, interesting rules about, you know, enrolls and who could be assigned to like work on certain things. And like that it was this really complicated thing. But like, that was how their business worked. They needed to be able to do that, like for their business. And this gem is designed to be this like very conventional off the shelf thing. And we literally could not like shoe horn. I mean, we could have spent a lot of time shoehorning it in there, but it became untenable to continue to like shoehorn these like really complicated rules specific to that business, uh, into this library. And at that point, it was like super hard to start pulling all that stuff out of there because it's everywhere. It's like littered throughout the code and it's broken all these and the, and the way that those that gem wants to work is it like once hooks into all of your models and it wants hooks into all of your controllers and it like ties all these bits together, you know, that was, that took far too long and became a huge, it became a major impediment to actually like providing value to that business because like now we were in this place where we had to pay down these debts to, in order to provide value to the business. And to me, that's the place I never want to be. Um, especially as a consultant, definitely as just a programmer in general and as a human, like, I don't want to be in a place where like I'm impeding, or my decisions are impeding the business, which is paying me money to provide value, to like, not be able to provide value. Um,
JAMES:
The business is asking you for features A, B and C, and you're saying, hang on, I'll get to that. As soon as I unbuild this mountain of doom I've constructed.
AMOS:
Isn't that what all design is though, is trying to, trying to, uh, not end up in the mountain of doom?
CHRIS:
We talked about this last week and there's this great Rich Hickey quote that, um, that I just, I love and we're down to all the time, but it's, um, you know, programmers know the benefits of everything and the trade-offs of nothing. And like that to me is what's, epitomise in that, right? It's like we knew all the benefits of using this, like off the shelf thing. And up until we had got to a place where it was no longer a benefit. And then, and now it's like, Oh, well now we're, now we've really painted ourselves into a corner here.
ANNA:
But given that it's so hard to predict that situation sometimes, right? Like how do you mitigate for this increased complexity that then becomes really difficult to remove, right? Like as we're designing systems and as we started, you know, I'm also starting to get to that book. Um, how it's, it's a, it's a really hard thing to do, right. Because we also don't want to like, over-optimize for something that's not going to exist. Right. And we want to solve the problems that we have at hand, but then we get into these scenarios, right. This is not a new problem that we see. I don't know if there's a good answer to that, but...
AMOS:
How about you James? Where do you stand on the answer to your question?
JAMES:
I don't know. I haven't decided yet. Um, I would say, um, I think you kind of hit on the key point when you were talking about, uh, removing something. And I think it's that dependencies in general are always generic. Like they have to be to serve a wider audience. Right. But when you know your problem, what exactly your problem is, you can write a more targeted, you know, specific for your needs thing. And almost surely, that's always going to be superior if you have the ability to do that, because you can solve what Anna just said, the problem at hand, right. The actual problem that you have. Um, and so that's almost always going to be better, but then, you know, when can we spend the time to do that? And when can we not spend the time to do that is a huge part of it. Like, which I think we hit on right off the bat. But, uh, but I want to throw you one more twist if I can. Um, yeah. Okay. So let's do one more twist. At Strange Loop, I had this cool aha moment while I was sitting at a talk about rust and node, it was about rust and node and, um, it was by Ashley Williams, it was a great talk. If you have not seen it, look at that. It's great. But she said at one point that rust doesn't cabinet run time by design, because rest is meant to run on everything, like your toaster. Right. And so it's got to be like the lowest level thing possible, and it doesn't have a run time and that's by design and that's a feature and that's awesome for us, which then led to this kind of aha moment in my head, which is in a Elixir - it's all about the run time, right? It's the beam, all the great things come from the beam, right. And there's run time and this ability to spawn process and have concurrent everything while basically just writing imperative code and stuff like that. So then my question is does Elixir use this equation of the trade-offs of rolling around and using an off the shelf thing. Does Elixir change that equations slightly, like, would you make different decisions in a Elixir on that continuum, then you work and say rust?
AMOS:
That's a hard question. As you can tell by the silence. I don't know that it changes it much for me. Like, I, I appreciate the run time and everything that it gives to me. And I did choose language that had these features instead of having to find a package that has those features. Right. But I don't, I don't know that that changes how I choose to pull in even more dependencies. I am, depending on the runtime because of my language choice. But that doesn't mean that I have to pull in more or less packages for other design decisions or choices. I don't, I don't know that it changes my thoughts on it.
JAMES:
So I think it does change my thoughts on it. And so to give the contrast to what Amos just said, for example, in Ruby, if I build a web app and I have a web application running and now I need to do some background processing, my first thought is go grab, uh, gosh, it's been so long. I can't remember...
ANNA:
Sidekick.
CHRIS:
Sidekick or Rescue - one of those.
JAMES:
Yeah, go grab one of those. And in a Elixir, if I need to do some background processing, that is not my first thought. My first thought is to add a supervisor to the tree, I'm launching processes under the supervisor I'm to go.
AMOS:
So you're coming at it from a different angle than what I was thinking. But so to me, that's something built into language, but are all dependencies like that? Or are they just built in?
JAMES:
I see what you're saying. Yeah.
CHRIS:
Yeah. I, I totally get what you're saying. And I think for me, I'm trying to think of the right way to say this. So like, in my mind, I mean, while we've sort of talked about, like, if you're an early stage company, it's like fine to pull in those dependencies and that kind of stuff. Right. Um, but I want to add like an addendum to that, which is for me, I think my goal over the past, however many years, definitely when I was doing full-time consulting, my goal as a consultant was to make it, so I never had to do that. Like I felt like that was like imperative on me as a, as a developer to get to a point where I could say, I don't need to pull a dependency because like, I've learned a lot about this and I could implement this myself, like, and implement it with the trade-offs of like myself, especially if it's something small. Um, and I felt like that was a thing that I wanted to like, get good at, not in sort of a, like, not invented here kind of way, but like more in like a, if we don't need to do this, but we're doing it out of necessity. Like I don't want necessity to ever drive a decision to like add complexity, if that makes sense. And so I think that dovetails super well with what, with, with like my enjoyment of elixir was all of a sudden I got this runtime and maybe this is kind of what you're getting at. It's like, I got this runtime where it's like, I'm not burdened anymore. Like I'm not burdened by anything I can do. Like, I can solve all these hard problems that were really hard to solve. And other runtimes they're just gone now and I can just build highly concurrent things and I can build stuff that is fault tolerant, like at this like really granular level. Um, and so, yeah, I think in that respect, elixir sort of helped, um, in a lot of ways like fuel that, which maybe isn't, maybe that's not a healthy or mature, um, uh, viewpoint to take, but it definitely like fueled that in me. Um, I think there's some potentially something to that.
JAMES:
Is that what you meant earlier, Chris, when you said it depends on who you have on your team? So if you have a team of people that are very capable of solving these problems in a reasonable amount of time, you would lean more on just solving the problem yourself and instead of playing in a bigger dependency or something like that.
CHRIS:
Yeah, I think so. I mean, and specifically, like if you had a group of people who had domain experience in the specific thing, like you're obviously going to be more inclined to bring, to do that stuff yourself, because like you understand the trade offs that are going to have to take place to do that. Um, and I think that like team that team composition, it's hard to ignore that because if you don't have people who have, so, you know, the subject matter sort of experience hard, won knowledge, you know, you're to be, uh, it's going to be a pretty big time sink to like spend a bunch of time on that. Not to say it's like not worth doing for your edification, but like from the point of view of like providing value to the business, like it may be that you just need to like lean on someone else's experience for a little bit until you gain your own, which is fine. That's like totally reasonable.
ANNA:
I like that a lot. And I think that kind of alludes to it. I think we talked about this a little bit last time, right? Like part of it, part of the complexity is a main and part of it is how we're building the system. But part of it is also exactly what you just said, Chris, like the team and the team's current understanding of the domain and the system, right? Like that definitely either lowers or increases the current state of complexity depending on their understanding. Yeah.
AMOS:
I mean, you can, you can look at the elixir ecosystem and see where, see that people gaining that knowledge. If you look at back a little ways, you start seeing where people are using Reddis from elixir and background cues and starting up all these things that look just like Ruby or, or any other language that you might use. And, and as the community learned more about the runtime itself, that stuff went away,
JAMES:
Now people just fire up NTS or even grab the dependency, right. Con cash or something, which is what I would call smaller dependency, which is just firing at BPS for you. Right.
CHRIS:
I've started using things like registry a lot more, um, because, uh, as opposed to, it depends on the use case, but as opposed to doing, um, using ets on my own, uh, like if I know I'm going to have a lot of things in sets, um, I'll often use registry because they're doing really smart, like bucketing and sharding across ets tables to make lookups really fast. And that like, they've done a ton of work on registry and that's like, uh, so that provides a lot of power, uh, over the normal abstraction that you would have to build on your own. And it's a re like, those are sorts of the things that I lean on more now is sort of these more, I dunno, it's, it's, um, I'm so poisoned by like philosophy of software design. Like my, I look at everything and through that lens now, and it's hard not to look at what he calls deep modules, right. Modules that provide a lot of power, but with a very small interface, like without, without actually doing too much, they're giving you a ton of power. Um, and registry, I think is one of those like registries doing a lot more than just like storing keys and values, but it's like they have this really relatively simple interface to just like put stuff in there. Same with Comcast, like Comcast is doing like smart stuff behind the scenes that are somewhat, you know, non-trivial to manage on your own. Um, and those are, those are good dependencies to like to pull in where, you know, if all of a sudden you were doing less with a shallower interface with like a wider interface with like more functions or whatever, that's, you know, you're naturally going to end up coupled to that thing in like much more gnarly ways. And it's not giving you a ton of power for the cost of that coupling.
JAMES:
So do you say you use registry for like, even things that are outside of its typical usages? Like not pubs, uh, not processed registration and stuff like that?
AMOS:
Yeah. I have the same, same, same thing. That was my mind blown thing was I never thought about using registry for anything except for processes.
CHRIS:
But I mean, if you think about like what, like the process register you're right. Like the point is like, you need to hold like this giant, like list of keys and values - just a big lookup table. And like, registry is really good at that. Uh, being, uh, being really fast, it can handle like tons and tons and tons of inputs, which as it should be able to, cause it's, it's meant for like registering processes, but it's kind of like misnamed, like it's really just a really good key value store local to your box. Like that's like highly efficient.
JAMES:
Those kinds of dependencies in some ways are more powerful, which I think is what you were saying. And I'm just catching up to you is that this dependency that just does this one thing really well, but is pretty narrow. And like you're only going to make a couple of calls to it started to put things in, take things out, um, that is a superior kind of dependency to something like a big framework that wants to take over a large portion of how you write your application or something.
AMOS:
So then why, I think I have an answer to this, but why aren't there more dependencies that are, are thin and deep? There's a small interface, but they're very powerful
JAMES:
In the Ruby community. There's a kind of this small subculture at that sometimes went by the name of the West code movement and they do have a bunch of dependencies like this, um, like, uh, Cuba as a web app. And then there was one that came later, I think, CRO or something like that, that replaced it. And then, um, Ome was like a database dependency. It was actually built off of Reddis, which was unusual, but it was like a complete database layer and under 500 lines of Ruby, I think, uh, and it did like models and add smaller to kind of be active record conveniences and things like that. Um, Mote was a rendering engine kind of similar to ERB and stuff like that. Uh, if I remember correctly, it was like under 50 lines of Ruby character. And like, um, there were these like very focused dependencies, but I wouldn't say they were mainstream. Like you had to get off the beaten path. Right.
CHRIS:
Was like the dry was all the dry RB, um, projects where that was that part of that whole kind of subculture. I remember seeing these, but I never really,
CHRIS:
I don't know. That's a good question. I'm not familiar with the dry RB stuff. What's that
CHRIS:
Like, it's just a bunch of little packages. They kind of do a lot of like functional kind of stuff, in a Haskell vein of functional. Like there's like a dry RV monads and a dry RV functers like, and stuff like that. To me, like, it's like not the stuff you're talking about. Cause I think the stuff you're talking about is, is the right is the right approach. Like you want a thing that does, like you consider a lot of stuff. Um, the dry RB stuff was like two was a little too small in my mind. Like it doesn't give you enough power for like attaching yourself to this, to this, to this thing, to this, to this dependency. That's the word. Um, it, for, for me, like, I don't want type classes as a dependency. Like that's not interesting enough, this isn't enough power for me.
JAMES:
Right. Right. If you have bunkers or something, you have this new tool to build things, but you're still building the thing. Right. Whereas like what you were saying with registering, it was like, Hey they, already built this awesome thing. And it's great at what it does and a couple of things that they didn't even intend it to be great at, you know?
CHRIS:
Yeah. And it sounds like that's the same thing. That's true also of like, um, uh, you said it was Ome the database engine that you're talking about here. Yeah. I mean, those are giving you a lot of power and it's a hard problem. Like storing stuff consistently in a database and like talking and providing the right interface to that is hard. That's a lot of power and that's worth having a dependency for, in my mind, to me in my like personal sort of, um, rubric
ANNA:
I'm a little bit of a larger question, which is James mentioned, you had to really go searching and that type of dependency is a little bit more off the beaten path, even though it's a harder problem to solve and they are deeper dependencies, but what does that say? The fact that those are the dependencies that are off the beaten path, right? Like about how we generally develop software and that most dependencies is the trend is for them to be more general. Right. And is that, is that the right approach?
AMOS:
I think that people gravitate towards that because when you're starting out, I don't know what I'm going to need. So it's easy to grab something like Phoenix and ecto that does almost everything that you can think of. Uh, but, but we, I mean, we do have this in the elixir community. You have, uh, Peter Sexton is a pretty big pusher of, of uh, cracks. Yeah. And, and, and other, um, I call them micro libraries. What's the, uh, what's the one for talking to it only talks to Postgres
CHRIS:
Mobius. Mobius. Yeah. Yeah. That's Ron Connery's jam.
ANNA:
I just think it's interesting the fact that those, that those dependencies exist, but you really do have to go looking for them, which says that most people probably don't go looking for them. Right. And maybe that's maybe that's because at first you don't know what you're building. And so you want something more generic.
JAMES:
There was definitely a pretty Epic thread on the internet at one point about, um, Cuba and whether or not it was okay to use Cuba for like, uh, consulting, web apps and stuff like that. And one side was our game. Yeah. We've done it for years. We've had great success with it. We're doing fine. It lets us, you know, it keep our dependencies manageable. If we ever problem, we can actually crack open the code and read it and have a chance of understanding it and things like that, you know? And that the other side was arguing. Yeah. But you're going against the norms. So like developers who come in, you know, they all are gonna know rails and not Cuba and that, and then they're going to have to learn the differences and you're having to pulling other things from the ecosystem to handle some things that rails handles for you, which then the other side counters with things like, yeah, it's true. But rails also does XML parameters, which we've never, ever needed or wanted or cared about having or things like that. It's a very interesting thing back and forth.
CHRIS:
I have such a hard time with that because I have a such a hard time understanding the mentality of, um, this is what everybody knows. So that makes it good.
ANNA:
It's kind of where I was leading with that.
AMOS:
Not good. It's business-wise. Like, cheaper.
CHRIS:
Well, but I can empathise with like, with like that logical argument, like I understand how you could make that logical argument. I just like fundamentally disagree with it so much so that it's like, it kind of blows my mind that people make that argument like,
ANNA:
Well, I think the point that I was trying to get to, and maybe it wasn't maybe, and this is not, maybe this is not a valid point, but like, as we think about building systems, right. And when we're first starting or we talk about these norms, it's like, okay, we'll just use this thing that does all these things we're not thinking about. I mean, and why are we not starting thinking about, well, what are the things I actually need? What am I trying to achieve with the system? Right. So we do, we lead, we tend to use these more generic tooling then maybe. And then again, like maybe setting, having a set of norms is not bad. I just think it's interesting that we have this paradigm that we're in, that gets repeated over and over again across languages and communities, et cetera.
AMOS:
I mean, look at, look at the other side of it too. So open source software, you're putting yourself out there and as a, as a library author, um, it, it feels good. Even if you're not in it for fame and glory, it feels good when people are using your system. So if you can make it more general and applicable to more people you're to get more, more people using it, more bandwidth, more feedback than if you have this surgical strike library that, that a smaller subset of, of the developers out there are going to use. So the community kind of pushes you mentally to go things that are more general because yeah, like, like, look, there's a Phoenix. Phoenix has got its own talk, every elixir comf all over the world.
ANNA:
Right. But then you think, think about the book you're reading right now. Right? Like it goes like it's not these things aren't bad. I just think it's interesting. Right. We talking about right now and what James is talking about, which I think is really awesome is like we write less code. But when we first start, we talk about, we do have this paradigm using these dependencies that are more generic that inherently generally involve bringing in more code. Right.
AMOS:
And they're probably more junior developers too, which makes it easier to just grab this general tool. Right. Yeah.
ANNA:
No, I totally agree. I just think it's an interesting -
JAMES:
That's actually an interesting question though, to me, is it easier to grab something like rails and teach someone rails where you have to teach resource ragging and a bunch of other things just to get going? Or is it easier to grab something like Cuba and teach them like the three things that Cuba does, you know,
AMOS:
It probably is easier to grab something like Cuba, but they are more likely to have, uh, even if they're like first year developer, they're more likely to have looked at rails or looked at Phoenix versus Cuba.
ANNA:
Well, and I think the trade off there, we're talking about productivity and time, right? Like if you were first picking up rails, you probably don't have a deep understanding of everything that's happening. And if you needed to really, if you were really starting to fight with rails, but something that you were doing, it would be hard to figure out exactly what was happening. But for the most part, you can be up and running quickly. Right. Whereas the smaller dependency, you have a much better chance of early on being like, Oh, I know exactly what this is doing. And if I'm running up, running up against it, I have a much clearer idea of how to deal with it. Um, but maybe I need you to be able to build more things, to get to the state of productivity. So,
AMOS:
I was going to say, as a developer, uh, you know, early on in my career, if I have a choice between going to a shop, that's doing Phoenix or I find the shop that's doing racks, I'm going to say, well, I hear about Phoenix all the time. So that's probably where it's best for me to get experience. So I'm going to the Phoenix shop. And as a business that's hiring, this is the reason why there are so many businesses hiring tons and tons of Java developers because they're, they're available. There's people that understand it. And it's cheaper often to hire a lot more Java developers than it is to hire somebody who's going to come in and, and create some brand new piece of software. That's more focused for you.
ANNA:
Yeah, no, I totally agree.
CHRIS:
I think there's also, I don't know how, I don't know how I know how I feel about it. Um, I'm well, it's complicated. Like there's also this thing of like, if you teach, I mean, cause this is the thing I think about actually is, um, Rails bridge, right? If you teach people rails, you can get like this whole group of people, jobs like, like the barrier or rather like the barrier to getting them a job is one less thing now. Right. As, not only do they know programming, but they know like this tool and you can now, like, I want to use it - as long as you can like sneak them in.
ANNA:
There's more opportunity available for folks who know rails.
CHRIS:
There. There's just enough people in the world who only want to hire, who wouldn't hire like a Ruby person, but would hire a rails person or like that would be a barrier to hiring someone whether or not that's like, whatever bias is like coming through in that decision. Like someone will make the decision. Like I want to hire rails people, whether or not that's right or wrong. And like, that's one less barrier now in order to like, get that person hired, which is like, that's like an, uh, no, that's really real, but like the trade-off is like, would they learn as much, if not more like about the underlying fundamentals, if you actually sort of build it up from first principles. Yeah. Actually probably. And like, is that person going to be like set on a better path? I dunno, arguable. Like I learned most web stuff through like Java struts and later spring. And then I learned rails and it's like, you know, you, you get there eventually. Like, you know what I mean? Like, like if you just keep doing this long enough, but you know, it's, it's tough. It's tough to say like, I mean, I think we could be setting people up for like, for better intuitions about design and about things if we couldn't approach it that way. Uh, but then the, the, then the problem becomes like, I dunno, like if our goals are like, you know, if you like, think about a bootcamp, right? Like if the goal is to like get all these people hired at the end of the day, some large amount of them hire like some percentage of them hired. It's like, you don't want to be the boot, like as a business, even you don't want to be the bootcamp that does like the weird thing, because -
ANNA:
Well, no, because you want to get, well, I mean, again, it goes back to like, I think the very beginning of this discussion, which is like, it depends on the goal for the thing, right? Like it depends on the entire context. Like how much time you have, how many resources you have and what is the end goal that you're trying to achieve. Hiring somebody or if it is getting a site up and running really quickly, right. Or if it is, and you don't have a lot of domain knowledge, right? Like what is the goal? And then how do you think about achieving that and then weighing again, because it's always gonna be trade-offs time with complexity, with knowledge, right. And then weighing those so that you think you're making the best decision.
AMOS:
Um, and which school do you want to go to when you're learning the one that teaches you a general purpose tool that gives you the ability to, to do way more, or do you want someone to teach you how to do HTTP requests and then you have to learn everything else.
ANNA:
It's totally true. Um, I think it's just interesting, right? We're talking about like now the benefits of smaller, deeper things, but where does that come into play? Right. And I'm not, I'm not, I'm not, it's like a little bit retort.
CHRIS:
Right, right, right. But I, but I do think there's something there. And then as you said this earlier, like, you know, you get, you, you get more junior people who like, it tends to be that June, I'm going to use quotes here, which you can't see all the podcasts. Like.
JAMES:
Chris right now is moving his fingers up and down.
CHRIS:
Junior people, you know, do, I think, tend to reach for more off the shelf tools or rather they, like, I think they reach for like rules. They reach for like, I don't have an intuition about this yet. And so in order until I gain an intuition about it, I'm going to use whatever the community has, like agreed upon. And there's like a comfort in that. That's all part of it. I think like, there's that comfort level of like, I'm using a thing that's well established. The thing that I want to point out is, and then why I'm saying junior and air quotes is like, we're all juniors at some level about something. Like if I walked in, uh, some other language or some other runtime or some other domain, I would be, Oh, I'm doing okay. So actually I'm doing this right now. Like, cause I'm learning. Um, uh, you know, in my other time, like I'm learning more about ML and I'm doing more ML stuff and this is, um, a new domain for me. And I'm leaning on a bunch of tools that like people say are good until I gain an intuition about how these things work. And I think that a lot of people, we catch a lot of people in that transition, like, especially cause elixir is still so new, we're catching a lot of people in that sort of transition from their previous thing to the next thing. And they're just looking for like that comfort for somebody to tell them, like, this is the thing to use. You can like trust us because we've been around here and worked and we're telling you that, right. They're not ready to make those sorts of leaps on their own yet, potentially, which might add to that.
ANNA:
If you want somebody to get there and to build the intuition, they need to have something that helps them develop that. Right. And so they need the tools to get there.
AMOS:
Unfortunately it kind of goes back to just like a startup money, right? You have, you have a limited amount of time. You want somebody to be effective quickly and possibly in a broad range of things. So again, teaching them the general purpose tool or pulling in dependencies is what they're going to do. So a junior developer is a lot like a startup. And, and so maybe until you've been a little further into product to where you have money and have an extended runway, you can't, it's, it's hard to make those. Trade-offs
ANNA:
Kind of going back to this continuum, the James was talking about, right. It's not one or the other. It's kind of like, where are you on that? And where does it make sense, James? What are you, what are your thoughts on what we've been talking about?
JAMES:
I'm doing all my homework here because I have to answer all these hard questions for my talks. I figured I'd just ask you guys and then say whatever you say. Um, that's cool. Now I love what you're saying. And like as, uh, a person I might instincts mean closer to Chris's, um, I really try to rely on, like, I remember one time I was working on an app that had this hash diffing dependency in it and it was just the library. They given two hashes of arbitrary nesting. Right. It would, you could call death and it would go through and figure out what the differences were between the two data structures. And I remember when I ran into that and I was like, how come they didn't just, you know, do a recursive, walk down the tree and figure it out. Like, it's like, I dunno, it's a little bit of code. It's like 20, 30 lines. Right. And figuring out or something. And, um, but it's pretty straightforward. And then I ran into this issue where there's library that we were using did something in a way I didn't agree with, you know? Right. Did you do that? I said, so then I ended up opening up the library and I was like, not much more than 20 or 30 lines of code. You never. And um, and those kinds of dependencies, I, I mean, I'm not saying it's wrong to have a hash dipping library or to use it or anything like that. But in my instinct, if I ran into that problem, I would just write the do function first. Um, because I feel like I can solve that problem. And I would be confident that I would solve it the way I want it solved in this particular instance or, or whatever. And so I do lean on that and that experience. And, um, I would rather use a Cuba if I can get away with it. Like if I'm building a blog or a book to talk about a, um, uh, a URL shortner right. Where you have, what three, enter your URL, bang. Okay. Here's your shortened URL. And then the one action that isn't even a page, it's just a redirect that takes the shortened URL and pushes you to the yellow line. Right. If you start that with rails new or Phoenix new, is that a bug? I kind of think it is, you know, like, I mean, couldn't you build that or just straight up plug. Like that's not very tough, right? Like whatever. And my tendency is to reduce that complexity, but I, I liked, what's been said about the trade-offs and why we make those decisions and why we sometimes push faster and stuff like that. Um, it is a lot of trade-offs and like, while I appreciate the junior aspect and being able to give them the more general tool and the support and the comfort, sometimes I counter that with sure. Yeah. You don't know what this does yet, but guess what? It's like 30 lines of code. Let's go read it. You know, like when it does, you know, and I, I feel like you can get a lot out of that angle too.
CHRIS:
That's how you drive your career to some degree in my mind, or at least it has been for me, like taking that extra time when I can find it to actually do the thing myself or to like, learn about it, you know, like the diffing examples, a great example. Like if you've never written a tree, like deferring to trees, that's a good exercise to go through. It's going to take you like a little more time the first time you do it, but it's a worthwhile endeavor because the next time you come across that problem, like, that's a, that's a tool in your tool belt now. Like, you know how to diff trees. You're like, Oh, I've seen this. Like I, I can diff trees.
JAMES:
And that problem comes up a lot in my experience of rehab, this state over here in this state over here, what do they have in common? You know?
CHRIS:
And it has all these knock on effects of like, you learn a little about, about data structures. You learn a little bit about, um, uh, recursion, you know, you, you, you start to get a feel for these things. Like it only good can come from that. I think like from encouraging people to do more of that.
JAMES:
I feel like I've monopolized the heck out of this conversation, but there are two other people on this podcast giving talks at gigs for the elixir. So I want to hear about stacking theory and cryptocurrency.
ANNA:
Well, the talk that I'm giving, it's not about giving before, but the talk that I'm giving, um, at gig city, um, is essentially, and I've talked a little bit about this before on the show, but really the goal is really to try and remove some of the complexity that I think people think exists around cryptocurrency. Um, especially before new to that domain. Um, and help them get an understanding of it actually not like under the covers behind all of the jargon. I mean, there's some complexity there, but it's not as complex as people think. And that if they're interested actually of getting like digging into crypto is not unattainable. And so we talk about kind of how crypto got to where it is, but also how you might implement a simple version of like a Bitcoin based cryptocurrency in elixir, which is given the power of the beam. Right. And given the nature of language, it makes it pretty easy to do.
JAMES:
So. I watched your talk from ElixirConf and it was great because I think I'm perfectly in the target audience. I knew almost nothing about, uh, Bitcoin and cryptocurrency and stuff. And like, it was super approachable for me and I got a lot out of it.
ANNA:
Oh, thank you. Chris, what about stacking theory?
Speaker 4:
Yeah. Um, I have the talk and, uh, index cards in front of me on my desk all stacked up. All the post-its that are attached to the index cards are stacked up, but yeah. Um, so stacking three, I think, I don't know. I think J Lewis may have invented that name. I don't know if there's prior art on that. Um, but uh, he talks about this, this, he wrote this medium post, um, that I've sort of like I read and it really kind of, it resonated with me. So it's like, it's informed a lot of things I've done since then. Um, about being able to sort of look at a system as a bunch of - looking at rather not looking at a system, but looking at like the operational modes of a system as like layers. Um, so layer zero might be okay, so let's like, let's take a Phoenix app, right? Cause it's an easy sort of thing to understand. And you've got a bunch of moving parts to a Phoenix app. You've got, you know, your, your actual, um, a web server like cowboy, which has to hook a port, has to start accepting requests has to, you know, be supervised and run. And then you have a database and the database has to, you know, hook to the database and like has to talk to the database, has to establish TCP connections, like whatever it's going to do. And the idea of the stacking theory stuff is like, if you look at how that system runs, we could actually break down each of those different modes into different layers. So layer zero might be the OTP app or the OTP release boots like it's comes up. And at that point you can start to move to the layer one and layer one might be, um, where we have a web server running, but we don't have a database yet, but like the web servers running and then layer two might be now we have our external connections to the database and you move between these layers of sort of a best effort kind of way. So you do things like, okay, I'm going to try to hook a port. Now, if you can't hook the port, you fall back to like layers zero or whatever, what this allows you to do if you implement that correctly. Um, which is kind of the, the, the post doesn't really go into like how you do any of this. It's just more of like an idea. Um, so the talk of, hopefully it's going to help demonstrate like how I've been trying to do some more of this. Um, but the idea would be like, if you have, you know, your database connection and your database drops out and you're, you can't, you can no longer talk to the database again. Then you don't like, uh, kill the app or try to like kick over the instance or anything like that. You degrade back down to like layer one and layer one can then, you know, part of that process could be like sending out alarms or it could be alerting, um, your load balancer to take you out a load balance or whatever, but you, you degrade gracefully again and then come back up as need be.
JAMES:
Um, this sounds like, uh, is it run levels in Linux where, when Linux comes up, it goes through various run levels. Am I right about this?
AMOS:
Yes. It does a lot of, um, hardware development stuff too. You do that same kind of thing. Otherwise, uh, you know, if something crashes as you're coming up, you can't even light up, uh, lights on your alarm panel if you don't have different like levels. So yeah, it's, it's the same thing. That's what, what makes those two, okay.
JAMES:
Macr OS uh, if you use the Mac, sometimes you see this when, um, if something bad happens to your Mac and you go on to Apple's website and you're troubleshooting, they'll tell ya, hold down these keys and start up to boot into safe mode and safe mode is like a lower run level than you usually run it. And it doesn't try to bring everything up. And because of that, there's actually a higher chance that your Mac will successfully boot into that scenario. Right. And then you can troubleshoot from there.
CHRIS:
One of the things I've been trying to sort of emphasize too, in this talk is like, you may or may not need to do this depending on your infrastructure and your application concerns like this is, I mean, these sorts of things are a real problem at Bleacher report because of like the amount of things that we run and the number of services, the number of nodes, the all that kind of stuff. And we, and we see those degradations, like on pure probability, we see those, those things a lot more, let alone like the, the kind of traffic spikes that we get. So, um, which tend to force degradations to happen, uh, or at least like increase the probabilistic chance that those degradations will happen. So we end up having, like, you know, I think it's a truism that basically any sufficiently complicated system runs in degraded mode, basically all the time, like something is not working almost a hundred percent of the time. Now, how much that affects you is, is really what we're trying to like balance. Uh, but the trade off is you bring a lot of these, like complexities into your system. When you ask the system to take, to handle it, like when you're starting to do these things yourself, you know, you're, you're owning a lot of that complexity and you really have to weigh whether or not that's like right for your system. So like, your blog probably doesn't need to do this, you know, but you know, any, like if you have a sufficiently complicated system that needs a certain amount of availability, you may want to start to like, approach this problem from that. And that, and for that standpoint,
JAMES:
It's cool too, to like, not have your app be two binaries, right. Either it's up or it's down, right? Like, you know, maybe these parts are up, I'm like this thing's down, you know, and not, not having to have those does. Um, this is my own curiosity, which you probably answered in the talk, but does, uh, elixir make this kind of building easier? Like, because of the way it's designed with fault tolerance and stuff like that.
CHRIS:
Yeah. We get, I mean, we get to, we have all these primitives that we get to just rely on to do all this kind of stuff like supervisors and whatever else, um, at is actually a big part of that, because I mean, we use EDS for basically like, uh, uh, uh, cash all the time, like a local cash, um, very often, especially around like circuit breakers. Uh, so if a downstream service is down or is like overwhelmed and we need to like back off of them, we'll just like kill the circuit and then fall back to a cash. And the cash has maintained inside that stuff like that. Um, and that's sort of a, that's another layer to this. Like, can we make outbound calls? Yes or no. Um, and like, what do we do when we fall over? So we just have, we just get this run of this great runtime, like, as we were talking about earlier with all these amazing primitives to start to put these pieces together. Um, but there's definitely nothing right now that I think there's, there's something right now that like, we'll do this for you. Like, you can't just use like the default supervision stuff, like one for one, um, out of the box, like you start to have to really like, think really pretty hard about how you're going to supervise these things. And you, you need to some degree get outside the bounds of the supervision, like of what the supervisors themselves want to do. So like one, like a concrete example of that is, um, one pattern we've used is to have a supervisor that starts, uh, effectively like a watchdog process and a dynamic supervisor. And the watchdog is like, what actually starts the real process that is managed underneath the dynamic supervisor and that way, and it monitors it. Um, and that way, if it crashes, it can do like exponential backoff, as opposed to like letting the, the normal supervision stuff like cascade and like kill the app or whatever. So we do you get those primitives, but sometimes you have to do that, like some of that stuff yourself, um, and like, you know, just use the tools that are there, um, in order to do those sorts of things.
JAMES:
That's super interesting
CHRIS:
Instead of relying on like your normal, um, restart intensities and all that kinda stuff. And we do that too, but there's, you know, you, you start to play around with a lot more of these like strategies and those sorts of things.
JAMES:
Have you ever seen Gen retry?
CHRIS:
I have not seen Gen retry. What is that?
JAMES:
I can't remember if it's Erlang or Elixir. But, um, uh, it is basically, uh, supervision with exponential back off and stuff like that. And, Oh, cool. That sounds, that sounds useful. Okay.
AMOS:
I think there's also one that's just called retry when but it may not be supervision trees, so that may be why.
JAMES:
I think Gen retry does like supervised but I may be wrong.
CHRIS:
That's, that's pretty useful. Cause I mean, getting the supervision stuff, getting that kind of stuff to work in the supervision trees is somewhat non-trivial right. Because it's not really what the supervisors would want to do. Right? Like that's, that's not really their normal mode. So you're getting outside of their normal mode to some degree. It's also why I was really excited when they brought like, when they built dynamic supervisor. Cause I was like, Oh sweet. We have this like really good tool that can do all this stuff for us now. Like I use that so much. Yeah. I'm excited to get it written.
JAMES:
I can say I've made the same place. I got all this stuff and it's like, Oh yeah. Now I have to put this together and turn it into a talk.
ANNA:
I'm excited to hear both of you, talks are gonna be great.
AMOS:
I'm pretty excited about the whole conference.
CHRIS:
I'm just excited that you're all coming to Chattanooga. I don't have to go anywhere.
JAMES:
It's going to be cool. I've never been there.
ANNA:
It's super nice. I like it.
JAMES:
We are also going to have a live recording of the Elixir Outlaws at Gig City Elixir.
AMOS:
I don't know if that was a good or a bad choice.
ANNA:
We'll find out
AMOS:
It'll be our first live recording. So, um, it should be exciting.
ANNA:
It'll definitely be exciting.
CHRIS:
I don't think anybody believes that we're like super put together, but now they're definitely going to have it confirmed that we're not.
ANNA:
That'd be great. It's going to be well they'll know. Right. They'll see. They'll see the truth.
JAMES:
Get to see the sausage made.
CHRIS:
I've been trying to think of what we should do. Like maybe like we should have a section of like Q and a or something like that. Like a little more engagement. I don't know. I feel like it's going to be weird if we're just up there talking to each other for that long in front of a bunch of people. Yeah.
AMOS:
Well that's, I, I have, uh, an extra mic that I'm planning on bringing. Um, so, so that'll give us a, another mic to have, I think we just have a hot seat. Just let everything, but anybody come up who wants to. Here's your, here's your platform to say whatever you want. Uh, so, um, on that, I'm going to go back to, to James is really quick on that continuum. He, you ask where on the continuum, should we put that point of, of picking a library or building it ourselves? And I'm going to say a 7.4.
JAMES:
I am making a slide right now.
CHRIS:
I was going to say 42.
JAMES:
"Amos says 7.4" - that's the slide. That man, right there in the audience. If you would like to go talk to him afterwards, he would love to discuss this with you. Can you have a second slide that says, Chris said, well actually it's 42?
ANNA:
I was gonna say, thanks James, for being on the show. It's a super fun. And for bringing such good question.
JAMES:
Thank you. It's been a long time since I've got to have a great discussion. Like this is really great.
ANNA:
Any parting thoughts from anyone?
AMOS:
I don't have thoughts.
ANNA:
Other than we're super excited about gig city,
JAMES:
Come and see all of us there.
AMOS:
If there are any tickets left, I think, but by the time this comes out.
JAMES:
There are a couple of tickets left. We had some, uh, we have, we almost ran out and then we had some sponsors that had like blocked tickets. And so we went to them and we're like, okay, we really got to know how many people are you sending because we're about full. And so that allowed us to release a few more. And I think we're at about 15 right now.
AMOS:
Sweet. Now's the time to scalp. We can make some money.
ANNA:
That's not what I was thinking, but. Sure.
JAMES:
That's not what you were thinking?
AMOS:
We should build a conference tech conference ticket scalping app.
JAMES:
Yeah. Bring all the bad parts of sports to tech conferences. I agree.
CHRIS:
That's good. I like where your head's at Amos.
AMOS:
Well, I know that the Anna's has got to get going.
JAMES:
It was good to see you and I look forward to getting to meet you in person.
ANNA:
Thanks. Y'all so super fun. It was so nice.
CHRIS:
Thanks for being on the show, James.
New Speaker:
Yeah. Thank you. Thank you.
AMOS:
And tell the family that we all said hello.
JAMES:
I will.
New Speaker:
See you later.
New Speaker:
Have a wonderful day. Bye bye.
Welcome to the Elixir Outlaws, the hallway track of the Elixir community.
AMOS:
Hey James!
JAMES:
How's it going?
AMOS:
Good. James looks more serious than the rest of us. That set up he's got.
ANNA:
I know, right? That is an awesome set up.
CHRIS:
Is my mic working?
AMOS:
Yes, it is now.
CHRIS:
Okay, Cool. I was talking to Amos for like 10 minutes and he was refusing to acknowledge me.
AMOS:
Well, I try not to acknowledge you in public. I don't want people to know I know you.
CHRIS:
It's bad for all of us.
AMOS:
Oh, James has a Kansas city shirt on.
JAMES:
I do have a Kansas City shirt on. Yeah. I can hear you, Anna but you're a little quiet.
ANNA:
All right. Thank you. Is that better?
JAMES:
I would still say a little quiet.
AMOS:
Yep. That's that every time you use that mic, you have to get really close to it.
ANNA:
Testing... That better?
AMOS:
That's a lot better.
CHRIS:
This, this whole setup process is just part of the show at this point.
JAMES:
Is this the show?
AMOS:
Yes. This is the show.
JAMES:
Is it too soon to make that joke?
ANNA:
No, that was perfect timing.
ANNA:
How's it going, everybody?
CHRIS:
They're going well, Anna, how are things going for you Amos? This is Chris speaking.
AMOS:
Chris keeps giving me books. Or suggesting books. So, um, I'm not getting anything done. And one of the books that he suggested to me, he was getting things done, which is just put off more stuff.
JAMES:
Amazing.
AMOS:
And, uh, the philosophy of design is another one, um, which has been, uh, a great transition out of Pivotus for awhile because you, you know how that one's going.
JAMES:
Amos is going to need a support group for that book.
AMOS:
I am. Yeah.
CHRIS:
Hey Amos. This is Chris again. I'm making sure that people understand who we all are, but you want to, uh, do you want to introduce our guests, which is a show first for having a guest on.
JAMES:
Have you not had guests before?
CHRIS:
We've had guests. We Just never introduced them.
ANNA:
It's going to be a show first. Perfect. Good job Chris.
CHRIS:
Yeah. Sorry. No, no, no, actually like being courteous to our guests and like introducing who they are, as opposed to just assuming or like everybody's clued in.
AMOS:
Uh, so, so today, uh, on the show we have James Edward Gray, the second. James, uh, has written a book. It was pretty prolific in the Ruby community and is doing elixir. And, uh, is involved in a conference coming up that we might be involved in. Maybe, uh, James, you wanna talk a little bit about yourself and that kind of thing
JAMES:
You guys are going to the conference?
AMOS:
Well, it depends on if Chris's wife has baby or not.
CHRIS:
That's true.
AMOS:
I might be watching his children while he's going to a hospital.
JAMES:
Holy cow. Wow.
CHRIS:
That's true. Yeah.
JAMES:
That's exciting.
CHRIS:
That's a non there's a non-zero chance that that might happen.
JAMES:
Yeah, Amos. That's who I am, I was really active in the Ruby community for many years. And if you listen to podcasts, I did the Ruby Rouges podcast for like three years. Um, and now I spend most of my time in the Elixir community doing Elixir-y things. And I recently had dinner with Chris and Amos. Anna Wasn't there.
AMOS:
We missed you, Anna. We had a great dinner and a great time there at Stragne Loop. Learned quite a few things. And then, um, the Erlang ICFP was, was pretty great. James, you, you made it to, I think just the first talk?
Speaker 2:
For that first tier, the morning, then I had to drive back.
AMOS:
It's a long drive.
JAMES:
Yeah, for sure.
AMOS:
What do you guys have on your mind today, I know that James sent out a few things that he wanted to talk about.
JAMES:
I brought questions.
AMOS:
I'm sure Chris will answer them for you.
CHRIS:
Oh yeah, yeah.
ANNA:
Chris is ready. He's wired, he's had coffee.
CHRIS:
Um, um, this is probably one too many coffees. If we're being honest.
JAMES:
Perfect for a podcast.
ANNA:
It's not the first time that's happened. I feel like we hear the one too many coffees situation often.
CHRIS:
It's because of the podcast. Like I intentionally do that. It's unhealthy for sure. But it's part of it.
AMOS:
When you can't get a good aerobic workout for your heart, just drink a lot of coffee.
ANNA:
There we go. It replaces the need for a good aerobic workout. Right. That's how, that's how that works.
JAMES:
All right. So can I ask questions?
CHRIS:
Yes, please. Yeah.
New Speaker:
Okay. I am going to give this talk. At Gig City Elixir and it's about using less code to solve our problems. Um, but along the way I touch on, I wanted to touch on several questions. So I'll set this up with a few questions. Let's say, I assumed that we would all on this podcast, agree that if I needed some functionality that exists in a heavy dependency, it would be like a big dependency with lots of things, but I only need a little bit of the functionality so I can either bring in this big dependency or I can just write, let's say like 20 lines of code. I assume we would all agree that I should just write the 20 lines of code. Does that seem fair?
AMOS:
Yes.
ANNA:
Yes.
CHRIS:
Yes.
JAMES:
So now let's go. The other way. I also assume we would all agree. I'll, I'll try to use a Chris example here to do raft consensus
AMOS:
That we may be going down a dark path.
CHRIS:
No, no, no. This is good. Let's do this.
JAMES:
This could take a turn for the wrose. If I had to get raft consensus, then in that case, I should probably go ahead and use the library or framework or whatever, because it's too error prone, and it's just too likely that I'm going to screw something important. Right.
AMOS:
And hopefully some free maintenance along the way that like, I don't know enough about raft. So, um, I'm hoping somebody else will take care of that part for me.
JAMES:
Exactly. Right. Okay. So here's, here's the actual question. This is like a continuum, right? Uh, on one end we resolved the problem ourselves and on the other end, lean on a dependency for it, the maintenance, for solving the problems we don't understand or, or stuff like that. And my question is most of the time, like where should we be aiming to land on this continuum? Like, do we want to be like right in the middle? Do we want to be a little closer to one side the other side? You know what I'm saying?
CHRIS:
Who, who wants to go? Cause I'm ready. I've got an answer. I don't want to be. I pushed back on these stereotypes that you all have created for me. Okay?
AMOS:
You created these yourself. What are you talking about?
CHRIS:
So in my mind, the thing that you're driving towards, if I was going to like, um, kind of like shape or frame the problem in a, in a specific way, and you can tell me if I'm off base on this framing, bringing in a dependency is a point of coupling. And we know that we could sort of take it as read. Coupling is bad. Like most of the best practices that exists in software development are in a lot of ways to like help reduce coupling because coupling keeps us from being able to move forward and change things and all that kind of stuff. So any dependency you bring in is a point of coupling. So to me, in my mind, the power of the abstraction, the power of the dependency, which is an ephemeral word that I don't have a good definition for at the moment, but like the power of it has to outweigh the cost of the, of that point of coupling. Is that a, would that be like a fair framing of, of, of, of the question some degree?
JAMES:
Oh yeah. I think so.
ANNA:
And how do you define when it does outweigh?
JAMES:
Yeah. That's right. Good question.
CHRIS:
Yeah. I think that's going to be, it's tough. I think that's going to be very dependent on where your project is in its like life and the context of the people that you have on your team working on it and yeah.
ANNA:
Can you say more about that? Depends on when, where the project is in its life.
CHRIS:
I don't want to - I battle internally with us all the time, because I've definitely worked with, I mean, as you have, right. And I'm like, uh, at C5, like you see plenty of early stage companies that are like, I need something I've got three months to like get a thing out into the wild. And I don't even know if this idea is good or not yet. And I just need something out there to even just prove that like I could make a business out of this.
JAMES:
Otherwise we run out of runway. Right?
CHRIS:
Yeah. Otherwise we literally don't have a business. And as a consultant, like, I feel like your job is to enable that like your job is to do that is to allow them to figure those things out. And if that means like cutting corners, bringing in extra, I mean, I think this is like, you see this in the proliferation of like Ruby gems, right. Not to like, I'm not, not to bag on Ruby or whatever. Like you see this in proliferation of lots of libraries and lots of ecosystems, like libraries that do things that arguably you should be doing but that you can defer to somebody else. Hey. Yeah. I mean, Left Pad is like the funny example. Right. But like, there's plenty of things that I actually think that, you know, you don't need a gem for whatever, but maybe you just do it because that'll like, you know, that cuts a couple of days off of your timeline. And if all you've got is days, like if you can measure your, your runway in days, those days matter.
Speaker 3:
To me, one thing that matters is what is your current domain knowledge of the thing that you might be pulling in? Do you have the time to put into it and the domain knowledge to put into it, or is it going to be a huge learning thing? So, because Chris asked me reading philosophy of software design, the first thing I thought of was he puts in a, uh, uh, complexity, um, function in here. And, uh, I think that it kind of comes down to the complexity of the problem, which, uh, he puts in here as, um, the complexity of each part that you're pulling in, uh, times the fraction of time that a developer is going to spend on working on that part. So, you know, if you're going to, it might be super easy, but let's say it's going to take four weeks to do. Maybe you don't have that kind of time or it's going to be really, really complex, but take a little bit less time. So I think that you have this complexity versus time thing that you have to think about whenever you're deciding to pull something in, and maybe your time is better spent just reviewing the code of a dependency that you might pull in to see if you think that it's worth pulling in or not
JAMES:
By going through the dependency and seeing how much of its paradigm it's going to impose on your code basically.
AMOS:
Right? Yep. And then can you protect yourself from that future change too? Like if the interface isn't that very big, would it be easy for me to wrap it inside of my own interface? So I can hide that a little bit, that, that coupling into one place so that maybe I can switch it out later for my own. It depends on like how littered in the code it's going to be, how prolific in the code are you depending on this thing.
ANA:
And also, I mean, I think time plays a big factor in that complexity component, right? Because how much of this talk about complexity depends actually very, in a very real way. On how much time you have to press point about getting something out for like, if you have to get it out within a few weeks, will there be more complexity because you have less time or is there a way that you can mitigate that? Right.
CHRIS:
I think a big part of it is, um, it's also the, the, and this, this comes down more to, uh, experience in just, you know, doing this a lot and kind of understanding, you know, getting a feel for these sorts of things. But I think that there's, you know, the, the time part of it is also like, what is the time spent? What, what not even like, what is the time you're going to spend upfront, but like how many touch points are going to be in the rest of the code? How much, how many, how often are you going to look at thing? Because in his equation where he's saying, you know, the total complexity of a system has all the individual pieces times the number of time you spend working on it, it's not even just upfront, it's like over the course of the thing. So if you can hide a thing that's as good as removing that complexity, like you don't deal with it anymore, um, from a cognitive burden standpoint. Yep.
JAMES:
So the interesting thing to me, it sounds like you're re almost, so I agree back up front, you know, when you have to move fast, the dependencies can be really valuable. So then when the app enters like the mid to late life stage, do we then go back and remove those dependencies in favor of replacing them with their own.
AMOS:
I knew that question was coming. Its a really good question.
ANNA:
Has anybody actually had an experience of that happening?
AMOS:
Yes.
JAMES:
But I wouldn't say it's common
CHRIS:
And it wasn't fun.
AMOS:
It was a very painful process.
ANNA:
What led you all to decide to do that then? Other than a dependency maybe is no longer supported or...?
AMOS:
Yeah. Yeah. Um, so that's what I going to say first. It was unmaintained dependency was the first example that I can think of. And another one that, um, was that it, it, we had certain performance characteristics that we had to meet that the dependency wasn't meeting, because it was more general than actually what we needed. We needed something. Maybe I would call it like a very surgical strike. I just needed to pull the appendix out. I that, which doesn't mean I had to cut the entire chest open. And that's a terrible analogy.
ANNA:
No, I mean, but essentially by removing the dependency, even though you needed it, initially you are making, you are lessening the complexity of your system, right. Because you are only essentially reintroducing the piece of complexity that you need.
AMOS:
But the, the big thing that made it hard to pull out was we did not wrap our dependency. So we had calls to this everywhere, and I'm not sure, I'm not sure how much that would have changed if we did wrap it because when you wrap it and you're looking at one interface and then you try to change it to another interface, unless you did a lot of upfront thinking, which when you're in that hurry runway, I'm running out of runway phase - you often, aren't doing as much thinking about what are the repercussions of the interface itself. And so when you do wrap it, a lot of times, it's really a facade of the exact same interface, which is not helpful.
CHRIS:
We had an, um, a project once at a C5 where, um, along, you know, previously, some other people had added, um, I'm trying to hear what the gem is called, whatever. It doesn't matter. I don't want to malign like their gem or whatever, but they had brought in like a gem to do, um, uh, authorization type stuff. And so they had all these rules and they had all these. And it was like all these classes about like how they did authorization and all those sorts of things. And there came a point where we realized the business wanted these really rich, interesting rules about, you know, enrolls and who could be assigned to like work on certain things. And like that it was this really complicated thing. But like, that was how their business worked. They needed to be able to do that, like for their business. And this gem is designed to be this like very conventional off the shelf thing. And we literally could not like shoe horn. I mean, we could have spent a lot of time shoehorning it in there, but it became untenable to continue to like shoehorn these like really complicated rules specific to that business, uh, into this library. And at that point, it was like super hard to start pulling all that stuff out of there because it's everywhere. It's like littered throughout the code and it's broken all these and the, and the way that those that gem wants to work is it like once hooks into all of your models and it wants hooks into all of your controllers and it like ties all these bits together, you know, that was, that took far too long and became a huge, it became a major impediment to actually like providing value to that business because like now we were in this place where we had to pay down these debts to, in order to provide value to the business. And to me, that's the place I never want to be. Um, especially as a consultant, definitely as just a programmer in general and as a human, like, I don't want to be in a place where like I'm impeding, or my decisions are impeding the business, which is paying me money to provide value, to like, not be able to provide value. Um,
JAMES:
The business is asking you for features A, B and C, and you're saying, hang on, I'll get to that. As soon as I unbuild this mountain of doom I've constructed.
AMOS:
Isn't that what all design is though, is trying to, trying to, uh, not end up in the mountain of doom?
CHRIS:
We talked about this last week and there's this great Rich Hickey quote that, um, that I just, I love and we're down to all the time, but it's, um, you know, programmers know the benefits of everything and the trade-offs of nothing. And like that to me is what's, epitomise in that, right? It's like we knew all the benefits of using this, like off the shelf thing. And up until we had got to a place where it was no longer a benefit. And then, and now it's like, Oh, well now we're, now we've really painted ourselves into a corner here.
ANNA:
But given that it's so hard to predict that situation sometimes, right? Like how do you mitigate for this increased complexity that then becomes really difficult to remove, right? Like as we're designing systems and as we started, you know, I'm also starting to get to that book. Um, how it's, it's a, it's a really hard thing to do, right. Because we also don't want to like, over-optimize for something that's not going to exist. Right. And we want to solve the problems that we have at hand, but then we get into these scenarios, right. This is not a new problem that we see. I don't know if there's a good answer to that, but...
AMOS:
How about you James? Where do you stand on the answer to your question?
JAMES:
I don't know. I haven't decided yet. Um, I would say, um, I think you kind of hit on the key point when you were talking about, uh, removing something. And I think it's that dependencies in general are always generic. Like they have to be to serve a wider audience. Right. But when you know your problem, what exactly your problem is, you can write a more targeted, you know, specific for your needs thing. And almost surely, that's always going to be superior if you have the ability to do that, because you can solve what Anna just said, the problem at hand, right. The actual problem that you have. Um, and so that's almost always going to be better, but then, you know, when can we spend the time to do that? And when can we not spend the time to do that is a huge part of it. Like, which I think we hit on right off the bat. But, uh, but I want to throw you one more twist if I can. Um, yeah. Okay. So let's do one more twist. At Strange Loop, I had this cool aha moment while I was sitting at a talk about rust and node, it was about rust and node and, um, it was by Ashley Williams, it was a great talk. If you have not seen it, look at that. It's great. But she said at one point that rust doesn't cabinet run time by design, because rest is meant to run on everything, like your toaster. Right. And so it's got to be like the lowest level thing possible, and it doesn't have a run time and that's by design and that's a feature and that's awesome for us, which then led to this kind of aha moment in my head, which is in a Elixir - it's all about the run time, right? It's the beam, all the great things come from the beam, right. And there's run time and this ability to spawn process and have concurrent everything while basically just writing imperative code and stuff like that. So then my question is does Elixir use this equation of the trade-offs of rolling around and using an off the shelf thing. Does Elixir change that equations slightly, like, would you make different decisions in a Elixir on that continuum, then you work and say rust?
AMOS:
That's a hard question. As you can tell by the silence. I don't know that it changes it much for me. Like, I, I appreciate the run time and everything that it gives to me. And I did choose language that had these features instead of having to find a package that has those features. Right. But I don't, I don't know that that changes how I choose to pull in even more dependencies. I am, depending on the runtime because of my language choice. But that doesn't mean that I have to pull in more or less packages for other design decisions or choices. I don't, I don't know that it changes my thoughts on it.
JAMES:
So I think it does change my thoughts on it. And so to give the contrast to what Amos just said, for example, in Ruby, if I build a web app and I have a web application running and now I need to do some background processing, my first thought is go grab, uh, gosh, it's been so long. I can't remember...
ANNA:
Sidekick.
CHRIS:
Sidekick or Rescue - one of those.
JAMES:
Yeah, go grab one of those. And in a Elixir, if I need to do some background processing, that is not my first thought. My first thought is to add a supervisor to the tree, I'm launching processes under the supervisor I'm to go.
AMOS:
So you're coming at it from a different angle than what I was thinking. But so to me, that's something built into language, but are all dependencies like that? Or are they just built in?
JAMES:
I see what you're saying. Yeah.
CHRIS:
Yeah. I, I totally get what you're saying. And I think for me, I'm trying to think of the right way to say this. So like, in my mind, I mean, while we've sort of talked about, like, if you're an early stage company, it's like fine to pull in those dependencies and that kind of stuff. Right. Um, but I want to add like an addendum to that, which is for me, I think my goal over the past, however many years, definitely when I was doing full-time consulting, my goal as a consultant was to make it, so I never had to do that. Like I felt like that was like imperative on me as a, as a developer to get to a point where I could say, I don't need to pull a dependency because like, I've learned a lot about this and I could implement this myself, like, and implement it with the trade-offs of like myself, especially if it's something small. Um, and I felt like that was a thing that I wanted to like, get good at, not in sort of a, like, not invented here kind of way, but like more in like a, if we don't need to do this, but we're doing it out of necessity. Like I don't want necessity to ever drive a decision to like add complexity, if that makes sense. And so I think that dovetails super well with what, with, with like my enjoyment of elixir was all of a sudden I got this runtime and maybe this is kind of what you're getting at. It's like, I got this runtime where it's like, I'm not burdened anymore. Like I'm not burdened by anything I can do. Like, I can solve all these hard problems that were really hard to solve. And other runtimes they're just gone now and I can just build highly concurrent things and I can build stuff that is fault tolerant, like at this like really granular level. Um, and so, yeah, I think in that respect, elixir sort of helped, um, in a lot of ways like fuel that, which maybe isn't, maybe that's not a healthy or mature, um, uh, viewpoint to take, but it definitely like fueled that in me. Um, I think there's some potentially something to that.
JAMES:
Is that what you meant earlier, Chris, when you said it depends on who you have on your team? So if you have a team of people that are very capable of solving these problems in a reasonable amount of time, you would lean more on just solving the problem yourself and instead of playing in a bigger dependency or something like that.
CHRIS:
Yeah, I think so. I mean, and specifically, like if you had a group of people who had domain experience in the specific thing, like you're obviously going to be more inclined to bring, to do that stuff yourself, because like you understand the trade offs that are going to have to take place to do that. Um, and I think that like team that team composition, it's hard to ignore that because if you don't have people who have, so, you know, the subject matter sort of experience hard, won knowledge, you know, you're to be, uh, it's going to be a pretty big time sink to like spend a bunch of time on that. Not to say it's like not worth doing for your edification, but like from the point of view of like providing value to the business, like it may be that you just need to like lean on someone else's experience for a little bit until you gain your own, which is fine. That's like totally reasonable.
ANNA:
I like that a lot. And I think that kind of alludes to it. I think we talked about this a little bit last time, right? Like part of it, part of the complexity is a main and part of it is how we're building the system. But part of it is also exactly what you just said, Chris, like the team and the team's current understanding of the domain and the system, right? Like that definitely either lowers or increases the current state of complexity depending on their understanding. Yeah.
AMOS:
I mean, you can, you can look at the elixir ecosystem and see where, see that people gaining that knowledge. If you look at back a little ways, you start seeing where people are using Reddis from elixir and background cues and starting up all these things that look just like Ruby or, or any other language that you might use. And, and as the community learned more about the runtime itself, that stuff went away,
JAMES:
Now people just fire up NTS or even grab the dependency, right. Con cash or something, which is what I would call smaller dependency, which is just firing at BPS for you. Right.
CHRIS:
I've started using things like registry a lot more, um, because, uh, as opposed to, it depends on the use case, but as opposed to doing, um, using ets on my own, uh, like if I know I'm going to have a lot of things in sets, um, I'll often use registry because they're doing really smart, like bucketing and sharding across ets tables to make lookups really fast. And that like, they've done a ton of work on registry and that's like, uh, so that provides a lot of power, uh, over the normal abstraction that you would have to build on your own. And it's a re like, those are sorts of the things that I lean on more now is sort of these more, I dunno, it's, it's, um, I'm so poisoned by like philosophy of software design. Like my, I look at everything and through that lens now, and it's hard not to look at what he calls deep modules, right. Modules that provide a lot of power, but with a very small interface, like without, without actually doing too much, they're giving you a ton of power. Um, and registry, I think is one of those like registries doing a lot more than just like storing keys and values, but it's like they have this really relatively simple interface to just like put stuff in there. Same with Comcast, like Comcast is doing like smart stuff behind the scenes that are somewhat, you know, non-trivial to manage on your own. Um, and those are, those are good dependencies to like to pull in where, you know, if all of a sudden you were doing less with a shallower interface with like a wider interface with like more functions or whatever, that's, you know, you're naturally going to end up coupled to that thing in like much more gnarly ways. And it's not giving you a ton of power for the cost of that coupling.
JAMES:
So do you say you use registry for like, even things that are outside of its typical usages? Like not pubs, uh, not processed registration and stuff like that?
AMOS:
Yeah. I have the same, same, same thing. That was my mind blown thing was I never thought about using registry for anything except for processes.
CHRIS:
But I mean, if you think about like what, like the process register you're right. Like the point is like, you need to hold like this giant, like list of keys and values - just a big lookup table. And like, registry is really good at that. Uh, being, uh, being really fast, it can handle like tons and tons and tons of inputs, which as it should be able to, cause it's, it's meant for like registering processes, but it's kind of like misnamed, like it's really just a really good key value store local to your box. Like that's like highly efficient.
JAMES:
Those kinds of dependencies in some ways are more powerful, which I think is what you were saying. And I'm just catching up to you is that this dependency that just does this one thing really well, but is pretty narrow. And like you're only going to make a couple of calls to it started to put things in, take things out, um, that is a superior kind of dependency to something like a big framework that wants to take over a large portion of how you write your application or something.
AMOS:
So then why, I think I have an answer to this, but why aren't there more dependencies that are, are thin and deep? There's a small interface, but they're very powerful
JAMES:
In the Ruby community. There's a kind of this small subculture at that sometimes went by the name of the West code movement and they do have a bunch of dependencies like this, um, like, uh, Cuba as a web app. And then there was one that came later, I think, CRO or something like that, that replaced it. And then, um, Ome was like a database dependency. It was actually built off of Reddis, which was unusual, but it was like a complete database layer and under 500 lines of Ruby, I think, uh, and it did like models and add smaller to kind of be active record conveniences and things like that. Um, Mote was a rendering engine kind of similar to ERB and stuff like that. Uh, if I remember correctly, it was like under 50 lines of Ruby character. And like, um, there were these like very focused dependencies, but I wouldn't say they were mainstream. Like you had to get off the beaten path. Right.
CHRIS:
Was like the dry was all the dry RB, um, projects where that was that part of that whole kind of subculture. I remember seeing these, but I never really,
CHRIS:
I don't know. That's a good question. I'm not familiar with the dry RB stuff. What's that
CHRIS:
Like, it's just a bunch of little packages. They kind of do a lot of like functional kind of stuff, in a Haskell vein of functional. Like there's like a dry RV monads and a dry RV functers like, and stuff like that. To me, like, it's like not the stuff you're talking about. Cause I think the stuff you're talking about is, is the right is the right approach. Like you want a thing that does, like you consider a lot of stuff. Um, the dry RB stuff was like two was a little too small in my mind. Like it doesn't give you enough power for like attaching yourself to this, to this, to this thing, to this, to this dependency. That's the word. Um, it, for, for me, like, I don't want type classes as a dependency. Like that's not interesting enough, this isn't enough power for me.
JAMES:
Right. Right. If you have bunkers or something, you have this new tool to build things, but you're still building the thing. Right. Whereas like what you were saying with registering, it was like, Hey they, already built this awesome thing. And it's great at what it does and a couple of things that they didn't even intend it to be great at, you know?
CHRIS:
Yeah. And it sounds like that's the same thing. That's true also of like, um, uh, you said it was Ome the database engine that you're talking about here. Yeah. I mean, those are giving you a lot of power and it's a hard problem. Like storing stuff consistently in a database and like talking and providing the right interface to that is hard. That's a lot of power and that's worth having a dependency for, in my mind, to me in my like personal sort of, um, rubric
ANNA:
I'm a little bit of a larger question, which is James mentioned, you had to really go searching and that type of dependency is a little bit more off the beaten path, even though it's a harder problem to solve and they are deeper dependencies, but what does that say? The fact that those are the dependencies that are off the beaten path, right? Like about how we generally develop software and that most dependencies is the trend is for them to be more general. Right. And is that, is that the right approach?
AMOS:
I think that people gravitate towards that because when you're starting out, I don't know what I'm going to need. So it's easy to grab something like Phoenix and ecto that does almost everything that you can think of. Uh, but, but we, I mean, we do have this in the elixir community. You have, uh, Peter Sexton is a pretty big pusher of, of uh, cracks. Yeah. And, and, and other, um, I call them micro libraries. What's the, uh, what's the one for talking to it only talks to Postgres
CHRIS:
Mobius. Mobius. Yeah. Yeah. That's Ron Connery's jam.
ANNA:
I just think it's interesting the fact that those, that those dependencies exist, but you really do have to go looking for them, which says that most people probably don't go looking for them. Right. And maybe that's maybe that's because at first you don't know what you're building. And so you want something more generic.
JAMES:
There was definitely a pretty Epic thread on the internet at one point about, um, Cuba and whether or not it was okay to use Cuba for like, uh, consulting, web apps and stuff like that. And one side was our game. Yeah. We've done it for years. We've had great success with it. We're doing fine. It lets us, you know, it keep our dependencies manageable. If we ever problem, we can actually crack open the code and read it and have a chance of understanding it and things like that, you know? And that the other side was arguing. Yeah. But you're going against the norms. So like developers who come in, you know, they all are gonna know rails and not Cuba and that, and then they're going to have to learn the differences and you're having to pulling other things from the ecosystem to handle some things that rails handles for you, which then the other side counters with things like, yeah, it's true. But rails also does XML parameters, which we've never, ever needed or wanted or cared about having or things like that. It's a very interesting thing back and forth.
CHRIS:
I have such a hard time with that because I have a such a hard time understanding the mentality of, um, this is what everybody knows. So that makes it good.
ANNA:
It's kind of where I was leading with that.
AMOS:
Not good. It's business-wise. Like, cheaper.
CHRIS:
Well, but I can empathise with like, with like that logical argument, like I understand how you could make that logical argument. I just like fundamentally disagree with it so much so that it's like, it kind of blows my mind that people make that argument like,
ANNA:
Well, I think the point that I was trying to get to, and maybe it wasn't maybe, and this is not, maybe this is not a valid point, but like, as we think about building systems, right. And when we're first starting or we talk about these norms, it's like, okay, we'll just use this thing that does all these things we're not thinking about. I mean, and why are we not starting thinking about, well, what are the things I actually need? What am I trying to achieve with the system? Right. So we do, we lead, we tend to use these more generic tooling then maybe. And then again, like maybe setting, having a set of norms is not bad. I just think it's interesting that we have this paradigm that we're in, that gets repeated over and over again across languages and communities, et cetera.
AMOS:
I mean, look at, look at the other side of it too. So open source software, you're putting yourself out there and as a, as a library author, um, it, it feels good. Even if you're not in it for fame and glory, it feels good when people are using your system. So if you can make it more general and applicable to more people you're to get more, more people using it, more bandwidth, more feedback than if you have this surgical strike library that, that a smaller subset of, of the developers out there are going to use. So the community kind of pushes you mentally to go things that are more general because yeah, like, like, look, there's a Phoenix. Phoenix has got its own talk, every elixir comf all over the world.
ANNA:
Right. But then you think, think about the book you're reading right now. Right? Like it goes like it's not these things aren't bad. I just think it's interesting. Right. We talking about right now and what James is talking about, which I think is really awesome is like we write less code. But when we first start, we talk about, we do have this paradigm using these dependencies that are more generic that inherently generally involve bringing in more code. Right.
AMOS:
And they're probably more junior developers too, which makes it easier to just grab this general tool. Right. Yeah.
ANNA:
No, I totally agree. I just think it's an interesting -
JAMES:
That's actually an interesting question though, to me, is it easier to grab something like rails and teach someone rails where you have to teach resource ragging and a bunch of other things just to get going? Or is it easier to grab something like Cuba and teach them like the three things that Cuba does, you know,
AMOS:
It probably is easier to grab something like Cuba, but they are more likely to have, uh, even if they're like first year developer, they're more likely to have looked at rails or looked at Phoenix versus Cuba.
ANNA:
Well, and I think the trade off there, we're talking about productivity and time, right? Like if you were first picking up rails, you probably don't have a deep understanding of everything that's happening. And if you needed to really, if you were really starting to fight with rails, but something that you were doing, it would be hard to figure out exactly what was happening. But for the most part, you can be up and running quickly. Right. Whereas the smaller dependency, you have a much better chance of early on being like, Oh, I know exactly what this is doing. And if I'm running up, running up against it, I have a much clearer idea of how to deal with it. Um, but maybe I need you to be able to build more things, to get to the state of productivity. So,
AMOS:
I was going to say, as a developer, uh, you know, early on in my career, if I have a choice between going to a shop, that's doing Phoenix or I find the shop that's doing racks, I'm going to say, well, I hear about Phoenix all the time. So that's probably where it's best for me to get experience. So I'm going to the Phoenix shop. And as a business that's hiring, this is the reason why there are so many businesses hiring tons and tons of Java developers because they're, they're available. There's people that understand it. And it's cheaper often to hire a lot more Java developers than it is to hire somebody who's going to come in and, and create some brand new piece of software. That's more focused for you.
ANNA:
Yeah, no, I totally agree.
CHRIS:
I think there's also, I don't know how, I don't know how I know how I feel about it. Um, I'm well, it's complicated. Like there's also this thing of like, if you teach, I mean, cause this is the thing I think about actually is, um, Rails bridge, right? If you teach people rails, you can get like this whole group of people, jobs like, like the barrier or rather like the barrier to getting them a job is one less thing now. Right. As, not only do they know programming, but they know like this tool and you can now, like, I want to use it - as long as you can like sneak them in.
ANNA:
There's more opportunity available for folks who know rails.
CHRIS:
There. There's just enough people in the world who only want to hire, who wouldn't hire like a Ruby person, but would hire a rails person or like that would be a barrier to hiring someone whether or not that's like, whatever bias is like coming through in that decision. Like someone will make the decision. Like I want to hire rails people, whether or not that's right or wrong. And like, that's one less barrier now in order to like, get that person hired, which is like, that's like an, uh, no, that's really real, but like the trade-off is like, would they learn as much, if not more like about the underlying fundamentals, if you actually sort of build it up from first principles. Yeah. Actually probably. And like, is that person going to be like set on a better path? I dunno, arguable. Like I learned most web stuff through like Java struts and later spring. And then I learned rails and it's like, you know, you, you get there eventually. Like, you know what I mean? Like, like if you just keep doing this long enough, but you know, it's, it's tough. It's tough to say like, I mean, I think we could be setting people up for like, for better intuitions about design and about things if we couldn't approach it that way. Uh, but then the, the, then the problem becomes like, I dunno, like if our goals are like, you know, if you like, think about a bootcamp, right? Like if the goal is to like get all these people hired at the end of the day, some large amount of them hire like some percentage of them hired. It's like, you don't want to be the boot, like as a business, even you don't want to be the bootcamp that does like the weird thing, because -
ANNA:
Well, no, because you want to get, well, I mean, again, it goes back to like, I think the very beginning of this discussion, which is like, it depends on the goal for the thing, right? Like it depends on the entire context. Like how much time you have, how many resources you have and what is the end goal that you're trying to achieve. Hiring somebody or if it is getting a site up and running really quickly, right. Or if it is, and you don't have a lot of domain knowledge, right? Like what is the goal? And then how do you think about achieving that and then weighing again, because it's always gonna be trade-offs time with complexity, with knowledge, right. And then weighing those so that you think you're making the best decision.
AMOS:
Um, and which school do you want to go to when you're learning the one that teaches you a general purpose tool that gives you the ability to, to do way more, or do you want someone to teach you how to do HTTP requests and then you have to learn everything else.
ANNA:
It's totally true. Um, I think it's just interesting, right? We're talking about like now the benefits of smaller, deeper things, but where does that come into play? Right. And I'm not, I'm not, I'm not, it's like a little bit retort.
CHRIS:
Right, right, right. But I, but I do think there's something there. And then as you said this earlier, like, you know, you get, you, you get more junior people who like, it tends to be that June, I'm going to use quotes here, which you can't see all the podcasts. Like.
JAMES:
Chris right now is moving his fingers up and down.
CHRIS:
Junior people, you know, do, I think, tend to reach for more off the shelf tools or rather they, like, I think they reach for like rules. They reach for like, I don't have an intuition about this yet. And so in order until I gain an intuition about it, I'm going to use whatever the community has, like agreed upon. And there's like a comfort in that. That's all part of it. I think like, there's that comfort level of like, I'm using a thing that's well established. The thing that I want to point out is, and then why I'm saying junior and air quotes is like, we're all juniors at some level about something. Like if I walked in, uh, some other language or some other runtime or some other domain, I would be, Oh, I'm doing okay. So actually I'm doing this right now. Like, cause I'm learning. Um, uh, you know, in my other time, like I'm learning more about ML and I'm doing more ML stuff and this is, um, a new domain for me. And I'm leaning on a bunch of tools that like people say are good until I gain an intuition about how these things work. And I think that a lot of people, we catch a lot of people in that transition, like, especially cause elixir is still so new, we're catching a lot of people in that sort of transition from their previous thing to the next thing. And they're just looking for like that comfort for somebody to tell them, like, this is the thing to use. You can like trust us because we've been around here and worked and we're telling you that, right. They're not ready to make those sorts of leaps on their own yet, potentially, which might add to that.
ANNA:
If you want somebody to get there and to build the intuition, they need to have something that helps them develop that. Right. And so they need the tools to get there.
AMOS:
Unfortunately it kind of goes back to just like a startup money, right? You have, you have a limited amount of time. You want somebody to be effective quickly and possibly in a broad range of things. So again, teaching them the general purpose tool or pulling in dependencies is what they're going to do. So a junior developer is a lot like a startup. And, and so maybe until you've been a little further into product to where you have money and have an extended runway, you can't, it's, it's hard to make those. Trade-offs
ANNA:
Kind of going back to this continuum, the James was talking about, right. It's not one or the other. It's kind of like, where are you on that? And where does it make sense, James? What are you, what are your thoughts on what we've been talking about?
JAMES:
I'm doing all my homework here because I have to answer all these hard questions for my talks. I figured I'd just ask you guys and then say whatever you say. Um, that's cool. Now I love what you're saying. And like as, uh, a person I might instincts mean closer to Chris's, um, I really try to rely on, like, I remember one time I was working on an app that had this hash diffing dependency in it and it was just the library. They given two hashes of arbitrary nesting. Right. It would, you could call death and it would go through and figure out what the differences were between the two data structures. And I remember when I ran into that and I was like, how come they didn't just, you know, do a recursive, walk down the tree and figure it out. Like, it's like, I dunno, it's a little bit of code. It's like 20, 30 lines. Right. And figuring out or something. And, um, but it's pretty straightforward. And then I ran into this issue where there's library that we were using did something in a way I didn't agree with, you know? Right. Did you do that? I said, so then I ended up opening up the library and I was like, not much more than 20 or 30 lines of code. You never. And um, and those kinds of dependencies, I, I mean, I'm not saying it's wrong to have a hash dipping library or to use it or anything like that. But in my instinct, if I ran into that problem, I would just write the do function first. Um, because I feel like I can solve that problem. And I would be confident that I would solve it the way I want it solved in this particular instance or, or whatever. And so I do lean on that and that experience. And, um, I would rather use a Cuba if I can get away with it. Like if I'm building a blog or a book to talk about a, um, uh, a URL shortner right. Where you have, what three, enter your URL, bang. Okay. Here's your shortened URL. And then the one action that isn't even a page, it's just a redirect that takes the shortened URL and pushes you to the yellow line. Right. If you start that with rails new or Phoenix new, is that a bug? I kind of think it is, you know, like, I mean, couldn't you build that or just straight up plug. Like that's not very tough, right? Like whatever. And my tendency is to reduce that complexity, but I, I liked, what's been said about the trade-offs and why we make those decisions and why we sometimes push faster and stuff like that. Um, it is a lot of trade-offs and like, while I appreciate the junior aspect and being able to give them the more general tool and the support and the comfort, sometimes I counter that with sure. Yeah. You don't know what this does yet, but guess what? It's like 30 lines of code. Let's go read it. You know, like when it does, you know, and I, I feel like you can get a lot out of that angle too.
CHRIS:
That's how you drive your career to some degree in my mind, or at least it has been for me, like taking that extra time when I can find it to actually do the thing myself or to like, learn about it, you know, like the diffing examples, a great example. Like if you've never written a tree, like deferring to trees, that's a good exercise to go through. It's going to take you like a little more time the first time you do it, but it's a worthwhile endeavor because the next time you come across that problem, like, that's a, that's a tool in your tool belt now. Like, you know how to diff trees. You're like, Oh, I've seen this. Like I, I can diff trees.
JAMES:
And that problem comes up a lot in my experience of rehab, this state over here in this state over here, what do they have in common? You know?
CHRIS:
And it has all these knock on effects of like, you learn a little about, about data structures. You learn a little bit about, um, uh, recursion, you know, you, you, you start to get a feel for these things. Like it only good can come from that. I think like from encouraging people to do more of that.
JAMES:
I feel like I've monopolized the heck out of this conversation, but there are two other people on this podcast giving talks at gigs for the elixir. So I want to hear about stacking theory and cryptocurrency.
ANNA:
Well, the talk that I'm giving, it's not about giving before, but the talk that I'm giving, um, at gig city, um, is essentially, and I've talked a little bit about this before on the show, but really the goal is really to try and remove some of the complexity that I think people think exists around cryptocurrency. Um, especially before new to that domain. Um, and help them get an understanding of it actually not like under the covers behind all of the jargon. I mean, there's some complexity there, but it's not as complex as people think. And that if they're interested actually of getting like digging into crypto is not unattainable. And so we talk about kind of how crypto got to where it is, but also how you might implement a simple version of like a Bitcoin based cryptocurrency in elixir, which is given the power of the beam. Right. And given the nature of language, it makes it pretty easy to do.
JAMES:
So. I watched your talk from ElixirConf and it was great because I think I'm perfectly in the target audience. I knew almost nothing about, uh, Bitcoin and cryptocurrency and stuff. And like, it was super approachable for me and I got a lot out of it.
ANNA:
Oh, thank you. Chris, what about stacking theory?
Speaker 4:
Yeah. Um, I have the talk and, uh, index cards in front of me on my desk all stacked up. All the post-its that are attached to the index cards are stacked up, but yeah. Um, so stacking three, I think, I don't know. I think J Lewis may have invented that name. I don't know if there's prior art on that. Um, but uh, he talks about this, this, he wrote this medium post, um, that I've sort of like I read and it really kind of, it resonated with me. So it's like, it's informed a lot of things I've done since then. Um, about being able to sort of look at a system as a bunch of - looking at rather not looking at a system, but looking at like the operational modes of a system as like layers. Um, so layer zero might be okay, so let's like, let's take a Phoenix app, right? Cause it's an easy sort of thing to understand. And you've got a bunch of moving parts to a Phoenix app. You've got, you know, your, your actual, um, a web server like cowboy, which has to hook a port, has to start accepting requests has to, you know, be supervised and run. And then you have a database and the database has to, you know, hook to the database and like has to talk to the database, has to establish TCP connections, like whatever it's going to do. And the idea of the stacking theory stuff is like, if you look at how that system runs, we could actually break down each of those different modes into different layers. So layer zero might be the OTP app or the OTP release boots like it's comes up. And at that point you can start to move to the layer one and layer one might be, um, where we have a web server running, but we don't have a database yet, but like the web servers running and then layer two might be now we have our external connections to the database and you move between these layers of sort of a best effort kind of way. So you do things like, okay, I'm going to try to hook a port. Now, if you can't hook the port, you fall back to like layers zero or whatever, what this allows you to do if you implement that correctly. Um, which is kind of the, the, the post doesn't really go into like how you do any of this. It's just more of like an idea. Um, so the talk of, hopefully it's going to help demonstrate like how I've been trying to do some more of this. Um, but the idea would be like, if you have, you know, your database connection and your database drops out and you're, you can't, you can no longer talk to the database again. Then you don't like, uh, kill the app or try to like kick over the instance or anything like that. You degrade back down to like layer one and layer one can then, you know, part of that process could be like sending out alarms or it could be alerting, um, your load balancer to take you out a load balance or whatever, but you, you degrade gracefully again and then come back up as need be.
JAMES:
Um, this sounds like, uh, is it run levels in Linux where, when Linux comes up, it goes through various run levels. Am I right about this?
AMOS:
Yes. It does a lot of, um, hardware development stuff too. You do that same kind of thing. Otherwise, uh, you know, if something crashes as you're coming up, you can't even light up, uh, lights on your alarm panel if you don't have different like levels. So yeah, it's, it's the same thing. That's what, what makes those two, okay.
JAMES:
Macr OS uh, if you use the Mac, sometimes you see this when, um, if something bad happens to your Mac and you go on to Apple's website and you're troubleshooting, they'll tell ya, hold down these keys and start up to boot into safe mode and safe mode is like a lower run level than you usually run it. And it doesn't try to bring everything up. And because of that, there's actually a higher chance that your Mac will successfully boot into that scenario. Right. And then you can troubleshoot from there.
CHRIS:
One of the things I've been trying to sort of emphasize too, in this talk is like, you may or may not need to do this depending on your infrastructure and your application concerns like this is, I mean, these sorts of things are a real problem at Bleacher report because of like the amount of things that we run and the number of services, the number of nodes, the all that kind of stuff. And we, and we see those degradations, like on pure probability, we see those, those things a lot more, let alone like the, the kind of traffic spikes that we get. So, um, which tend to force degradations to happen, uh, or at least like increase the probabilistic chance that those degradations will happen. So we end up having, like, you know, I think it's a truism that basically any sufficiently complicated system runs in degraded mode, basically all the time, like something is not working almost a hundred percent of the time. Now, how much that affects you is, is really what we're trying to like balance. Uh, but the trade off is you bring a lot of these, like complexities into your system. When you ask the system to take, to handle it, like when you're starting to do these things yourself, you know, you're, you're owning a lot of that complexity and you really have to weigh whether or not that's like right for your system. So like, your blog probably doesn't need to do this, you know, but you know, any, like if you have a sufficiently complicated system that needs a certain amount of availability, you may want to start to like, approach this problem from that. And that, and for that standpoint,
JAMES:
It's cool too, to like, not have your app be two binaries, right. Either it's up or it's down, right? Like, you know, maybe these parts are up, I'm like this thing's down, you know, and not, not having to have those does. Um, this is my own curiosity, which you probably answered in the talk, but does, uh, elixir make this kind of building easier? Like, because of the way it's designed with fault tolerance and stuff like that.
CHRIS:
Yeah. We get, I mean, we get to, we have all these primitives that we get to just rely on to do all this kind of stuff like supervisors and whatever else, um, at is actually a big part of that, because I mean, we use EDS for basically like, uh, uh, uh, cash all the time, like a local cash, um, very often, especially around like circuit breakers. Uh, so if a downstream service is down or is like overwhelmed and we need to like back off of them, we'll just like kill the circuit and then fall back to a cash. And the cash has maintained inside that stuff like that. Um, and that's sort of a, that's another layer to this. Like, can we make outbound calls? Yes or no. Um, and like, what do we do when we fall over? So we just have, we just get this run of this great runtime, like, as we were talking about earlier with all these amazing primitives to start to put these pieces together. Um, but there's definitely nothing right now that I think there's, there's something right now that like, we'll do this for you. Like, you can't just use like the default supervision stuff, like one for one, um, out of the box, like you start to have to really like, think really pretty hard about how you're going to supervise these things. And you, you need to some degree get outside the bounds of the supervision, like of what the supervisors themselves want to do. So like one, like a concrete example of that is, um, one pattern we've used is to have a supervisor that starts, uh, effectively like a watchdog process and a dynamic supervisor. And the watchdog is like, what actually starts the real process that is managed underneath the dynamic supervisor and that way, and it monitors it. Um, and that way, if it crashes, it can do like exponential backoff, as opposed to like letting the, the normal supervision stuff like cascade and like kill the app or whatever. So we do you get those primitives, but sometimes you have to do that, like some of that stuff yourself, um, and like, you know, just use the tools that are there, um, in order to do those sorts of things.
JAMES:
That's super interesting
CHRIS:
Instead of relying on like your normal, um, restart intensities and all that kinda stuff. And we do that too, but there's, you know, you, you start to play around with a lot more of these like strategies and those sorts of things.
JAMES:
Have you ever seen Gen retry?
CHRIS:
I have not seen Gen retry. What is that?
JAMES:
I can't remember if it's Erlang or Elixir. But, um, uh, it is basically, uh, supervision with exponential back off and stuff like that. And, Oh, cool. That sounds, that sounds useful. Okay.
AMOS:
I think there's also one that's just called retry when but it may not be supervision trees, so that may be why.
JAMES:
I think Gen retry does like supervised but I may be wrong.
CHRIS:
That's, that's pretty useful. Cause I mean, getting the supervision stuff, getting that kind of stuff to work in the supervision trees is somewhat non-trivial right. Because it's not really what the supervisors would want to do. Right? Like that's, that's not really their normal mode. So you're getting outside of their normal mode to some degree. It's also why I was really excited when they brought like, when they built dynamic supervisor. Cause I was like, Oh sweet. We have this like really good tool that can do all this stuff for us now. Like I use that so much. Yeah. I'm excited to get it written.
JAMES:
I can say I've made the same place. I got all this stuff and it's like, Oh yeah. Now I have to put this together and turn it into a talk.
ANNA:
I'm excited to hear both of you, talks are gonna be great.
AMOS:
I'm pretty excited about the whole conference.
CHRIS:
I'm just excited that you're all coming to Chattanooga. I don't have to go anywhere.
JAMES:
It's going to be cool. I've never been there.
ANNA:
It's super nice. I like it.
JAMES:
We are also going to have a live recording of the Elixir Outlaws at Gig City Elixir.
AMOS:
I don't know if that was a good or a bad choice.
ANNA:
We'll find out
AMOS:
It'll be our first live recording. So, um, it should be exciting.
ANNA:
It'll definitely be exciting.
CHRIS:
I don't think anybody believes that we're like super put together, but now they're definitely going to have it confirmed that we're not.
ANNA:
That'd be great. It's going to be well they'll know. Right. They'll see. They'll see the truth.
JAMES:
Get to see the sausage made.
CHRIS:
I've been trying to think of what we should do. Like maybe like we should have a section of like Q and a or something like that. Like a little more engagement. I don't know. I feel like it's going to be weird if we're just up there talking to each other for that long in front of a bunch of people. Yeah.
AMOS:
Well that's, I, I have, uh, an extra mic that I'm planning on bringing. Um, so, so that'll give us a, another mic to have, I think we just have a hot seat. Just let everything, but anybody come up who wants to. Here's your, here's your platform to say whatever you want. Uh, so, um, on that, I'm going to go back to, to James is really quick on that continuum. He, you ask where on the continuum, should we put that point of, of picking a library or building it ourselves? And I'm going to say a 7.4.
JAMES:
I am making a slide right now.
CHRIS:
I was going to say 42.
JAMES:
"Amos says 7.4" - that's the slide. That man, right there in the audience. If you would like to go talk to him afterwards, he would love to discuss this with you. Can you have a second slide that says, Chris said, well actually it's 42?
ANNA:
I was gonna say, thanks James, for being on the show. It's a super fun. And for bringing such good question.
JAMES:
Thank you. It's been a long time since I've got to have a great discussion. Like this is really great.
ANNA:
Any parting thoughts from anyone?
AMOS:
I don't have thoughts.
ANNA:
Other than we're super excited about gig city,
JAMES:
Come and see all of us there.
AMOS:
If there are any tickets left, I think, but by the time this comes out.
JAMES:
There are a couple of tickets left. We had some, uh, we have, we almost ran out and then we had some sponsors that had like blocked tickets. And so we went to them and we're like, okay, we really got to know how many people are you sending because we're about full. And so that allowed us to release a few more. And I think we're at about 15 right now.
AMOS:
Sweet. Now's the time to scalp. We can make some money.
ANNA:
That's not what I was thinking, but. Sure.
JAMES:
That's not what you were thinking?
AMOS:
We should build a conference tech conference ticket scalping app.
JAMES:
Yeah. Bring all the bad parts of sports to tech conferences. I agree.
CHRIS:
That's good. I like where your head's at Amos.
AMOS:
Well, I know that the Anna's has got to get going.
JAMES:
It was good to see you and I look forward to getting to meet you in person.
ANNA:
Thanks. Y'all so super fun. It was so nice.
CHRIS:
Thanks for being on the show, James.
New Speaker:
Yeah. Thank you. Thank you.
AMOS:
And tell the family that we all said hello.
JAMES:
I will.
New Speaker:
See you later.
New Speaker:
Have a wonderful day. Bye bye.
