103: Offline Data Sync: Cautionary Tales

Jon (00:01.203)
Okay, good.
Jon (00:06.55)
Welcome back to another gone mobile Alan. What what's what are we up to on this Friday?
Allan Ritchie (00:11.838)
you're right into it because it's Friday right? We are going to talk about offline battle tactics and more specifically I'm pretty sure you've got some war wounds here and this is a hot topic for me this is this is like my bread and butter when we talk about a lot of Maui mobile stuff so but I love your war wound I love when you go on a rant so I want to hear it I want to hear it they all want to hear it everybody wants to hear this John going to rant
Jon (00:13.93)
Yeah, we gotta get going.
Jon (00:23.179)
Yeah.
Jon (00:37.558)
I'd surely you have one too. Like who hurt you in this space?
Allan Ritchie (00:42.542)
Um, well, I'm in open source. You are too, actually, technically so.
Jon (00:47.07)
Yeah, I mean, everyone in open source has been hurt. So.
Allan Ritchie (00:49.914)
Okay so a product? Oh I- many. Many. I- I've seen some I've seen some things that have caused my eyes to bleed. That's why I have grey hair. More than you for some reason, but I do.
Jon (00:52.062)
I don't know, or just experience, project, experience. Yeah, okay. I've seen some things.
Jon (01:05.226)
It's coming in here. I don't like the beard is getting uh, you know more gray than it when it started I didn't used to sport it but covet. I was like, let's try. Um, why not? Not that big no, I got once it gets too long it gets annoying so I had to trim it back
Allan Ritchie (01:16.434)
Did you go like full out? I don't even remember.
Allan Ritchie (01:22.986)
See, I'm French. I don't get hair. I only get hair on my break right here, dead center. And that's it. That's it. At least I'm not going bald, but that's it. It's just this one spot on my face and it doesn't... Yeah. French-Canadian, eh?
Jon (01:27.008)
Yeah.
Jon (01:31.787)
Yeah.
Jon (01:35.838)
So speaking of hairy experiences, so we're talking about off, did you say that already? We're talking about offline stuff? Yeah.
Allan Ritchie (01:39.354)
Yes, let's hear the rant.
Allan Ritchie (01:43.598)
Yeah, offline battle tactics and then I added the scars point because we I want to hear you rain. I've heard it and I want to hear it
Jon (01:48.002)
Yeah. So I think you have, yeah, I've probably told it a few times. Maybe others have heard it too. So when I, I have an app, right? Did you know that? Okay. I have an app. It's about pools. Yeah. If you have a swimming pool, it helps you figure out your chemistry and do that kind of thing. And one of the aspects of it is it stores logs. You're like information, right? So if you do test your water, you can store like all the different numbers that you test with.
Allan Ritchie (01:57.498)
Yeah, I did. I've seen it. It's about pools and stuff, right?
Jon (02:16.79)
You can store like when you do maintenance and expenses and all the different stuff. So anyway, the idea was let's make this app. Let's have this data. I need a database. Okay. Um, how do I want to make that work online? Because let's face it, like users of apps kind of, what are the chances that somebody's going to like go back up their data that's locally stored and then know how to restore it and like do the good thing of, you know, being smart about it and not come back.
to me like a year or two later and be like, I lost all my data, what do I do? Right? So gotta have it stored online in the cloud. And that's what I decided the feature that I would make people pay for, which kind of makes us even hurt more because it's like, okay, now people are starting to pay for the app, a yearly subscription. And also if, well, we're ranting and scars, like if you ever have an app in the app store, if you've published one and you get reviews from people, people are...
are the worst. Like the reviews that you get. So like this pool app is, is meant to help you deal with your own chemistry. And if you go to the pool store, if you, anyone who owns a pool knows, like the minute you step in the door of a pool store, you're not walking out with like spending less than a hundred dollars. Just that, that's a good, you know, that'd be a good experience. So they, they want you to come in and it's a whole, I won't get into that side of things. We should, we could do an episode on pools one day. And, um, it's a whole thing.
Allan Ritchie (03:35.122)
Yeah, okay.
Jon (03:44.402)
And so this is helping you save money by not having to go bring your water into the store and test it and figure out what you need to do with it. So if you have a pool, you have a little bit of extra spending money usually. Right. And so I charge $8, 7.99 us a year. And the amount of reviews never, you know, seems to go away that have people that are like, I'm not paying $8 a year for this thing to help run my pool. Right. So, um,
Ah, I'm so scarred by App Store reviews as well. It's terrible. So if you're gonna like leave a review on an app, don't be whining about like the cost of it or something. People are working and building these things, right?
Allan Ritchie (04:29.154)
I feel like John's winding me up at the same time because he knows how I approach some of these things.
Jon (04:33.822)
It's like open source development, right? Like you're spending your time and giving it away and, uh, arguably it's even worse for, for OSS dev because you're not charging money for it.
Allan Ritchie (04:45.48)
Did, can you bleep out my saying?
Jon (04:48.109)
Hold on, let me get the button ready.
Allan Ritchie (04:51.122)
Fork and fuck off.
Jon (04:53.262)
I, yeah. So anyway, I have the app, it has this feature, it does offline data. And when I built this originally back in the day, I built it with Xamarin, I built it with Xamarin Forms. Part of why I wanted to build it was a good experience of just like using the product and getting a better understanding, you know, what the customer experience is like, whatever. So I started looking into solutions, like how do I do offline data sync so that
somebody can put something, you know, a database record in the thing and it gets to the cloud and if they log in on another device and, and I, because I've been such a cross platform user of, of mobile too, right? Like I was an Android user for many years. I'm iOS, you know, I've, I've come around and seen the light and I've been iOS ever since, but I wanted it to work everywhere and I wanted to be able to sign in on any platform with that account and get that data down.
So I started looking at the options and there wasn't a lot out there to begin with back in the day, but there was one.
Allan Ritchie (05:56.064)
There was not. There's a firebase database.
Jon (05:57.718)
Well, there was some there, what, there, no, that wasn't back then. That wasn't around. I did this like a long time. Like this app is, uh, I want to say 20, like 16, maybe 2015. I don't know if it was out, it was like early days and I don't think they had all the different versions of what they offer for database stuff now. Right.
Allan Ritchie (06:05.094)
What year?
Allan Ritchie (06:16.03)
Hmm. All right.
Allan Ritchie (06:23.582)
Alright, alright. I know you went with the cool guy though because it was really sequel light and The one you're about to say that it's right at your lips. I'm waiting for you to say it because then everybody's
Jon (06:29.215)
Yeah, SQLite was there.
Jon (06:33.322)
Uh, so there was a thing called realm. And funny enough, I think I knew about them from going, I think I went one of the, the Google IOs that I went to, there, there was people from realm, like they're promoting it and passing out like, I don't know, cups or something like that, like coffee stickers, who knows what it was. Yeah.
Allan Ritchie (06:37.483)
Yeah.
Allan Ritchie (06:52.55)
The concept was cool. What they had, it was an interesting ORM. The way it did its data persistence and like you had to be on the same thread. So I personally, I never used it. I used it with a couple of companies that I went in and consulted with and it was a nightmare, a nightmare, but not like yours. Yours was different.
Jon (07:04.408)
Yeah.
Jon (07:11.487)
Yeah, so.
Jon (07:15.078)
It was, and the basic functionality was okay for what I needed. There was a few things that were probably in hindsight, red flags. Well, definitely were. Um, you know, one of the things was like, there wasn't an easy way to kind of capture a view of all of the data. So like the way it worked was kind of on a per user basis, which was great in terms of the functionality I wanted. Cause I only wanted you to be able to access your own data, right? Like that's fine. Great.
Allan Ritchie (07:42.438)
Yep.
Jon (07:44.598)
But the, one of the problems was like, it was hard to be able to kind of, um, as I found later, trying to migrate out of realm, it was hard to like get everyone's data and like be able to back it up and migrate it over. It was very proprietary. You were stuck on their system and you know, you had to do things their way. But the bigger problem was, so here's, here's the thing that happened. I had this all working. It generally was okay. And then I went and I launched the app.
And put it on the app store. And, and because I wrote this app and I partnered with, um, a website and a, another company to do it. And, you know, and that in hindsight was excellent decision because it's been a great relationship. But we had users immediately, like a good amount of users, which is great. Good problem to have lots of traffic. We had, you know,
Allan Ritchie (08:38.198)
I just want to stop you and just so people that aren't, they don't get to see your face as you're describing this. You can almost see John amping up as he goes further into the story. It's quite entertaining. Oh, you're just getting fiery. I love it. Let's go.
Jon (08:46.55)
Oh yeah. It's been a while since I've relived this. So this is good. Yeah. So, you know, get, we get users and like starting to get people logging in and subscriptions and, you know, I'm starting to like monitor things like, yeah, okay, this is great. This is working. And then all of a sudden, you know, get like the first email or whatever from some of you like, uh, I can't log in. Oh, that's weird. Okay. Well, let's, let's take a look. Oh, I can't log in now either.
Uh, well, it's, it's working sometimes. So every 10th time I try it seems like I get through, but it takes a while. And then after I finally log in, things are okay-ish. So long story short, the way that they had stuff set up, the, the login service was its own individual process thing, and it talked to the realm service. And the realm service itself was, I think written in like C or something. And it, it seemed to be okay.
But the login service was no JS based and like it just is, I didn't even have that much traffic. It immediately died out and like it was like I was getting, you know, DDoS on the login service when I had maybe five or 10 people at a time, you know, trying to log in. So the whole thing just crawled to a halt and I'm, you know, panicking cause like this is the, Hey, I'm ready to go. This is the moment. This is the release.
And so like I started getting in contact with them and I get in touch with some of their developers and they're like, Oh yeah, this is a problem. And it's like, what do I do? And eventually after like two days of just nonstop working, I had to rewrite the login service myself and I could only do this because of the, you know, to be fair, the engineers at least were like holding my hand a bit in terms of
Here's what the flow is expecting and how it pans off to the database service and all this kind of stuff, right? So they're like, one option would be to rewrite this yourself. So it was faster. It's like, this is your product people, come on. You should be doing this for me like right now. But they didn't. And so I rewrote the whole thing. And I think I even did it in Node because of some reason that, some thing they were using from it. But I'm like, how come I can make this work?
Jon (11:11.478)
fast enough, but you can't. And I had to like set up all these different like reverse, like engine, some proxy configs to get things routing to the right places. And like different endpoints were needing to be set up for each user and stuff like that. It was, it was the most painful thing I have ever done. I think from a development perspective and just having to do it quickly because people, you know, I've got this onslaught of users and there's problems and nobody can get at the thing. So I eventually got it going again.
But to the point where I'm like, I'm not comfortable making this my stack now. Like I need to get off of this as soon as possible. And then that was a whole other process because like I said, you couldn't easily query everyone's data. I think there was a way to do it. They had like an upcharge product, but like even all the stuff I went through, they're like, no, you'd have to buy that too. And it was like a lot of money, right? I'm like, I don't have that money right now. Um, I'm like, I just, I need to get my data out. So I, what I ended up doing is.
I rewrote a different backend as well, right? On top of rewriting the login service, I had to like totally figure out, okay, what's my new strategy for offline data? And then I made the app, I put an update out that would kind of go like, okay, if you're launching the app, if you were a Realm user, if you haven't been migrated over yet,
Um, go get all your data and then like send it up to the web service so that I can store it properly because I couldn't get that user accounts data outside of using their own client and product. And then I got it all moved over. And then I had to have this thing running for like a year because over the course of time, as I was rebuilding the new backend, like I'm still using this thing and I'm getting thousands of users signing up and not everybody opens the app. To do, to have that migration happen. Like.
You know, immediately, right? So I had this thing, probably at least a year, I had this thing running on top of the other one and doing these migrations. And I only like a year or two ago, finally, like totally tore the rest of every part of that updating migration piece out of the software. It was, it was so painful. And, and I will, I know things that like they've gotten, I think they were bought by couch base, right? Like, I think like realm is still a thing. And I know they were adding like Maui support and stuff.
Allan Ritchie (13:28.75)
Yep. Is it counter-face? No. Is it q- Yeah, MongoDB.
Jon (13:33.782)
Wasn't it? Oh, my was Mongo, right? Yeah. Sorry. Couch base is another product out there for, or was first for similar stuff. So I will never use realm again because of that experience alone. I don't know. Maybe they've gotten things under control and it works way better these days. I know one of the things I still didn't love about it. And I never did originally was that you had to like subclass their types for your data objects and stuff. Um,
And so like just to use it required some infiltration into your code base that I don't love.
Allan Ritchie (14:09.382)
Is this where you started your I must build it myself from that point on? Is that the battle scar that did it?
Jon (14:14.586)
You know, that might be the origin story.
Allan Ritchie (14:17.83)
because you are like, you will avoid a dependency at absolute maximum cost until you realize like, yeah, I'm not building this, like this I'm not building.
Jon (14:24.82)
Yeah, like fool me once.
Yeah.
Allan Ritchie (14:29.682)
But yeah, well, a lot of people, so I did a lot of consulting back in early Xamarin, there was two companies that I saw actively use Realm and actively destroy themselves with it, right? Like, now they had other problems like bleeding through the tears of the engineering process. Right, so like your backend data going all the way through your UI, you know, there's reasons we don't do that stuff anymore. But you know, I saw those
Jon (14:44.046)
Sure.
Allan Ritchie (14:59.856)
people generally trip and fall on their face and that was just realm. But if we go back to offline, I mean did not offline is such a common thing in mobile and the number of times I've seen companies fall on their face with that is tenfold like
Jon (15:03.627)
Mm-hmm.
Jon (15:16.67)
Well, because it's not necessarily an easy problem to solve.
Allan Ritchie (15:20.406)
It definitely isn't. It definitely isn't. But what I tend to see is like with all engineers, right? I think maybe this comes as we're older. It's
You know, you want to build this abstract thing that you can reuse over and over and over again. And to be honest, once you start getting into like an enterprise database where, you know, you've got row level security, row level security, even column level security, like certain bits of data or views or whatever, syncing becomes nearly impossible on an abstract level. Right. Like it really does. But people try to bet that wheel.
Jon (15:34.784)
Right.
Jon (15:53.567)
Yeah.
Well, and what does it mean? You know, I think like where it falls down. Yeah. Exactly what you're saying. Like, how do you also define what it means to sync your data and how to handle, um, conflicts and like, this is all. Yeah. Right.
Allan Ritchie (16:11.854)
Oh, yeah, I haven't even gotten down that rate because.
resolving conflicts sometimes is a business decision, like whose data is right. Maybe they both are. Maybe there's two audit histories. Who knows, right? Um, they try and put in these conflict resolution things into some of these, these over engineered frameworks, and you end up doing more code trying to figure out how to resolve the conflict as opposed to that wins or they both go in and, and just like, just do the business. Right. And it gets even worse because these,
Jon (16:18.294)
Mm-hmm.
Jon (16:40.351)
Yeah.
Allan Ritchie (16:44.616)
You know, there's always, it's like OData. Have you ever used OData?
Right? And you spend, you're like, oh, wow, this at first, you're like, this is pretty awesome. You know, I can filter, I can query, and it's the same thing with graph QL or graphical, whatever you want to call it. Right? You spend all that time, you come up with this great model and then you spend more time for all the time you saved for writing all these queries and stuff. Now you spend trying to throttle them because people find complex ways to query your object model or ways to hack it or issue deletes or, so you spend all that time.
Jon (16:55.494)
Mm-hmm.
Jon (17:14.007)
Yeah.
Allan Ritchie (17:19.552)
that you just spent using that thing, you now have to secure it. You spend more time there. And security can be hard because then you lock somebody out and you're like, oops, ehh. Right? And so your abstract thing that was all pretty is no longer pretty.
Jon (17:34.826)
Yeah. So do you want to, do you want to hear what I actually ended up moving to? Like, and part of, part of the, I, I guess part of the success in the simplicity of what I moved to is also due to the fact that I'm like, Oh crap, I got to do this thing fast. Like, what's the, what's the easiest way I can do this that doesn't feel like it's going to paint me into a corner in the future. So, you know, I basically got to the point where I'm like, okay, well I have like,
Allan Ritchie (17:38.626)
cough
Yes.
Jon (18:04.478)
My data is simple, at least for the app. It's logs, right? Like I have a, if you think of it from like a C, like a.NET and an object oriented perspective, I have a base log type. So every log has a timestamp and is associated with a pool and has potentially like the current weather conditions attached to it. And then I have different types of logs, you know, that subclass that basically. And so like my table, I've basically,
pretty much two tables. I mean, I use, um, Cosmos DB. So it's like, it's SQL document, which there are pros and cons to having gone that way. Generally it's been great. Um, but the, the data part of it, yeah, really boils down to like, okay, I have logs you're, you're going in pools. Like you have a pool, uh, table, I guess, and a logs table. Well, pools are something you don't create that often. And so this is again, like where you get into thinking about this, it's really,
building the solution for your business state, right? So pools don't change often. You basically set that up once, cause I only really ever put a swimming pool into my house and my yard one time, right? Maybe I edit it or who knows, it can change, but it's not gonna be that often. So, thinking about how to do that offline, it's like, well, it doesn't really matter too much, as long as that kind of gets updated at some point. And then,
I'm creating new logs most of the time. Sometimes I'll edit them or sometimes I'll delete them, but most of the time it's creating new logs. And so I decided, I said, you know, for this case, I'm just gonna say if, if when you're creating or editing something, either a pool or a log, you have to be online. If you're not online, it's gonna air out and it's gonna say, hey, check your connection and try again, right? Now I have plans for, I had plans, I have plans and I don't know if I'll do them because it's been fine.
Um, for basically trying to say like, well, could I store those log, like maybe not for a pool, like maybe a pool, you always have to be online, but maybe for adding a log, I could say, okay, I'll keep that cashed offline and I'll send it up the next time, uh, your phone connects and we'll deal with, you know, problems of, um, conflicts in just a simple way of there are no conflicts. You can't edit, you can only add a new log. Like that's one way I could go with it. Right.
Jon (20:30.63)
If you add a log in two places, well, they just both get added and you can go delete one later if you feel like you didn't want both of them. So I'll, I'm really just sending things up, right? So you have to be online to send things up. And then before you can send things up, you act also have to like check with the server to say, do I have the latest set of information and it's going to pull the latest information down before it sends stuff up, not that there's really any conflicts there, but just so that it kind of.
you make sure you've got things sorted out. So that's worked out pretty well. I don't really have people complaining like, oh, I'm trying to save logs and I can't because I'm offline. It's like people are online when they try and save them. So it's been great.
Allan Ritchie (21:10.75)
Well, and generally they're near their house anyways, unless they've got like Wi-Fi from, you know, 20 years ago, they're probably already.
Jon (21:14.014)
Yeah. Well, and that has been a thing. Like people don't necessarily have wifi that reaches where they're logging from, but most people have cellular connections on their phones. Right? So it's, it's really not been an issue or what they'll do is they'll like put the information in and then make sure they're back in range of their wifi when they hit the button or, or like when they hit the button, if it's like, no, no connection, it doesn't.
go away like they can just try again when they get back to a connection state.
Allan Ritchie (21:44.318)
Here's the thing that I love because this is this is a thing that I see companies do a lot is You haven't added offline yet, right? You're thinking about it, but you have no
Jon (21:52.086)
Right. Yeah, I made a plan to be able to if I wanted to.
Allan Ritchie (21:56.942)
Right. But a lot of companies will just be like, Hey, we need this out of the box. And they don't understand the amount of engineering that has to go into these things. Right. Like even when you're like, let's say we're not rolling our own, but we are doing, um, we're just going to do data syncing, like straight data syncing, which was, which is what I usually suggest. But then you start thinking about what about deletes.
Jon (22:03.307)
Mm-hmm.
Allan Ritchie (22:19.246)
If I delete it on the server, if I'm in a multi-user environment, what happens if something that like a work order, I love these ones because this is common, right? So I have work that I'm assigned to do, but what happens if my manager decides, no, Alan's not going to do it anymore? So I come up from the basement to go do this log. So I don't have access to the information because I shouldn't see what everybody else is working on.
Jon (22:26.923)
Mm-hmm.
Allan Ritchie (22:44.89)
But that work order has been removed from me. So the data won't pull anymore. So it means I have to pull different data to actually get it off my phone. So it deletes and updates like that.
And again, we're talking about a security level model is completely different. And a lot of companies don't think about what it's, it's a delete. Just remove it from the device, but it's not that easy. I, cause the normal tactic, right. Is to just say, get me everything since the last time I got data. Right. So your phone will say, well, I got data two hours ago. So give me everything that's done since. But what it's not going to do is it's going to be like, well, you don't have that assigned anymore because hopefully your query is smart enough to keep those things that I should.
Jon (23:03.968)
Yeah, just get rid of it.
Allan Ritchie (23:24.96)
shouldn't see from being sent to me. Some don't and I've seen a lot of that too where they're like, it's a good thing I don't see this data moving because you're sending me a lot of stuff that I shouldn't see. I've seen financials move across the wire and it's like, what are you doing? What are you doing?
Jon (23:26.466)
Mm-hmm.
Heh, yeah.
Jon (23:36.618)
Yeah. Oh yeah. Yeah. Didn't, didn't think that went through. Well, and that, and delete. So deletes was one of the things that I did have to think about a little bit and what I ended up doing with it, because again, like you could delete it on one device, right? And that would sync up to the cloud. But like you're saying, how do I know on the other device that should go away now? So I actually, I don't know if I love this in hindsight so much, um, but I, I basically made another.
type of record, which is a delete record, which says, you know, this other record identifier has been, should be deleted, is deleted. And so I sync those down and then I process through those as well.
Allan Ritchie (24:07.772)
Right.
Allan Ritchie (24:18.714)
So in that smart, but like I said, even with updates, right? So something as simple as being updated and removed from me now has to still trigger up to me somehow. So you end up with all these extra events, right? So it's either you, you end up rolling, um, you end up doing like replication model, right? Like a data, like truly like a database. And, and that has its ups and downs too, because now it's like, well, no.
then that allows the API for me to kind of free flow updates of any column through, right? So that's why I generally tend to say, just write your APIs, man. Just write them. And then if, for instance, if you're doing something that you shouldn't be, have the API error out, you could signal right there, right? When your data goes to sync and say, you know, here's a toast message. Yeah, this work order that you tried to complete, your manager must have already done it for you. So, yay.
Jon (25:15.106)
Yeah, it's no longer assigned to you yet.
Allan Ritchie (25:15.918)
And maybe that just removes it. Maybe it removes it from your device. Maybe it lets you know or maybe it tells you no, you need to add some, how long did it take you or something, right? But that's it's all business decision writing an abstract for that is I don't want to say it's impossible, but I have yet to see it succeed. And
Jon (25:35.278)
But it's heavy, right? It's a lot of work. And then like to understand the inner working so that you do the kind of other parts of the implementation using that correctly is hard. So that like in this case, so my simple, you know, the simple thing turned into, okay, I have like three record types, like just always ask the server and say like, the one I have, the newest thing that I have locally, the newest timestamp for like any changes for this, for a log is this timestamp. And that timestamp isn't.
Allan Ritchie (25:44.855)
Exactly.
Jon (26:04.77)
local thing. It's on the record data that came from the server, right? So it's all, you know, the server is the only thing that ever says here's a timestamp on a record. And so I get that information down and I just say, for logs, here's the newest timestamp I have, like give me anything that's changed since then. And then the server's like, oh, well, there's these other logs and then there's these delete records as well. So go take these and update yourself and you should be perfectly current in your state of data.
Allan Ritchie (26:33.902)
And see that that's again another great model. You didn't have to invest too much down on that. And it just it's gonna work and it works for your business.
Jon (26:44.106)
And the code is easy for it. It's really not hard.
Allan Ritchie (26:47.478)
It's not, it's not in that case. But like I said, there's so many, as soon as you start to break out of that, what about parent child, right? So you start, you start, okay, well I deleted the parents. So then you're, you now have to deal with all the children. Do you let the database do that? Do you have code that does that again?
Jon (26:53.663)
Yeah.
Allan Ritchie (27:04.314)
And you don't necessarily know what the developer's built, because a lot of the times these things might be built after. They're like, oh yeah, we need offline. Well, you probably should have ramped up to be prepared for that, right? You know these ins and outs. So that's why we're calling it end scars. So it's tactic and scars, right? So you deleted the parent or reassigned it or whatever the heck you did. Now you gotta get all that child data off my device, because one, I probably shouldn't have it. Maybe it was assigned to me wrong. And now I can see all the bonuses for everybody else
Jon (27:14.295)
Right.
Jon (27:18.474)
Yeah.
Allan Ritchie (27:34.208)
going out. It's like I'm using terms I've seen, right? Hint hint. Right? Maybe I shouldn't see those. Maybe it was should have been a manager that got that data, right? So they should be clearing off my device too. Because you know, it's easy to pull a SQLite database off a rooted phone. So just it's all these things. Right.
Jon (27:51.03)
Oh yeah, yeah. No, it's not, not hard to do. So yeah, they had no, no sensitive data there either. Right. Um, and so like the, another aspect of this, I mean, I guess backing up a step, you know, I think we, we kind of glossed over a bit. Maybe you mentioned it, but like the easiest way of course to do this all is just to not do it.
Allan Ritchie (28:11.186)
just to not do it, right? You can, I'd say pull your data or cache it for the stuff that you're reading actively, right? And then the second you're online.
Jon (28:19.288)
Mm-hmm.
Allan Ritchie (28:22.982)
You know, you can you have you're looking at your cache data and it goes, oh, you're online. You haven't you haven't pulled data in five minutes. Let's try and get it. You're looking at the cache data and when it refreshes, we'll tell the user, hey, you got new data. But you've done that you've done the good thing is that the creates the updates like any actionable thing that the server needs to be aware of. You tell them simply, hey, no, you know, block the button or put a nice toast or I can't I call it a toast a banner.
Jon (28:47.039)
Yeah.
Allan Ritchie (28:51.366)
Put a nice banner that says, hey, you know, your phone sucks, or you're, you know, welcome to, um, Leamington where there is no cellular internet, right? Hey, um, sorry, I had to, I had to do that. I had to do that to you. Um, sorry, John's got enough wifi in his house to, to support his entire city, everywhere. So, um, you know, it just.
Jon (29:00.67)
Yeah.
Jon (29:10.331)
Oh yeah, exactly.
Allan Ritchie (29:17.734)
Those kind of things help the user know, hey, this isn't working well because you're not online. Just get online.
Jon (29:23.67)
Well, and I know you're a fan of doing that in your apps a lot too, right? Like you, you do a lot of, um, uh, I guess monitoring of connection state to kind of update your app UI and that kind of thing.
Allan Ritchie (29:37.806)
Not so much the app UI, but it's simple things, right? So if you're doing, a lot of people take pictures, right? Like it's very common in an app to take a photo or a video or something, right? Like even, does Poole Mouth take photos?
Jon (29:50.59)
I, so this is another thing where I like, you know, when I first did this, I'm like, oh yeah, I should make it so you can take photos. Like what if you want to log, you know, that you did some maintenance or whatever and you want to attach your photo with it, which would be really cool. And something that I kind of thought, yeah, I'll get to that, but I haven't yet. And you know, there's been a few people here and there that have been like, Oh, it'd be nice if I could do this. But I'm like,
I don't know, it's not in demand. So like, why, why build it out before you need it again? Right.
Allan Ritchie (30:15.718)
Here's the pool this year. Last year.
Allan Ritchie (30:22.158)
I guess I've worked with enough apps where they tend to take photos, in which case photos are going to be served back, right? So they're going to and from the server. So one of the things I generally tend to see is, you know, people take the photo, they hit, you know, save it. But I don't know about you. I know you've got the iPhone 15. How big are the photos at the largest? Just a guess. Right. And
Jon (30:41.602)
Oh, geez. Oh, I mean, they, I don't even know anymore. Probably 15 megs or something. Yeah.
Allan Ritchie (30:48.814)
You know, people just, we kind of take LTE for, you know, we take advantage of it. It's fast, but it's not fast enough that I can press save, swipe my app away and, you know, all right, I'm getting the pull. Yeah. So, and then what generally tends to happen there is that the developers might make the mistake of shipping that JSON with the image or vice versa, right? They ship one without the other.
Jon (30:53.84)
Yeah.
Jon (31:00.027)
Yeah, it needs a minute. Yeah.
Jon (31:13.292)
Mm-hmm.
Allan Ritchie (31:15.278)
And then somewhere in there, the user swiped away their phone or LTE died or who knows, right? And all of a sudden that request that was to save that log just went or it crashed.
Jon (31:25.226)
Yeah. Well, and if you didn't, then if you didn't cash it somewhere that you can retry from, like it's just lost, right?
Allan Ritchie (31:31.89)
Hopefully, hopefully you try catching you were able to display a pop up saying, yeah, you know, we think internet diet or something. Right? Otherwise, you know, that thing is just gone in the wind. Maybe your app crashed. If you hit the pop up, you know, and you said, okay, and you hit submit again, okay, you might get lucky. You might. And if the user then the user has to come in and do it again, whatever, if the app totally went to sleep and silence itself, and they had backgrounded the error.
Jon (31:38.847)
Yeah, yeah.
Allan Ritchie (32:01.174)
it's just going to restart the app because it's been asleep so long and all that goes. So it's dead. The data is gone. And in some cases, if you haven't done it right, they may not even know. Right. That that's the scary part about it. So I've had it. So now you, it's a shiny thing. It's, there's a shiny HTTP transfers. It's, it just uses some background services and stuff to make sure that photo goes into a queue, maybe some Jason data.
Jon (32:03.735)
Yeah.
Allan Ritchie (32:30.848)
push or pull. So we're uploading or downloading, right? And it's going to say, as soon as you got internet, start pushing. If there's a hiccup or something that's not server related, it's just going to try it again or resume it later. Right? So people don't realize they need that. And so really underused library. I use it a lot because I know I need it, but this happens so often. I don't even think people realize that this is, it's a bad problem, right? And hopefully they're not losing.
Jon (32:33.643)
Mm-hmm.
Jon (32:53.196)
Yeah.
Jon (32:59.494)
I've been avoiding doing it for my stuff. So right now, I basically, like I said, if you're saving something, luckily for me, the record size is tiny. So it doesn't, no, exactly.
Allan Ritchie (33:11.818)
You don't have to worry at 8K, right? It's gonna fail right there. You can at least put a loading up in their face. But if it's not saving within like two or three seconds, and you don't specifically say user, please keep the phone open, which is a little bit lame. That's a bit like.
Jon (33:18.347)
Yep.
Jon (33:23.519)
Yeah.
Jon (33:28.17)
No, I, and I made the most like annoying message. So like I do the loading thing, right? I have a thing that that's blocking the app from interacting until something happens, either it fails or it goes through. And then if it fails, I like my probably the worst thing I've done in building an app is I have the most generic message that's supposed to be like kind of silly. Right? Like, uh, what, what does it say? Cause I think it's like.
Your phone seems to have gone swimming or something. Like it's something really just terrible, right? Like check your connection and try again, like clear the water out of your phone. Yeah. But at least like it's then it's okay. I know it didn't work, right? So I can at least either try again or not be like thinking, oh, it's great. I'm good. And nothing happened.
Allan Ritchie (33:57.756)
Nice.
Allan Ritchie (34:04.431)
Nice, nice. Oh, that's not bad.
Allan Ritchie (34:18.578)
But see that that'll still like it's 8K at the very 8K is huge actually. That's like a still a word doc, right? Like a page.
Jon (34:22.227)
Yeah.
Jon (34:26.342)
Oh yeah, like that's bigger than my most records would be, like, you know, one or two kilobytes, maybe. Because it's, well, it's not even Json. Well, is it? Yeah, I send up JSON. Uh... Yeah.
Allan Ritchie (34:35.678)
So it's going to clear quick. It's going to clear fast enough that you can put that thing up there. Users generally know, let's give it a few seconds, right? But if you're uploading 15 megs, it's not going to be a few seconds. It's going to be more like a minute. They're not going to wait. They're going to think your app took a dump or you know, something, something's not working. And believe it or not, sending a progress on an upload or download that there's nothing really out of the box, right? You have to kind of plan for that.
Jon (34:48.231)
No. Yeah.
Yep.
Jon (35:04.365)
Mm-hmm.
Allan Ritchie (35:04.834)
So they might not want to wait. So just put it in a background cue and let it go.
Jon (35:10.262)
So what other, like we've talked about rolling your own and I know that's, I think both of us are on the same page there. That's generally the preferred mechanism. Have you looked at other kind of abstractions or implementations lately?
Allan Ritchie (35:27.2)
There's one that's been floating around for years. I think it kind of came and then it died and then it came back with Azure mobile apps. Now, again, it's a good attempt.
Jon (35:34.029)
Mm-hmm.
Hey, that was around when I was choosing like between realm and stuff. And I did try it out then. Um, you know, I don't like full disclosure, right? This is just my own personal stuff I was using it with, right? Um, I had some issues with it and, and maybe it was, maybe it was me not understanding how to properly implement some of the, um, conflict resolution stuff, but like I got to points where I had like local data loss in the, and trying to sync things. And at some point I'm like, this has been a lot of work. I don't.
want to keep down this path. I'm not being successful with it. You know, and then I think I saw realm and that looked like, Oh, it's going to solve all my problems. Um, right. So I, I kind of abandoned that. I haven't looked at it in recent years. I know it's gotten, you know, a few, this was like maybe not even fair to call the same thing. Cause I think it was like the Azure, like the, the mobile tables service or there was like a related Azure service.
Allan Ritchie (36:15.346)
Lo and behold.
Jon (36:32.758)
paired to this at one point, the thing that I used anyway. So I don't know if this is exactly the same or not, or if this is like the evolution of that.
Allan Ritchie (36:39.51)
Well, Azure Mobile Services does require that you, I'm pretty sure it uses OData under the hood. It does have, like you had to do your controllers a certain way. It did allow you to do some filtering and stuff, so that was good. It did allow you to manage batches and stuff, like some good attempts.
Jon (36:45.559)
Okay.
Allan Ritchie (36:58.342)
But again, it took over everything. Like you had to give it authentication control because it was gonna, you know, what if it had to renew data? Which is another thing. What if you have to renew your access token while you're getting ready to sync and, like it goes on and on and on, right? So this thing really, it kind of took over that entire layer, like front to back, how data was queried, how it moved between the devices. And you were like, you had no say in it anymore. You could call push and you could call pull.
Jon (37:02.67)
Mm-hmm.
Jon (37:26.336)
Yeah.
Allan Ritchie (37:28.55)
That was about it and have fun. Hopefully it works.
Jon (37:33.822)
Yeah. And then even like, as I said, understanding like how the, the conflict resolution stuff where it, cause like maybe the default out of the box was kind of like first, you know, first one wins or something like that. Um, but, but that wasn't, I forget when I was looking at it, I thought, no, the default thing isn't going to work for me. Cause I have, you know, this whatever case and then going into that, you know, down, down that rabbit hole of trying to understand how they're
Allan Ritchie (37:43.73)
Yep.
Jon (37:57.37)
Their implementation worked and how to actually use it and do what I want, make it do what I want to do. Wasn't that easy either, right? Like it required work. I'm not saying they did a bad job of it. It's just, it's another concept to learn of another product.
Allan Ritchie (38:12.758)
Right. It's also hard for them to account for all the possible variables. So this thing doesn't work in the background, right? So it's you had to call push and pull, but
Jon (38:16.929)
Yeah.
Jon (38:21.362)
Right. Which you could then do in the background, right? Like you could set up a background job and do that all yourself.
Allan Ritchie (38:25.614)
You could, you could, you can do, you can do a worker process from Android. You can do a BG task in iOS. You can use shiny jobs. Doesn't matter. Right. It's it's, you still had to do that code fine. But the problem is, is you stopped losing concept of what had synced and what hadn't. So the other problems that tend to happen is let's say I pull from a Delta, but my app hasn't run in two days. So all of a sudden there's this wad of data.
Jon (38:30.55)
Mm-hmm.
Jon (38:50.135)
Mm-hmm.
Allan Ritchie (38:53.198)
So generally people say, well, give me everything since that. All right, here's your 2000 records. Enjoy. Right? You don't know like your phone can say, okay, give me all 2000, try and save it. It backgrounds, it fails to store. What's going to happen the next time it sinks?
Jon (39:10.046)
Yeah. Who knows.
Allan Ritchie (39:10.806)
So I never saved it. So my last successful sync was two days ago. Well, I don't realize it failed. So now it's three days. All right, here's your 30,000 records. Enjoy, right? So it's thing, again, you had to start working on streaming data. And so it didn't become just a delta. It became a where am I in the motion of data, right? And...
Azure didn't do that and a lot of people didn't realize that it couldn't do that. I don't know if it does now. I did see it not do it then. Same with push right now. Push is a little bit easier, but people would be like, okay, well, I'm pushing data, I should know if it fails. Okay, great. What happens if you push a batch and the server stupid and doesn't do it in a transaction, which yes, I have seen that. Right. So two passed eight failed.
So which two passed, which eight failed? Right? Do I know? I don't know. You could assume probably all of them failed. But then the next time I got, I got to keep sending these batches. Right? So it's just the problems go on and on and on. And that's why I tell people just stop using frameworks and, and really then that way you can control your own destiny as you go. Right?
Jon (40:12.706)
Right. Yeah.
Jon (40:29.258)
Yeah, yeah, exactly. Um, and, and then that you have the flexibility to have like one of the other things that I haven't had run up against. Um, you know, one was like, yeah, if I was using realm, it was, I have to use their library in my app and then, then you start, you know, this is me again. And then like you said, origin story, maybe this is where it comes from. It's like, oh, well what happens when I, you know, when they're not updating their thing to work with the newer version of something that I need to, to work with, like I think.
Back in those days, there was something a few times that they're like, oh, there's a new version of Xamarin or something. And I was waiting on them to update their stuff to work with it, because there's some change, right? So you run into those situations where you're kind of not able to choose your own destiny in terms of what's running on the device and on mobile. But even then with my own solution, I started down the path and for years I was using light DB.
in the app for the local offline storage. And that worked pretty well. I was quite a big fan of it actually. And the reason I chose it mostly was because my objects aren't complicated, but there is a little bit of nesting in them. And because I'm using Cosmos, like Cosmos is a more document DB style database, right? So like when I, yeah, so when I stored like a log,
Allan Ritchie (41:49.658)
You exchange your documents. Yeah.
Jon (41:53.546)
Like I said, sometimes I'll try and capture the local weather conditions and I attached that to the log. And rather than doing that relationally, like having a, you know, weather record in another table that was related back to the log table. I'm just like, not like I'm only using this information together. So like just put it all in one document and then I pull it back out. Um, so light DB made that easy because it was more document oriented to, um,
Allan Ritchie (42:11.075)
Right here. Yeah.
Jon (42:21.578)
But eventually when I started doing the switch to Maui, there were some, there's differences in the runtime and there were some issues earlier on with like AOT compilation of stuff that I was using with LightDB. And then I had to do all this work to make it skip over AOT and you know, certain assemblies. And then I lost some startup performance and stuff as it, like it just, it kind of spiraled. And then finally like, okay.
At least because I built my own solution here, I can move over to something else. And I did, I moved over to SQLite and I'm reasonably successful and happy there right now. So like that's another part of it too, right? If you control the pieces, when you need to shift them around, you're not prevented from doing so. It's kind of nice, it's free.
Allan Ritchie (43:10.458)
And sequel light's not going anywhere, right? It's been, it's probably the most tried and tested, like tried and true all the way through. So.
Jon (43:12.755)
No, for sure.
Jon (43:18.186)
Yep. Um, I, I avoided it at first kind of, because like, I, anytime you bring like a native thing into your app, there's, I'm going to say there's problems, obviously it works, but there's the, the potential for it complications and like builds and linking and stuff. Although SQLite's been such a, like you said, it's such a mainstay that kind of stuff is all figured out anyway. So it really isn't a problem. Um, but I, part of, and the other part of it was like avoiding
One, I didn't have, I didn't like, I used to be a DBA. So this is also what hurt me too. So I know how to write SQL, but like, I don't want to write SQL. Like I'm not a DBA anymore for a reason. It's not what I want to be doing. So I'm like, I liked LightDB because I didn't have to do any of that. And so SQL, and I know there's ORMs and stuff. And I did, I did actually start trying to use Frank's, you know, very popular SQL net light.
Allan Ritchie (43:53.327)
Going back, yeah, going back in time.
Allan Ritchie (44:01.298)
Okay.
Jon (44:17.783)
whatever thing's called.
Allan Ritchie (44:17.938)
Did you... you avoided that? I didn't even know you went straight sequel. I... oh yeah man, I would have been making fun of you then. You never told me that though. You didn't tell me on purpose. I know you didn't. No, you didn't.
Jon (44:21.602)
Oh, wow. I mean, come on. Who are you talking to?
Jon (44:29.246)
I think I did know. So I started, I started using that, but I think, what was it? There was something in how I had to, like, I ran into something where I would have needed to make my objects derive from an interface or a type that he had. I, there was something. Yeah. I didn't want the attributes. And then I started looking, I'm like, can I change his thing up so that I don't need to do that? I'm like, yeah.
Allan Ritchie (44:47.622)
You had to put attributes on it. You didn't want to put attributes. There we go. OK.
Jon (44:58.058)
could, but then I'm like, well, why don't I just write the sequel myself? Raw.
Allan Ritchie (45:03.155)
I do wish that there was external mapping, right? So the objects were a little bit left alone. I'm not a fan of attributes too. It triggers my OCD, I guess, in code, but it's.
Jon (45:07.322)
Yeah, that was...
Jon (45:14.762)
That was the thing because I share that set of objects with like my server stuff too, right? So I'm like, I don't really want this leaking into that part of it. And like, I don't know. I just, I didn't want to do it.
Allan Ritchie (45:24.666)
The purest EDDs are coming after you, John. They're coming with pitchforks and fire.
Jon (45:28.862)
That's fine. So I ended up, um, I actually, actually I went super low level. Uh, I'm just trying to scan through here and maybe one, one day I'll figure out screen sharing so I can show this kind of stuff. Where, where did, yeah, I, I wrote. I wrote my own stuff for like doing the super raw API, like SQL light net, like, oh no, did I, maybe I did go, okay. I didn't go that bad.
Allan Ritchie (45:37.116)
I don't know.
Allan Ritchie (45:54.611)
Oh
Jon (45:55.142)
I went SQLite connection. I went with the mono, the, the Microsoft data SQLite API. Okay. I start, I st I thought, because I started trying to just use like the really ugly SQLite net or SQLite API. Yeah. Like the statement prep and all that kind of stuff, right?
Allan Ritchie (45:59.234)
Yeah, okay. Thank goodness you used at least an ADO.net provider.
Allan Ritchie (46:10.106)
Well, that's yeah, the low level stuff that's actual like no data readers. Yeah. I don't think people realize how much, uh, ADO.net really actually did for you and ADO in general, but.
Jon (46:21.526)
Yeah.
Jon (46:24.85)
And it was kind of, it kind of felt familiar back to my days of writing like, you know, ASP, um,
Allan Ritchie (46:29.242)
You did not write it like that. Come on. You did not.
Jon (46:32.642)
the sequel. Yeah, for some stuff. I made websites back in the day like that. I eventually used ORMs, but you know, early on I did like the ADODB, the readers and all that fun stuff.
Allan Ritchie (46:45.746)
Thank goodness we've evolved. Thank goodness.
Jon (46:48.31)
So I did arrive eventually at SQLite and the thing that kind of saved me, like I said, because I wanted to do documents was the JSON support with SQLite, which is pretty sweet. I don't know if you've used stuff there. It's nice, yeah.
Allan Ritchie (46:59.982)
Oh, I do remember that now. Oh, I can't pretend like you hit it for me then. Cause I do remember he had that conversation. And then.
Jon (47:07.37)
Yeah, so like you can create a table that has, um, like it's, it's just the JSON column, right? So like all my object is stored there, but then you create these sort of like meta or pseudo or calculated columns, I guess. And so I'll have like the few that I care about on the log document, right? Like, like I said, everyone has a timestamp, uh, the pool that it's associated with. Uh, and then like the type of log that it is. So I have.
these calculated columns that are like, oh, pool ID text as, you know, JSON extract from the JSON value column. Then you give it like the path, like the JSON path specifier, right? Like dollar sign dot pool ID, which is like the path where that value is stored. And it's great, it works perfectly, love it.
Allan Ritchie (47:57.582)
That, okay, that makes sense. I do remember that conversation now.
Jon (48:01.706)
And you can create indexes on those calculated columns too, which is, was wonderful. So I have all that SQL just written out, you know, lovely in a file.
Allan Ritchie (48:09.355)
It's like you just have to go against the grain, don't you? You just had to.
Jon (48:18.571)
Yeah. mostly in like startup times and stuff too.
Allan Ritchie (48:21.562)
Which that's weird because see light DB was apparently known for being like massively, massively huge, like, like fast. Like it was, it was known for being a document DB that was mega fast.
Jon (48:28.17)
Like, you... Oh, okay.
Jon (48:35.663)
It was really the initialization. So one, because there was some bug in the runtime where it didn't know how to AOT what I was doing. So that was that thing, and that made me make it just be like JIT, or like interpreted mode for that assembly on iOS, right? And on Android, of course, but.
Allan Ritchie (48:43.867)
Yeah.
Allan Ritchie (48:57.838)
And that triggered you. You weren't pleased with that. But it does slow.
Jon (48:59.798)
Well, no, but that did actually show up in like speed scope logs. So like what I was we were trying to like look at app startup time with like, you know, some real world apps. I'm like, wow, then my app is a real world app because it's in the world and it's real. And so we did some scanning of that and like light DB continually kept showing up as like the biggest bottleneck in startup time for me. So hopefully that's fixed.
Allan Ritchie (49:21.07)
interesting. Well, if you're happy with your SQL like JSON stuff anyways, what difference and you wrote SQL again. So it brought you back to your root.
Jon (49:27.702)
Yeah, exactly.
I wrote SQL again. It did kind of like, you know, again, it's like all of these things that, um, we do that were like, is it Stockholm syndrome or is it, you know, do I really like it? Like same with MSBuild, right? It's like, oh, it was kind of fun.
Allan Ritchie (49:45.91)
I'll use a lot of NED Framework myself on the server. I have no problems there. I can identify where things are gonna go wrong, and I'll still go back to some hardcore SQL because all the kids these days are like, we'll just use NED Framework, and they've got their ways of doing it. They wanna pull out JSON, and I'm like, no, here's how you do cross-joins and some clever sub-selects and stuff, and they're like, I don't like the SQL stuff, and I'm like, why not?
Jon (49:48.525)
Mm-hmm.
Jon (50:05.579)
Yeah.
Jon (50:11.44)
Yeah, it's hard.
Allan Ritchie (50:12.306)
This is how you do it, man. Unless you want, you know, your queries to take six hours. This is how you do it in six seconds. Like slow.
Jon (50:18.302)
Yeah, well, and you're not using entity framework on your mobile app, right? So.
Allan Ritchie (50:23.35)
No, exactly. But, and you can still do that complicated SQL if you have to, which I have. So that's why I just, I'm good with SQL. All even today.
Jon (50:29.592)
Mm-hmm.
Jon (50:33.75)
Yeah. No, it wasn't bad.
Allan Ritchie (50:37.294)
Now, have you ever had to do other offline stuff? You know what? We should probably save that for another episode. So that's another episode, but.
Jon (50:45.702)
Now I'm curious though.
Allan Ritchie (50:47.354)
Oh, well like doing stuff where you don't have internet. How do you work with other devices? Oh.
Jon (50:53.786)
Oh, like, oh, yeah, interesting. Like device to device communication. I, I did a long time ago. I had a, I think I had even did a talk on it at one of the evolve conferences. There was, um, Google had this thing, like, uh, an SDK for like nearby communication. I know they still do, and they have like different stuff around it, but this was like early days of it.
Allan Ritchie (50:59.995)
Great.
Allan Ritchie (51:18.299)
Yep.
Jon (51:20.911)
And I did something with that and that was kind of cool where you could like have the app running on two devices And it would talk to each other some way. I don't know that that's my only real experience
Allan Ritchie (51:27.95)
Yeah, like a peer to peer. I've had to do one. I did a POC on it. We didn't actually get this to production, but there was a case for two or three companies, something like that, where they had, there was no internet. They were up in the air, or they were at some site where it was way off in the boonies, where cell towers were non-existent.
Jon (51:48.17)
So like not even an access point that they could like, cause I mean, if that's the classic thing, right? If you have no internet connection, maybe you still have like a local wifi hub.
Allan Ritchie (52:00.242)
If we go back to when the timeframe when this was there, like Wi-Fi is on every plane now practically, right? Cause they've got the satellites to deal with it. Like that infrastructure wasn't there. If you go back even as far as five years ago.
Jon (52:07.815)
Yeah.
Jon (52:11.742)
And I guess like to. Yeah, true, and you couldn't just, you know, obviously for many regulation reasons, I'm sure just be like, well, I'll just throw a wipe an access point in the plane.
Allan Ritchie (52:21.79)
Oh God, that is such nonsense. If it takes Wi-Fi to interrupt plane transmissions, we got problems, man. That's like the oldest urban myth that they try and do on flights. Please turn off your Bluetooth headset. I'm sure they don't want any interference and I'm sure it does cause us a little bit. But if it really causes that much, I really don't want to fly on a plane. I really don't. It's nonsense that it blocks it. Operate on a different frequency, man.
Jon (52:29.814)
I agree. Yeah, airplane mode and all that.
Jon (52:44.226)
Yeah.
Jon (52:50.142)
Yeah, exactly. But still, you wouldn't have been able to actually do that for those reasons, I would assume.
Allan Ritchie (52:56.59)
Right. So they had like these pay terminals where they had to sync data to make sure like this customer purchased that in that seat, etc. And they didn't have a Wi-Fi terminal. Again, this was more than five years ago now. But we had to do something like a peer to peer network. See they, but this became my POC. And they never implemented it. But they wanted credit card data.
Jon (53:17.715)
So, but why?
What?
Allan Ritchie (53:23.642)
Right? So if I on terminal a, I made a purchase and then the customer goes, well, the sandwich tasted like junk, which let's face it, that's most airplanes. You're getting a sandwich that's cooked. It was, it was nuked five hours ago, man. Like you're eating, you're eating.
Jon (53:24.415)
Yeah.
Jon (53:32.574)
Yeah. Like you knew what you were buying. Come on. Yeah. Right. It's like, it tastes terrible. I want to refund. All I have left is one bite.
Allan Ritchie (53:42.523)
Yeah, which is it. It happened, man. It was like a prime use case. Like it sounds funny, but it was a prime use
Jon (53:45.641)
Yeah.
Jon (53:49.886)
So, so is that, I was just gonna ask like, well, why, why did you have to, like, why, if it was offline anyway, like it's presumably storing up the credit card info so that it can sync up later on the ground. But so, so you needed to communicate because there might be several terminals around the plane being used and no matter where they wanted to like get a refund from, you needed any of them to work?
Allan Ritchie (54:00.366)
right so it would do that but it
Allan Ritchie (54:11.394)
Exactly, because they'd have like on the big airliners, right, the ones with multi rows, right? So if you had two rows, they might not know the cart that they had down there, because you'd think they were a bit more organized, but they really weren't, right? So what had to happen is these terminals, they were always on, thankfully, is they'd be looking for each other, right? And just start syncing data through BLE, because again, Android, they'd have Android for the cheap ones.
Jon (54:16.695)
Yeah.
Jon (54:23.948)
Yeah.
Allan Ritchie (54:39.758)
And then some of them would have tablets, right? So you might even have one that gave it to a customer and said, here, order on this tablet. I can't I tell you, no, you got to order on there and then use your credit card for crying out loud. Fine. Right? So you would do this in the terminal, of course, how to tap and it would put that in, but again, that data, including credit card would, would fly over. Now it was encrypted, but I mean, how encrypted could it be encrypted enough? Let's put it that way. Nobody's scanning Bluetooth up there.
Jon (54:46.082)
Right.
Jon (54:56.215)
Mm-hmm.
Jon (55:00.54)
Yeah.
Jon (55:09.167)
Yeah, I mean, and if you are like, realistically, how are you going to know how to decipher that if you did any reasonable amount of encryption?
Allan Ritchie (55:18.894)
And I mean, it's the funny thing about Bluetooth up there is it kind of, I don't know, the way it would connect with sometimes even though you were side by side, it would take time. It's weird. It's the signals aren't working the same way that they do on the ground. Let's put it that way. So these devices would get beside each other and go, hey, I know you. I know you too. Here's my stuff. Give me your stuff. And they would trade. And so when somebody gave back one of those disgusting sandwiches, or the one bite, they'd be like, yeah, I can refund you.
Jon (55:28.523)
Yeah.
Allan Ritchie (55:48.938)
if they had been in range. So that's why it never fully took off to production because some people would be like, I paid for it. Then they give you the product and you'd be like, the hell is this? Did somebody eat this once already? I don't want this.
Jon (56:00.781)
So.
Jon (56:04.446)
You know, I'm just thinking now, I mean, we've missed this opportunity because now everything is wifi and I would imagine that most of these terminals are actually connecting now to the ground. Um, but does that mean I could have like brought along with me a card that I had like already had deactivated and just use that.
Allan Ritchie (56:22.749)
There was also the potential for that. Um, you wouldn't, right? So.
Jon (56:25.226)
Yep. Because how would you know?
Other than like the name, I guess they could try and go after you. But like for the amount of money, it probably would be, I would they do that? I don't know.
Allan Ritchie (56:38.428)
Yeah, you're figuring out these were the problems, right? So and it did happen. There was lots of. How would you know? Right? You wouldn't.
Jon (56:42.695)
Yeah.
Jon (56:47.174)
Oh, I mean, that reminds me, I know we're running up on the clock here, but there used to be when I would fly for Xamarin days for travel. I don't, I don't know if it was just one airline or multiple. There was a way that you could, so like you'd get the wifi right. And then you'd have to pay to like get internet access, but the way that they had it set up, it was all like the captive portal and like you're connected to the internet, but you're not through their paywall, but you still need to be able to get, you know,
to their paywall to pay to get there. So anyway, if you set up like a proxy server and you had it running on the right ports that they allowed some ports through always, you could do like the whole reverse tunnel. So I always had free wifi on the planes because I was reverse tunneling home into my machine that was sitting here waiting for it.
Allan Ritchie (57:32.469)
Of course you did.
Allan Ritchie (57:37.802)
It's the networking hardware has gotten a little bit more sophisticated since thankfully, but yes, and it wasn't really, it still sucks up in the air though. Wi-Fi still sucks. Imagine thinking data. If you're thinking data up there, you want to have a good background experience. Don't do something crazy with frameworks because if you're in the air, let me tell you, it's going to suck.
Jon (57:43.199)
Yeah. That was always fun.
Jon (57:48.578)
It's not that bad.
Jon (57:58.498)
Yeah.
Allan Ritchie (58:02.434)
Even that even they have that the expensive internet option now up in the air. I fly enough still that I see it and you're like, yeah, it's cheaper now. It's like 10 bucks for a three hour flight. But you're like you can have the $25 version and watch Netflix. I'm like, but I can watch Netflix on the plane. Anyhow, you know, that 25 bucks doesn't get you that much more. It still sucks. You're like, oh, great, I can watch it in, you know, 320p now. Great.
Jon (58:09.739)
Yeah.
Yeah, it's not that bad.
Jon (58:24.994)
Yeah, it's...
Jon (58:28.67)
Right, and then like, you know, every once in a while it's going to cut out because, you know, the signal cuts out and all that fun stuff. It's a good, it's a good environment to test your, your offline stuff on.
Allan Ritchie (58:34.275)
Yeah.
Allan Ritchie (58:39.138)
I mean, you could just test on the emulator and say, this is, oh, well, I guess if you wanna leave your, if you don't wanna leave your house. But I wanna leave my house, I do like going out. I wanna fly in a plane. All right, John, I can see you're flipping the library of the week, which I changed on us because I thought this was more appropriate.
Jon (58:42.246)
Wow, what fun is that?
Jon (58:50.45)
Exactly. All right.
Jon (58:58.602)
Yeah, no, I like it. So what did you pick?
Allan Ritchie (59:01.258)
I picked SQLite-PCL-NET. I always get the two mixed. Is it.NET-PCL? I think it's PCL-NET.
Jon (59:05.058)
Sure, I think that's it.
Jon (59:11.882)
SQLite net, I think it's SQLite net PCL. Yeah, yeah. I mean, if you type in like SQLite and Maui or Xamarin or whatever, it's gonna come up because that's like the de facto thing that people use, I think, right?
Allan Ritchie (59:15.451)
If you type them in some order, it'll come up. Right?
Allan Ritchie (59:27.694)
Well, it's got the things that I like about it and why I mean, I'll still use the Microsoft version because it's ADO.net if you need it. But you know, there's the, oh, you're, you want to even talk about the wall.
Jon (59:35.733)
Yeah.
Jon (59:39.93)
I want to throw an honorable mention in there too, right? Because SQLiteNet is cool, is great, but all of these, and also the Microsoft data, what is the thing called? I just said it before. The one, well, the Microsoft data SQLite, it also sits on top of the SQLite raw or whatever, the green, I don't know the right name to call it, but that's Eric, whose last name, yes, thank you.
Allan Ritchie (59:45.001)
It's built.
Allan Ritchie (59:54.43)
the raw green. Oh sorry Microsoft. Yep.
Allan Ritchie (01:00:07.426)
Eric Stink. He is the godfather of SQL 8 in.net.
Jon (01:00:11.358)
Right. And all of this, you know, is possible because that thing has been built and is maintained and updated too. And I know like that, you know, early on Maui, you know, done at six days, I think it needed some work from him to get things back up and going to. And that obviously happened, which is great. But yeah, that's a that's a big foundational piece of a lot of this, too. So.
Allan Ritchie (01:00:32.834)
Actually, it's good you called that one out because he really is the godfather as much as Frank is the credit. Frank did some nice stuff. But like this thing sits everywhere. If you have SQLite on your device, you need to you need to give the godfather his due.
Jon (01:00:37.15)
Yeah, yeah, Frank put a nice layer on top, right? But.
Jon (01:00:47.306)
Yeah, like that's one that like if we're talking open source like everybody owes him a beer for sure Yeah So that
Allan Ritchie (01:00:53.538)
at least. But Frank's library does have a couple nice things on it besides being an ORM. It doesn't do parent child stuff. That's fine. It doesn't need to. If you're doing parent child, you don't really... whatever. But it does do stuff like table updates, which is a big one. If you've got a model and you say create me the table, if you change your model and you create table again, it'll update the database.
Jon (01:01:00.598)
Yep.
Jon (01:01:05.005)
Mm-hmm.
Jon (01:01:16.929)
Yeah.
Jon (01:01:22.85)
That's a whole other part of all of that, right?
Allan Ritchie (01:01:24.362)
It sounds like, oh, well, that's not a big deal. All the kids are going, yeah, we could do that in any framework. Kenya, yeah, migration, don't get to start on migration. But.
Jon (01:01:28.734)
Migrations, yeah.
Yeah, I've been fortunate that for my app, I just, if I do change that, because again, I'm only using mine really as like a local cache, right? Like anytime, if I create a record, it goes, first it goes to the server and only after it goes to the server and it's good there, does it come back from that request and then get saved locally, right? So I'm never writing to the database locally before I know it's in the server. So it's all read only basically.
Allan Ritchie (01:01:56.079)
until
Jon (01:02:01.098)
And so if I change stuff, I'm just like, eh, just blow it away and create it again. Sync the data.
Allan Ritchie (01:02:05.999)
Well, and I mean, he has got the ability, so it's going to do the create columns, the drop columns. It does that all seamlessly, which means you don't have to track the version of the database. Just say here's my model now. I don't care what it was. Here it is now. It'll drop the column, create the new ones. It's just...
Jon (01:02:10.796)
Yeah.
Jon (01:02:23.41)
But also, and that is lovely that it does that, but I think the whole migration thing is another good reason to consider using the JSON stuff in SQLite even too, right? Because you can, no, but really, you store a document, you can add whatever you want to it, and you don't have to necessarily change your table.
Allan Ritchie (01:02:42.706)
This is true. Turn it into a SQLite becomes document database. I think the only thing we found that SQLite didn't do, and this was something you and I were working on for the longest time, was that the good, the true geocalculations, that it has an extension, but getting that extension over was.
Jon (01:02:45.322)
Yeah, that's what it is. That's how I'm using it.
Jon (01:02:52.811)
Mm, yeah.
Jon (01:02:57.918)
Yeah, so get in.
Jon (01:03:03.91)
Was not easy because it's not because eric didn't put it in the box, you know, which he already put a bunch dealt with a bunch of them, so
Allan Ritchie (01:03:10.362)
Well, SQLite, the raw, the lowest level, the C SQLite has some of that stuff in the box, but it doesn't have the extension for the real hardcore stuff. Now there's other companies out there, which we won't go into, that do maps and stuff, and they have their own thing on top that does geo.
Jon (01:03:17.693)
Okay.
Jon (01:03:25.966)
Mm-hmm.
Jon (01:03:30.454)
Yeah, because we were trying to like search by geo, right?
Allan Ritchie (01:03:34.23)
Yeah, we were doing some crazy stuff, right? And it just, I think we got it working on one platform and then the other, I was like, I give up. I give up for now, we'll come back to this. But it, you know, it was awesome. SQLite can bolt these things on if you need to. It just, it wasn't out of the box. We had to do that work.
Jon (01:03:43.435)
Yeah.
Jon (01:03:53.91)
Yeah. All right, well, I think, you know, that was a really cathartic episode for me. So thanks everyone for listening to our troubles and rants and raves and all of the wounds that we have. It is Friday. We're recording actually today on Friday. So yeah, I think that's about that time of day.
Allan Ritchie (01:04:09.121)
Do you need a beer after that one? You look like you're.
Allan Ritchie (01:04:17.285)
reliving a nightmare though.
Jon (01:04:18.918)
Yeah, so, you know, while we go and fetch a beer for our Fridays, it would be great if you're listening and it was also Friday for you when you hear this probably that you would go and use that Friday afternoon and evening to leave us a review on the Apple Podcasts app. That'd be great. Appreciate it always. And if you have... I mean, just like really corny segues. That's fine. And if you have feedback, comments, questions, whatever.
Allan Ritchie (01:04:37.384)
Dad jokes this never end Sheesh
Allan Ritchie (01:04:42.82)
That's good. That's good. It's good.
Jon (01:04:49.306)
If you want to complain about the jokes or if you have even worse jokes gone mobile that io drop us a line and We'll probably see if we can get those bad jokes on a future episode. See I'm expanding now I'm asking not for just questions and comments, but also jokes
Allan Ritchie (01:05:06.906)
I don't know. I don't know. That's opening. That's opening the floodgates, but we'll see. Who knows? That's true. That's true.
Jon (01:05:08.69)
Yeah, that's probably bad. All right, well, we get to moderate them anyway, coming in, right, so. All right, Alan, have a good weekend. Thank you all for listening. See you next time.
Allan Ritchie (01:05:17.146)
You too. See you, everybody.