文字起こし 🔗
Jethro: Welcome to Transformative Principal, where I help you stop putting out fires and start leading. I am your host, Jethro Jones. You can follow me on Twitter at JethroJones.
Jethro: Welcome to Transformative Principal. Today, I am very excited to have Bella Davis on the program. She's the founder and CEO of Monarch AI Technologies, which is a privacy first safety platform that helps K12 students intervene before harm occurs. And, uh, she has done tech work for a long time in the United States and Germany.
And, uh, what this Monarch system does is it analyzes patterns across text, images, and audio and video to identify early signs of bullying, grooming, self-harm, threats of violence, and mental health concerns. And the key here is that trained staff review flagged signals before action is taken, which means that human judgment is going to be at the center.
So this is, uh, something that we definitely want to, uh, to think about and talk about as it relates to kids being safe in school, uh, from, uh, peers, but also from adults who are acting inappropriately, and especially if they're using school provided devices. Uh, you know, we're gonna get into all that. So Bella, welcome to Transformative Principal.
Great to have you here.
Bella: Thank you. Thank you for having me
Jethro: So, uh, tell me about, uh, where this idea came from and why you chose to focus on schools, uh, rather than, you know, anything else you could be doing in tech
Bella: Yes. Um, well, th- I always wanted to create my own company. It was always there. And I always thought that if I create my own startup or my own company, I wanted to create something that will have an impact and that will actually do something good. somebody in my family did experience, as a child, uh, sexual assault, and that's something that has been very heavy, um, and carried, um, to this day because the person has to deal with mental health problems. And so I'm, I'm very, very touched by everything that touch children, even though I don't have any. And one day I was sitting on my couch in Berlin, and I was watching Zuckerberg talking to Congress and with all the parents, uh, in the back with a photo of, photos of their children. I was very shocked, um, to see the response and the lack of, let's say, compassion. crazy enough, I decided at this moment to do something about it. So Monarch initially started as a parental control, and then last year, exactly a year ago, we did the switch and pivoted towards, uh, building a pattern recognition for schools. So yes, so it, it came about by having a love of children and for the children that I don't have yet. So I'm building this also for them.
Jethro: Very good. So, uh, something that's unique about your company is that you are, uh, solo bootstrapped, and you're not doing this with a bunch of VC money. Uh, so you can, I think, be more privacy focused in that regard. Talk to me about the privacy focus of, of your company and, and how, how you, how you stay privacy focused when you're reading everybody's messages and watching their videos and all that kind of stuff.
How does that work?
Bella: It's all about how you process the information, right? So initially, when I started the company, I still lived in Berlin, and in Europe there is very, very, very strong regulation around AI. So initially, when I started the company, I started building it with the focus of actually building it for the world, in Europe, for Europe and in the US. So when I finally moved back to the US, I kept this mentality with me, um, by building under, let's say, European standards instead of the American standard where there is not too many regulations around AI here yet. So how we process the information, we actually talk to all the principals that we are currently working with, and from what I've understand is that the school have a responsibility to make sure that the kid is safe when using school devices
Jethro: Yep.
Bella: school ground.
So that's number one. All the information belongs to the school or the district, and we do not save anything at all. So that means that we will not, um... All the information is anonymized. So Bella, for example, I will never know that her name is Bella. I will never know that she's 15. That means that this information that is collected stays on the school, on the school side, and we do not see it because it's simply anonymized with, uh, let's say, a serial number or a number that is assigned to the devices on itself
Jethro: So let me make sure I understand that correctly. You, on the company side, see reports and you see a serial number, not a student name. If somebody puts a student name in the, in the text message, uh, is that also anonymized?
Bella: Uh, on the text mess- it-- Well, it-- To, to clarify, we are not looking at text messages, we are looking at texts.
Jethro: Yes. Thank you
Bella: goes on Google and type, um, the dosage of X medication to, let's say, harm themselves, um, we will see that, right? And then we'll report it to the principal, and then they can decide if it's relevant. Now, if it's something that has to do for bullying-- with bullying, for example, and there is a name, yes, we are able to analyze that information, but we do not save it. Because like I said, we send all the information directly to the dashboard, and then they can analyze and see if it's relevant or not. But we do not save anything, and after thirty days, whatever we see that might have been, let's say, the name of a child, it will be discarded after thirty days
Jethro: And are the schools deleting it after 30 days also?
Bella: Uh, the, the responsibility on our side is, is to delete the information, but the school is not able to, let's say, remove the information that they see on the dashboard, but of course, they can discard it,
Jethro: Yeah
Bella: Um, but yeah, we, we do not see anything that has to do with PII at all
Jethro: Yeah. Okay. So one of the challenges with this kind of technology is
Load full transcript
that, um, it leads to a lot more reports of things that are happening, and, uh, some of them are serious, some of them are not serious. Um, and the only way to really know is to go investigate. And, uh, when I was a principal, my challenge with this stuff was that I spent all of my time investigating these reports and could not spend any time doing the other things that I needed to do because I can't get a report that somebody is harming themselves or others and do nothing about it.
Like, I have to do something. And my job went from being a school leader to being someone who is just chasing down kids about things that they are writing online. And, uh, and that's not, that's not the job that I signed up for, and that's not a good job to have, and that's not a fun job to do either, and that also does not solve the deeper problem.
So first, how do you help schools manage the workload that comes with this?
Bella: So initially when we started building the pr- the product, we made sure to use, um, AI that was, uh, kind of already trained to identify harmful content, right? And then what we did, also w- are working with university to actually give us their proprietary research to improve the model. So that's something that actually do not exist right now. And third of all, once we send the information to... Because we are still in pilot stage, we are still testing the product. Um, once the school receive the information, then we rely very much on their feedback, right? In order to improve the system and the solution, so then we don't actually spam you with information that are irrelevant. So we really much focus on, for example, if there is a gun violence threat, I think that as a principal you would like to know that.
Jethro: For sure
Bella: much focus on everything that has to do with life-threatening. We, we focus on being proactive, not reactive, because a lot of the tools that maybe you used back then were identifying keywords and sending you all that information.
That means that you're overloaded. What we focus on is detecting patterns that are very serious, and then on your side, and you also, uh, we are re- we actually require to have two people that review the information, and only leaders. So you can have a vice principal or a principal, so then if you are overloaded, the vice principal can take over and also look at all the notification that has been received.
Jethro: Yeah. Well, I mean, this is really one of the challenges that, uh, there needs to be a solution for, of how do we deal with, with all these things that kids are saying? And ultimately, my belief is that the, the way you deal with this is that you go back a step before, and you do good teaching of, of what appropriate behavior is and online safety and all that kind of stuff.
Um, but this also doesn't get rid of the problem, Bella, because a lot of kids will just use their own devices or not use the school devices to do the exact same things that have been happening. And so, uh, some aspect of this being, you know, something that, uh, that kids are, are doing, uh, it's still an issue, and this doesn't, this doesn't solve that issue either
Bella: Yes. It doesn't s- solve the whole issue, the whole problem, but it's a step, right? Because if companies are not doing anything, opposite, all the information, most of the tools that are being used in schools right now, they actually use that information and they repurpose it and sell it. we are doing, we are taking the other approach, and we are trying to find an avenue to provide a solution, right? Um, so obviously I don't know where the product will evolve, um, because like I said, I started as a parental control and now I am building, you know, a pattern detection for the school. So my hope is that it continues to at least send you the information, and then if you can at least save one life or two lives, to me that's already a big step
Jethro: Yeah, absolutely. That is out- totally worth it. And, uh, my concerns about this, this stuff is not to say that it's not important and we shouldn't do it, it's just to say we need to, uh, we need to think about the bigger picture as well. So does Monarch provide any support in w- how to teach those things to kids or anything like that?
Or is it just providing the technology? And it's okay if it just provides the technology, that's fine too
Bella: Yes. As you can see my, by my under eye cyc- circle, I'm very tired. I'm a solo founder. I do a lot. I'm marketing, I am sales, I
Jethro: Yeah
Bella: know, everything, product managers for now. Um, but the goal obviously is to make sure that we are actually teaching as well. Yes. So that has been... I've received so many ideas, but like I said, I'm a one-woman show with a lot of support from strangers and engineers that really believe in the product. Um, but as soon as we are able to, then the next step will be to actually educate parents. And we also have a building on partnership with, uh, children, uh, protection organizations such as the Heat Initiative, for example. That is a big, big organization that very much advocate for the children. Um, and we are trying also to get others on board so then we can do a partnership and they can also spread the word on behalf of Monarch AI and assist the parents any way they can
Jethro: Yeah. Yeah, I think that's really good and, and essential for our, uh, for, for what we're doing in the future. One of my big concerns, uh, several years ago when social media came out, is that I did not believe that schools took a proactive enough role in, in co- uh, basically complaining about how things are going.
And I feel like they're doing the same thing with AI right now. They're trying to do a little bit more, but their, uh, their approach is, is more, um, I think, uh, misguided in that they're focusing on making sure kids don't, quote-unquote, "cheat" without understanding that our world is completely changing, and the way that everybody does business is going
to be much different.
Um, and not just the way people do business business, but the way people interact and do different things is going to change dramatically because of what AI, uh, makes possible. And,
Bella: Uhum
Jethro: and just saying, "You're not allowed to use AI on this assignment," or, "You are allowed to use AI on this assignment," is just totally missing the boat.
So, you know, that's an- that's another thing that, again, schools are not living up to the, the challenge to educate appropriately, and they're missing a huge opportunity. And, and that's something that, uh, you know, I think is, is just really important. And we can't just have access to these tools and then just think that kids are gonna figure it out on their own, because adults can't even figure it out on their own
Bella: Exactly. And pretend that the issue is not there either is not gonna make it go away. So that was also the whole idea, uh, when I came up with the, with this project, with this company, is that it is here to stay, and it has been here actually for a long time. People think that when they scroll on YouTube, for example, this is not AI.
It
Jethro: Yeah
Bella: because it knows what you like, what you are looking for. So it's been here for a long time. Now, how do we use it to actually try to do something good with it and figure out together, um, that's my goal as a company and as a individual, working with principal, working with child protection organization, working with everybody that is concerned with this, and move forward and finding a solution that is adequate.
Jethro: Yeah. And so one of my major concerns also is that kids get exposed to just being in a surveillance state all the time. And, and I have very real concerns about everything that we do, everything that they do being recorded, being reported to someone, and that they're, that they're constantly being watched.
And that's, that's one of my concerns. It's great to focus on privacy, but part of that privacy is not recording everything they do in the first place. How do you deal with that?
Bella: We are actually not recording. So w-
Jethro: Well, come on, Bella. You are
Bella: No, honestly, it is not. It is not a tool that like, see that camera, for example, we see the green button,
Jethro: Mm-hmm.
Bella: it is not there looking at what constantly is happening. So for example, right now, the prototype works with text and audio.
Jethro: Mm-hmm
Bella: we are-- The AI is trained specifically on understanding if something is harmful in
Jethro: Mm-hmm.
Bella: So if we identify, for example, like I said earlier, for example, on a Google Doc that a kid is bullying someone, then at that point we will send the information. But it's not on twenty-four/seven looking at what the kid is writing. First of all, it's very expensive for me.
Jethro: Yeah.
Bella: afford
Jethro: Yeah
Bella: store all of that information. No, we are just looking at what's harmful. We'll send it to the principal, and then we discard the information immediately
Jethro: Yes, but what I'm saying is you are, you have to look at everything to find what is harmful, correct?
Bella: We have to find ways to look at the screen, yes. But it's not like we are recording like TikTok or the other companies. No, we are not doing
Jethro: Yeah. So, so that's the concern that I have is that our, our kids are, are being taught by how we're acting that everything that they write or record or say through a, a voice thing on their school computer, that it's all being surveilled basically. And, and that's what I, that's what I have a concern with.
Help me understand, like I do wanna keep kids safe, but I also want them to have privacy and be able to make choices. And when they know that everything they type into a school computer, something is going to evaluate whether or not that is appropriate or not, that is a surveillance state and, and I have concerns about that
Bella: Yeah. I understand, I understand your concern. Um, from our side, we see it as trying to prevent bad events from occurring. So obviously, each school has a choice, and for example, not all schools have the same stats, right? So some schools have a lot more issues than others. As a principal, you
Jethro: Yeah, for sure
Bella: um, one of the principals that I talked to recently, for example, told me that the parent noticed that the child behavior had changed at home, and she had no idea why the child behavior had changed.
And then when they finally took the Chromebook and looked, they realized that the kid was being bullied by multiple children, and the kid was actually planning to commit suicide. So what do we do? What do we do? Because it's impossible for a principal to look at every single child and to try to identify which child is at risk. So what is the solution,
Jethro: Yeah
Bella: So that's where we are coming from. We are trying to, instead of having a principal looking at each of the tablet one by one, which is unfeasible, how can we have a tool that is here and that can have the role of the principal on those tablets to make sure that each of the children that are being sent at school every morning come back safe and, you know, nothing bad is happening to them, or the parents doesn't find them, you know, hanging from a closet, for
Jethro: Yeah, for sure. And, and certainly we don't want that to ever happen to any of our young kids. And, um, and that is, that is an absolute tragedy for sure. And, um, and my, my fear is that sometimes we say we, we want to prevent that so badly that we are willing to do anything to, to do that. And to a certain extent, we should be willing to do anything.
But I have a concern when that means that we are surveilling everything that our kids are doing, and, um, and certainly we don't wanna use school devices to do anything bad to other people. Uh, and, you know, I just, I don't know what the right answer is, to be honest.
Bella: know.
Jethro: And, and that's, that's the challenge
Bella: That's exactly, that's the challenge, right? It's, it's difficult. What do we do, right? Do we sit back and we watch as we keep seeing, like, things coming up in the news? This has happened again on this social media platform or at school, or this teacher was actually grooming your child, and you had no idea.
What do we do, right? What is the solution? That's the solution that I came up
Jethro: Yeah.
Bella: now, and obviously, I'm open to feedback to figure out how we can make it better. Um, if there is a way, for example, to, you know, prevent becoming a surveillance solution, because that's not what I want. Because it has no value for me anyway because the information that I get, I am not using it.
I am not reselling it. And I'm not even able to see who is the child behind it. Only the principal is able to unmask the children. So from my side, we are purely focused on the detection and on making sure that we are sending the information to the principal so then they can work with a counselor or at least let the parents know if the child is in danger
Jethro: Yeah. Yep. Yeah. And, um, so when, when you do flag something, what, what happens then? Does the system automatically notify the principal? Uh, is M- Monarch itself involved, or what does that look like?
Bella: No. So as soon as we identify... So for example, one example is let's say the kid goes on Google, I think I mentioned it earlier, and type medication to potentially harm themselves, right? Okay. That's for example a tier two. if, for example, we notice two weeks later that this time the kid has found the appropriate dosage and is trying to figure out more information, now we know that this is a tier one, so it's really a threat to life and the kid might potentially do something. So every time there is an alert, the principal only receives it. Only them are able to unmask the child. Tablet, tablet number 15, for example, we do not know where it is. And then at this moment they decide if it is relevant or not, and then at that point it's on the school to decide what they wanna do with that information.
Jethro: Yeah. Okay. That makes sense. Um, and so when, when those things come in, are you keeping track of how many reports a school has, how many incidents they've had? Uh, is that something you can see or is that something only the principal can see? 'Cause you need to keep some data for your, for your company to say, "What we're doing is working," right?
So, s- so what, what do you keep and what do you use for your, for your records and for marketing and, and telling people why this is worthwhile? How do you manage that?
Bella: So we will absolutely be able to see the amount of incidents, um, that the school has received. If one specific tablet is receiving more incident than others, for example. Um, the type of incidents, right? Is it bullying? Is it grooming? Is it violence, right? We will be able to classify those, but that will be it really. Um, and that will be-- what that what will be of value to us. And then of course, we are also receiving the feedback from the principal to know if the information that the AI is providing is correct or not, or if it's something that's a false positive as well, because that's the only way that the solution will get smarter at understanding and not flood them,
Jethro: Yeah
Bella: of irrelevant alerts and notifications.
Jethro: Yeah. The, the irrelevant and not real issues are, are the ones that are the real problem. And the real problem there is that if you get five in a row that are not real problems, then you stop trusting the system, and then you might as well not even have it at that point. So
Bella: much.
Jethro: so that, that is a, a really important thing.
How do you, how do you keep track of that and, and make sure that that is as accurate and as timely and important as possible?
Bella: Well, we are still on the pilot, right? So every single school that we are working or will work with, they know, right, that they are contributing to something that's new, that doesn't exist yet. So they understand that there is false positive. So they are very, very, uh, supportive actually. Um, so right now there is no major problems.
It's working as intended. Um, but of course, if we notice that the false positive are increasing, then we'll take a step back and review with the engineering team and the AI, uh, engineer as well to see what's going on. But one particular thing that I will really want to emphasize is that we are very much focused on academic research as well. So we don't only rely on what's coming from the school, um, we really want to make sure that it is also backed by research from university, so then we are actually delivering the best product possible back to the school
Jethro: Yeah, very good. All right, uh, my last question each time is, uh, well, before I get to that, how do people get connected with Monarch and, and what's the, what's the best way for them to do that?
Bella: The best way is to go on www.monarchai.io or to contact me at support@monarchai.io and I will be very happy to talk to you, provide a demo or a pilot, and if you wanna become a customer, happy to contribute towards that as well
Jethro: Excellent. Uh, what does it, what does it cost for someone to use this?
Bella: So right now, actually, we are offering it for
Jethro: What? Oh my gosh
Bella: yes. Listen, I have been working on this for three years. This is truly a sacrifice that I'm
Jethro: Ja
Bella: Um, this is my own contribution. Uh, so for the pilot, it is absolutely free for now. Um, but as soon as we start scaling, then obviously it will be about $10 a tablet
Jethro: And dollar a device?
Bella: Yes.
Jethro: Good. Uh, good to know. Excellent. All right, uh, last question is what is one thing that a principal can do this week to better protect students in their daily work?
Bella: Hmm. I think you did mention that, and I know that principals are overwhelmed because I've been talking to a few
Jethro: Yes
Bella: I feel very sorry about that. I think, um, the one thing that principal could potentially do is to talk to families and encourage them to reduce the amount of digital devices that they have at home. I think at the end of the day, it's very important to be able to communicate with your children, so I suggest a flip phone, back to the old days.
Jethro: Yep
Bella: that's enough. And then, um, obviously it's, it's impossible to remove all the sm-smartphone, but that's a step. so yeah, that will be one advice that I provide to the principal to also reduce the amount of issues that you see at school because of the smartphone
Jethro: Excellent. All right. Very good. Uh, Bella, thank you so much for being here. That's monarchai.io. Please go check it out and reach out to her while it's still in the, in the beta, and she'll, she'll take however many she can before she, before she gets going, so. All right. Thank you so much, Bella. I appreciate you
Bella: Thank you. Thank you