Walter Haydock: Welcome to the Deploy Security Podcast, where we go deep on the privacy, security, and compliance implications of using artificial intelligence. I'm Walter Hadock, the founder of StackAware, and as always, I'm joined by Steve DeFor. Steve, good to see you.
Stephen Dufour: You too, sir.
Walter Haydock: So we've got an interesting lineup today. I'm gonna kick things off talking about a recent development from the White House. There's been a lot of movement in the area of AI, but the thing is, we're not calling it AI anymore. It is now called superintelligence because apparently AI makes it seem fake. So the president recently announced that. There is going to be a superintelligence force. So I was I was joking with a friend of mine who's also a Marine Corps veteran that we might be involuntarily recalled to the the superintelligence force or that it was the AI force for for for a hot second there. But looks like it's gonna be a civilian run organization primarily, although there is some representation from the Department of War. It is going to be run by Jay Clayton, who's the director of national intelligence. Although I did serve at the DNI as well, for for one part of my career. So yeah, I'll be interested to see exactly what this is, because there's been a lot of a lot of heat from the White House. I would say not a ton of light when it comes to AI or superintelligence regulation, but you know, there there have been quite a few. announcements from from this administration about AI and you know I look forward to seeing what this what this one means. On a related note, on the on the note of superintelligence, the White House released a accord on superintelligence with the CEOs of five of the hyperscalers and also Nvidia, which doesn't really do much of the compute but does the hardware design. And they Establish what they call a morally binding agreement regarding artificial intelligence controls, such as robust internal controls, creating an internal team to monitor AI-related safety and security issues, external auditing, and then creating an independent committee of the board of directors to oversee any issues and ensure their remediation. So this is a one-page document that is pretty light on details and I would say was written at a pretty pretty basic level. You know, some of these things are are are a little open ended saying, you know, we believe implementing these controls and audits is critical to ensuring a safe future for everyone and each of our companies are committed to doing this. I mean, yeah, that's it's good to hear. But I think I think the biggest news was that it was signed by the president of the United States instead of United States. So
Stephen Dufour: Ha ha ha.
Walter Haydock: That was unfortunate to to see that. And you know, hopefully we'll get some some spell checking on on future versions. They probably should have run this through Chat GPT to I d identify
Stephen Dufour: That's exactly what I was gonna say.
Walter Haydock: any any errors there. also, you know, Google does have a pretty powerful spell checking engine through through docs. But yeah, a lot of a lot of heat coming out of the White House on on this topic. Not a ton of light. And you know, my skepticism has been stated before, so I'll turn it over to you, Steve.
Stephen Dufour: Yeah, what what does it what does your gut tell you about this? Are they setting the stage for like I mean they they renamed artificial intelligence to superintelligence in all the documents, right? And Elon Musk is saying he's gonna rename his his company type type of thing. And it it just it just for me, it just seems like smoke and mirrors like It d it doesn't mean very much. I'm wondering what it's a distraction from. I'm not some big conspiracy theorist or anything like that. It just it just seems rather I guess I guess mundane. It's it's like, we're gonna rename this thing. Politically is this the set things apart from China, Europe, or any any of the con let's let's say the competition to the industry of the United States and artificial intelligence type of thing. It's like, you're doing artificial intelligence. We're doing super intelligence. So I'm waiting for China to come back and say, well, we're doing super duper intelligence, right? Like super intelligence squared, right? Right.
Walter Haydock: S D I S D I
Stephen Dufour: I I am I am I am glad that these frontier model companies like OpenAI and Anthropic sign this document because I I want to say it was two weeks ago or something to that effect, where we've got a lot of testimony in Congress where it sounded like these companies were begging for regulation, right? And I also heard some commentary regarding they're begging for regulation as in you guys regulate us and we can handle the regulation because we're already you know multi-billion dollar companies anyway. And and if the if the United States government enforced some sort of regulation, it would basically squash all the competition because the larger companies are the only ones that can potentially keep up with some sort of regulation. I am glad they signed it, even if it's just more symbolic in nature than anything else. I'd be very interested to see what those what that inventory of controls looks like and also what those committee meetings look like like So have board of directors, and the document that you were show showing earlier said something about a board of directors committee that is in charge of this thing. It's like, okay, so who who's the board of directors committee going to actually report these results to? And is it gonna look like anything we've seen in the past where not I'm not calling open AI out, it's just the first example that popped in my my mind, open AI's model. models break out, which they're supposed to do, right? They break out and gotta have a field day on on the internet. So are we going to see more like that? Because the follow up news to the initial breakout story was I forget who it was at at OpenAI OpenAI. It might might have been the CEO, it might have been somebody else, basically said there's tens of thousands of these types of incidents under investigation right now. So if that's the case, we're gonna get bombarded by all these investigations. Like what what does the output look like? Does it look like marketing or does it look like best practices being pushed throughout the artificial intelligence industry? Those are my questions.
Walter Haydock: Yeah, I think it's all remains to be seen. I mean, there there have been there have been a lot of these types of things under the Biden administration. There was a magnificent seven accord on AI safety and I mean, I'd love to see all this effort directed toward a comprehensive AI governance bill that kind of clarifies everything for the country and get industry involved. Let them you know, make suggestions and then have Congress vote on it and then we can kinda not put this to bed, but we can decide as a country where we're gonna where we're gonna go with with AI governance at the at the federal level. So I'd love to see a lot more effort focused on substantive things. I agree that a a symbolic gesture is better than no gesture, but I would love to see some some actual legislation. Although I mean, frankly having fifty different state level laws, and then frankly, some states have multiple laws and regulations that overlap with each other at the state level, like California. You know, that makes it help stack aware, frankly, like 'cause it creates demand for AI governance services. But, you know, I don't think that's best for our country.
Stephen Dufour: Yeah, hundred percent agree. And just so you know, I verified we're live on YouTube and LinkedIn right now. And we might have had a comment twelve hours ago from Michael Crumley. I d I didn't notice it at first until I came in and verified. And he basically says that yeah, all of all of the task force and things like that. It's just policy theater, right? Like just different wording and this thing that the larger company company sign as far as like the the moral contract or whatever wording you used earlier basically means nothing. Without enforcement from the government and they can only enforce their own laws, then it doesn't mean
Walter Haydock: Mm-hmm.
Stephen Dufour: very much. So thanks for the comment, Michael.
Walter Haydock: Yeah. Awesome. So you wanna take the next one?
Stephen Dufour: Yeah, let's do it. I'll try not to ramble too much. All right, so we're gonna share this window. Let me know if you can see it okay.
Walter Haydock: Yeah, I gotcha.
Stephen Dufour: Can. All right. Awesome. All right. So the next article, and let me copy and paste this into the chat for all the thousands of viewers. All right, awesome. So Florida Woman uses Claude as a diary. I've heard of this before. And if you take advantage of the memory settings within Claude, it can be something really good to keep track of things over time. What I wouldn't recommend is actually putting super personal things in it because these companies do have mechanisms to monitor. And at some point an actual human gets eyes on your chats. Like it's just the way it is. So it if you say that in the terms and conditions at OpenAI or Anthropic, within the terms and conditions, they reserve the right to monitor chats. Like They just do, especially when it comes to threats to human safety. So if you go in and you start using Claude and start putting personal information, and it doesn't have to be identifiable like within the chat, but we're talking about how you feel about a certain political candidate, how you feel about your wife or husband. And in this case, this Florida woman reported that she wanted to kill, let's see here. Yeah, she wanted to kill I wasn't it the CEO of some company? I'm sorry, I for I forgot exactly who she was gonna kill. Like the main point of the article was she had vote voc not vocalized, she had input her desire to murder someone. And then in a follow-up entry in her diary, she said she purchased a gun. Those initial interactions on the back end of Anthropic got flagged somewhere. And when they saw the follow-up, that she had purchased a gun in her diary entries. They contacted law enforcement and law enforcement became involved in in I believe she was arrested. So the lesson here for me is if you're gonna keep a diary, keep it on a bookshelf, like what's over my shoulder, and control access to it, right?
Walter Haydock: That is definitely true. I would say also, you know, don't put homicidal meandering in into something that someone else can can see. although I'm happy if she was serious about it that someone else did see it so that we could you know get some intervention there. Yeah, I mean I think there's been a fair amount of scrutiny of the hyperscalers, OpenAI and Anthropic, specifically because there have been incidents where folks have put this type of information into the chat bots and they've been flagged by the safety classifiers and the Company has not done anything. I know there was a shooting in Canada where some folks did not, or OpenAI did not take action, even though they had some some inklings that that something was going on. So I can expect that they're going to be a little more quick to notify law enforcement about these types of things. And then, you know, you create a whole set of there, there are the practical implications, and then there are the the kind of moral or the the ethical considerations as well, because, you know, some people treat these systems like they are pretty, you know, close confidence. C close confidants. They provide a lot of data to them. but I will say in, you know, people put a lot of pretty sensitive stuff in Google Drive.
Stephen Dufour: Yes.
Walter Haydock: They, you know, put stuff in SMS conversations, not iMessage, but SMS conversations, which are not more more or less not encrypted. you know, they say things on phone lines that are, you know, unencrypted. the president and vice president were the victims of phone-based surveillance from China in 2024. So It's important to understand who can read your content and whatever the medium is. So yeah, I think I think we're gonna see more incidents like this. And this is just kind of the the new wave of of people expressing themselves and expressing themselves in ways that suggest that they might commit crimes. So I I anticipate we're gonna see more more occurrences like this.
Stephen Dufour: Yeah, and I a hundred percent agree with you because it's in these companies' best interests to treat Treat reports that are flagged in the system as seriously as possible and basically like shift the burden of liability, right? Like we did what we were supposed
Walter Haydock: Mm, mm, mm.
Stephen Dufour: to. And from a human standpoint, of course, you want to protect people first and foremost. So
Walter Haydock: Mm-hmm.
Stephen Dufour: if somebody is using your platform, and we've seen over the years Facebook fail numerous times whenever it comes to
Walter Haydock: Mm-hmm.
Stephen Dufour: Let's just say active threats, right? Somebody made a post on on Facebook, it got flagged, but nobody got a got around to reviewing it, or
Walter Haydock: Mm-hmm.
Stephen Dufour: it was themed free speech or whatever, and then somebody
Walter Haydock: Mm.
Stephen Dufour: went and harmed other human beings. So I I believe, now that we're seeing more and more reports of this, that people are gonna realize how not private, like it's not public, but it's definitely not private that the these interactions are being and and you listed off a whole bunch of them where you know sms iMessage so I include iMet iMessage because under the law these these companies if they're subpoenaed for messages during a certain time frame due to court proceedings or anything like that they have a legal ob obligation to give them that data as as long as you know there there's a warrant like like pulling this information. So I hope people keep that in mind whenever they're using it. And then it doesn't have to be for nefarious pe for nefarious reasons as well. Like it can just be like, hey, I'm I'm having issues with my significant other. Can you give me some tips to, you know, like calm things down or how c how do I become a better communicator and stuff like that? Like these are very personal things that I might not want other people to know about, including the people at at open AI or anthropic or the judge, just in case the these records get subpoena for some reason. So yeah, a hundred percent agree.
Walter Haydock: Just quick quick correction on iMessage, if you are using advanced data protection or you are not backing up to the cloud, Apple has said they do not have the ability to retrieve the content of your iMessages.
Stephen Dufour: even even if it's subpoenaed, like completely private?
Walter Haydock: It's not yeah, they Apple doesn't have the ability to decrypt the data. So that was a big issue in the UK because advanced data protection basically makes your entire iCloud account, including any backed up iMessages, unrecoverable to Apple itself. So advanced data protection is actually illegal in the UK.
Stephen Dufour: Okay. Because they want the right to be able to go and subpoena those records. So and in the United States,
Walter Haydock: Yeah. Yeah.
Stephen Dufour: like we're good to go here?
Walter Haydock: It's legal. Yeah. So I don't know what'll happen if I show up in Great Britain with ADP on my iPhone. you know, I doubt I doubt anyone's gonna say anything, but you know, I mean this has been it this has been kind of an ongoing thing. I I I have strong feelings on this. I think it's kind of silly to to talk about you know. mandating limits on on encryption strength and we've kind of w America goes through this debate like once every five or ten years. And, you know, I think I think we there was another episode of this last year when the UK was was was trying to ban advanced data protection. but frankly I I think you're not gonna you're not gonna outrun technology with with with legislation in this specific case and I think there are other ways that law enforcement can can get the data they need, but I do believe that metadata is something that law enforcement could get if if they if they subpoena that in the United States. But content from iMessages I believe is not it's not recoverable, not technically recoverable by Apple.
Stephen Dufour: Yeah, thanks for informing I I had no idea that that was a thing in the United States. So really appreciate it. And so
Walter Haydock: Yeah, yeah, yeah, yeah.
Stephen Dufour: the debate that you mentioned earlier is the debate around dual use. Right. So like if we if we have let's say AES 256 or 512 or whatever it happens to be when it comes to encryption,
Walter Haydock: Mm-hmm. Mm-hmm.
Stephen Dufour: like that will protect all the good guys' messages and data, right? Like this is good stuff,
Walter Haydock: Mm-hmm.
Stephen Dufour: but the bad guys can also use it. Right. So like law enforcement
Walter Haydock: Sure.
Stephen Dufour: governments typically want some kind of workaround or back door. That way they can go back and monitor for previous incidents or occurrences of of a particular event.
Walter Haydock: Yeah, and I and interestingly, that hacking in twenty twenty four that victimized the president and vice president, the Chinese intelligence services used the Kalea Communications Assistance to Law Enforcement interface for nine different mobile providers to get in and retrieve the content from the victims. So the lawful intercept capability of the United States government. Was used by a foreign adversary to spy on mem future members of the United States government. So I would consider that to be kind of a a self-owned and you know, I think I think we're kind of moving past the point where that type of infrastructure really makes sense. And especially because nine different providers were victimized through the same attack vector. I view that as as pretty serious and I think we should reconsider how how how we're doing lawful intercept. yeah, so there's that. And then I forgot my next point, so I'll stop there.
Stephen Dufour: All right. We we could go to the next article. All right. And that one's mine with the healthcare one, right? All right.
Walter Haydock: Yeah. Cool. Cool. All right. Yeah, yeah, yeah, yeah.
Stephen Dufour: All right, here we go. All right. In summary, let me put this in the chat. Boop. It's my verbalization of a copy paste. All right. So a recent report produced by Blue Cross Blue S Blue Shield Association. So that's the top level blue. Then you have all the state level blues underneath it, like Blue Cross Blue Shield of Arkansas or Texas, whatever it happens to be. is claiming that hospitals that are using AI-related tools have already contributed to nearly one billion in additional expenses over two years by claiming patients had more challenging illnesses to treat. And and we were talking about this earlier. And when I read this, I was like, It's a hallucination, some kind of note taking app that the doctor is using, or a medical coding AI related tool that automatically does coding based on the notes that the doctor provides. It hallucinated. Like, it's going in there coding in a particular way. And then I I sat here and thought about it a little bit more. Like that that's the negative of AI, right? Like the probabilistic outcomes. I was thinking about it. I was like, well. What if the AI succeeded? Like it was a true positive where it read the notes, went in and then coded it, coded it, and it was 100% effective in its mandate. Like it did the right thing. This means that people who are visiting their their provider for for whatever reason already had some of these issues, and now they're being coded properly, which is a huge win for. for end users or receivers of medical care because what once the insurance companies like I I think it was Dr. Oz that said something like when these AI tools are gonna be used, there's gonna be this inflationary effect at first. So right now we have this baseline, right? Everything's good. We start using the AI tools, there's gonna be an inflation around the true positives, I I call them. where the AI tool was effective in whatever it was designed to do, either taking notes or medical coding. Right. And so a billion dollars a billion dollars is a lot of money. and I'd be very interested in seeing what the outcomes of these are. Like are the I I seriously doubt that the medical professional is going in and verifying things, but like the the actual like coke medical coding team at the hospitals or or the the provider locations if they're going back and verifying some of these things. And this goes back to something you and I have talked about. And I know it gets overused where you have the human in the loop. So let let's say you have let let's say thousands of medical claims, right? And they're all coded in a specific way. And then you you get an AI tool that's gonna help you deliver faster, deliver better, things like that. But you still have a medical coding team that goes in. And if you have thousands per day, there it's really hard to keep up with that quality assurance process. And so you do what I say like samples. So you have let's say a team of five, and your goal per day is to key a five percent of the total claims from let's say the previous day or something like that. I'd be very interested to see the feedback for these AI related products, either the the note-taking apps that listens to conversations between the provider and the patient, and then and then from the medical coding side, exactly what their rate of success is, right? Like how many of those particular codes that made up the $1 billion that Blue Cross Blue Shield Association is talking about, with what what the provider is seeing. Like where's is is there a disconnect? How much of that disconnect is real and how how much of it is imagined because the association's seeing it in dollars. I I want to know from the provider standpoint, but also the person receiving care from their from their standpoint how how effective these tools actually are.
Walter Haydock: Yeah, I mean, there's certainly the case of this surfacing more severe conditions that were were there. And then also there's the incentive side of things. Like who
Stephen Dufour: Yeah.
Walter Haydock: who who knows what the tuning and the prompting was of these systems, especially because the hospitals and the other providers are gonna want the more severe coding from a reimbursement perspective. So, you know, I was telling someone this earlier today, like you take You take AI and you throw it at something, it's just going to accelerate whatever direction you're headed. So if the incentive structure is all messed up, which I think it is in the US, then you're just gonna throw AI on it. You're just gonna accelerate that screwed up incentive structure. So I think we may be seeing a variety of things going on at the same time here.
Stephen Dufour: Yeah, I absolutely like you you nailed one of the issues being being the incentive structures. Yeah, absolutely.
Walter Haydock: Cool. All right.
Stephen Dufour: I will stop my share.
Walter Haydock: Next one. Yeah. So the last item I want to talk about is the Wall Street Journal reported that OpenAI fired three safety researchers for sharing information with an AI safety group. and it to me it actually wasn't even clear who they shared it with, but I believe it was Metter. the it's the model evaluation and something something organization. And I've I've reviewed quite a few of their reports because they do third party analyses of open AI and anthropic models. And actually they they As I mentioned a couple of weeks ago, they prior to the GPT 5.6 Soul incident with Hugging Face, they predicted or they noted that Five Pox Soul had both expansive cyber offensive cyber capabilities and was excessively aggressive in cheating on the evaluations that it got. And that was before the Hugging Face incident. So Metter has some interesting observations and they've got they've got some good information, but it seems like they You know, it doesn't a hundred percent say exactly what they did, but it looks like they you know, these researchers were providing sensitive data to Metter and this other company, Redwood Research. I mean, really I'd be I'd be interested to see you know, exactly what the data they provided was, because Metter does receive confidential information from OpenAI under NDA, which they they admit in their reports. They say we are bound by certain kind of prepublication. reviews from from OpenAI. So it sounds like they went these researchers went off the reservation a little bit to provide this this data to to Metter. But I'd love to I wish I knew what they provided. I'd love to see what that information was so that I could evaluate, you know, the the severity of it because there's been you know a lot of talk about existential risk. And you know, if if three folks felt like they were justified providing this data to a third party, then I think it's worth discussing what that was. If it was just commercial information that they shouldn't have been giving, then that's one thing. But if it's safety information, then I think OpenAI would be well served by describing at least what what what the offense was. But they've been a little tight lipped here.
Stephen Dufour: Yeah. And I I r I think with so many debates going on around let's say the ethics of artificial intelligence use, both personally and the workforce and then also like globally, with so many data centers being built and so many people becoming aware and just sick of these things. it's probably in open AI's best interest, like you mentioned. to be forthcoming with the information, right? Because it it's one thing where, like you mentioned before, it might be some intellectual property that the company owns or like sales forecasts or leads or or whatever it whatever it happens to be. It's basically secret to the company, right? and divulging that secret wouldn't necessarily harm a human being, right? It's a trade secrets type of thing. And like that's one thing, but like like you mentioned It would be very interesting to know exactly what was disclosed because if it comes from a frontier model that could potentially do harm to a human, right? Like if we turned it on and it took over the entire company. Like, what do we do? And we had to restore from backup, you know. Kind of kind of an over the the top example, but disclosing instead of reactively disclosing how. you know, the the model broke out of jail or something like that. Why don't you proactively disclose some of some of these things that you're investigating and what you're doing to stop them from happening in the future. I look at it as an opportunity from from the marketing standpoint, the reactive one where open AI broke out or anthropic like gave bad advice and like influenced somebody to do something bad or something like that. Like those those are missed opportunities to fix it before it becomes a problem. And it and it's also a missed opportunity from a marketing standpoint, because if you stop something bad from happening, right, like you have a great opportunity, especially in in the world in which we live, where so many people are now starting to be very anti anti-AI from a human standpoint, from data center standpoint, or even from company standpoint, people are a lot of times just sick of seeing what they deem as the AI slop of the world. Right. So like these these companies are I believe are really missing an opportunity to proactively discuss some of the positives, but also some of the negatives that could happen that they're stopping.
Walter Haydock: Yep, I agree. They could have turned it into you know, a marketing marketing event by saying, Hey, you know, here here's what we found, here's here's what we did about it. but I guess we we may never know what happened.
Stephen Dufour: Know anybody at Open AI we can contact and be like, Hey, what was this thing?
Walter Haydock: I do know some people at OpenAI.
Stephen Dufour: Mm.
Walter Haydock: I I probably won't ask about that.
Stephen Dufour: Okay.
Walter Haydock: What about this public embarrassment that just happened? So yeah, I won't yeah, I I
Stephen Dufour: Can I get a quote? Like you're a journalist?
Walter Haydock: yeah, I I I won't I won't be asking them that. I've I've asked enough out of OpenAI. They they admitted us into the the Daybreak Blue cyber program. So I'll I'm gonna I'm gonna hold hold off on my asks for now.
Stephen Dufour: You want to go to the last article?
Walter Haydock: Yeah, yeah, yeah. You go for it.
Stephen Dufour: Let's see if I can actually find it. Alright, we're gonna do this share, this article, and I'm gonna grab it and put it into our chat. Millions of viewers. There we go. Alright. This one is a Tennessee woman is seeking $10 million after being jailed for nearly six months. So what happened was I believe her name is Angela Lips. She is a grandmother. She was babysitting like a friend or a neighbor's child and she got arrested. And she spent some amount of time in the Tennessee jail before she could be extradited over to, I believe, South Dakota. And what and then she contacted a lawyer. She eventually got out. She spent nearly six months there and she's suing. So what happened on the back end was there was a fraud that happened in Let me let me make sure that was South Dakota. North Dakota. Sorry about that. City of Fargo, North Dakota. There was an incident of fraud, and they use facial recognition software and social media to face match. And she came came up in the results, like this is a potential hit. So they issued a warrant based on her online information that that is publicly available, she got arrested. And now the city of Fargo, North Dakota potentially has to pay $10 million, right? This is this is probably like a worst case scenario for anybody, right? Where you have law enforcement performing an active, realistic and might be everyday investigation using a particular tool that uses artificial intelligence, facial recognition, and you get dinged. Right. Like you're just living your best life. And the article doesn't go into her background if she has any priors or anything like that. But regardless of that, she's just living her life. She's got a house, car, children, grandchildren, and from all accounts, she's just living her life. And then because of a let's say a governance failure, right? So the technology is meant to do one thing it is to provide facial recognition matches. And then for somebody to review those matches. Right. And I really think it's up to governments to put on their scrutiny hat, right? Come up with the process around human review, multiple human reviews, especially if it especially if it comes to taking somebody's rights away. Right. And then like where in the process does it say she gets to spend six months in jail? Right. Like there are probably so many failures here. at a personal level, I hope she gets her money. And I hope that the city of Fargo improves their process, right? and and I'm not I'm not too sure about the interstate requirements behind, I guess, information sharing, but also like let's say the state of Tennessee making sure that the state of North Dakota did their due diligence and everything is good on their end because on their side in Tennessee, they're detaining somebody on good faith that North Dakota did all that they were supposed to. So I I think this is a case of multiple failures, both at the let's say at the police department level, there was a warrant issued so judiciary there as well, state, State of North Dakota, the state of Tennessee. And then yeah, I'm I'm really wondering why she spent so much time in jail, especially if she had a lawyer involved at some point. Was that just toward the end? And that's why she was released. Because according to her, and and to multiple people in this article, she looks nothing like the photo that was flagged as her. So I'm I'm really curious about more. I personal level, I hope she gets paid.
Walter Haydock: Yeah, I mean, I think it's unfortunate in our criminal justice system that it's not automatic that like if if you're not found guilty, I feel like it should be automatic that you get some sort of payment from from the government. Because if it's like, oops, we got the wrong woman or the wrong guy and you go through a trial and your name is dragged through the mud and you spend all this time and money on lawyers, at the end of the day, it's just like, sorry. You know. So I I could see I could see certainly some sort of like automatic payment if you're if you're arrested and not charged. If you're charged and you're not convicted, like I I I c I think the government should pay you damages for for what happened. And then, you know, it'll make if especially you want to look at incentives like prosecutors. I mean, maybe you might say that then only prosecutors will go after really high likelihood cases if they know that their department is gonna lose money, if they if they need to pay, you know, pay people who they prosecute who who don't get convicted. But, you know, I could see I mean, I think the government should should reimburse people if if it can't convict them, then, you know, then it then I guess it was a mistake. So it it shouldn't have happened in the first place.
Stephen Dufour: Yeah, and for those watching and those that click the link, as you read this, when you when you read the quotes in this article, like here, I'll give you an example. The lawsuit says investigators analyzed an image tied to a fake ID used by a suspect. So that means a suspect of who's suspected of using of committing fraud in the state of North North Dakota had a fake ID. And they used that image on the fake ID to do the facial recognition stuff. So it's like, so you're using a fake piece of like a faux piece of government issued ID. It's already fake to begin with. Why would you use that as your source of truth? Because she might have used a real image that she found from Facebook or something to that effect. So it's like it's almost it's it's very similar to like a what do they call it? The fruit of the poisonous tree. Right. Like in the criminal justice world, like
Walter Haydock: Mm-hmm.
Stephen Dufour: if you go out and you don't follow the process, anything you find or you you gather, such as information or evidence, if you don't follow the process, it's inadmissible. So you're already starting really like you're starting the investigative process very, very poorly because you're using something that can't be verified as accurate, like to start your investigation to begin with. So like Great opportunity for improvement here. I like that they're attempting it, but they need to like pull that from production and go back to beta testing and training and making sure that the entire process is vetted. that way nobody goes to jail unnecessarily. And I think you nailed it. yeah, again, she lost her home, she lost her car. I'm sure her her neighbors and people who knew her. knew of her arrest and extradition over to North Dakota. It's just a freaking nightmare. You spend your entire life building a life and it's just ripped apart by a mistake. And the mistake is at the governance layer.
Walter Haydock: Unfortunate note to to wrap
Stephen Dufour: Ha.
Walter Haydock: things up on, but hopefully we we will see some improvement in in the future. You got anything else before we close out?
Stephen Dufour: man, I have so many questions I could ask you. I think I think it was a year ago I asked you a unpr like a unscripted question. You wanna you wan you want another unscripted question?
Walter Haydock: Well, I know we're gonna do predictions later this year. So if you're if it was predictions, then let's hold off because we're gonna do that. Okay.
Stephen Dufour: Okay. It is not predictions. So
Walter Haydock: Okay.
Stephen Dufour: I love your company, Stack Aware. I've been following you on LinkedIn since you did vulnerability management. and now that we're a couple of years in with let's say AI being deployed to the masses, right? Like it's in your third parties, whether or not you ask for it, it's on your laptops and you can't turn it off. Like it is freaking every it's in law enforcement. Right.
Walter Haydock: Mm-hmm.
Stephen Dufour: It is literally everywhere. It's even in cases, case studies and law summaries, like it's it's literally everywhere. And What If you could go back three years and give yourself some advice regarding AI governance, knowing everything you know now, what would it be? This is Walter thinking, by the way, audience.
Walter Haydock: If I could give myself some advice on AI governance. You should start with structured data for everything and make that the source of truth. And then you can do AI inference on top of it. And that's should be true for basically anything. Wherever possible, have structured, validated data, and then you can interpret that using AI tools or, you know, just traditional analysis techniques. That's that's what I would say. I for a period of time thought that there was going to be basically like structuring data wouldn't matter at all because you could you're just eventually going to be able to have everything could be unstructured and you could just ask AI to extract it. But we're certainly not there yet. And AI is not perfect at pulling pulling unstructured data. I think combining the two capabilities is very powerful. And, you know. I mean, I've actually been kind of impressed at some of the recent advancements. Like, you know, I'm using the MCP for Xano, which is my no code development tool. And, you know, what what we do is we allow read access for the MCP so that it can pull down the architecture of all the functions, it can see all the functions, how they, you know, essentially build on each other, but any changes to the code are approved by a human before going into production.
Stephen Dufour: There you go.
Walter Haydock: But you know, it's a it I think that's the right combination because we a lot of our data processing is just standard deterministic, you know, operations on on on data. But AI can accelerate that. And when you inference on top of that structured data, you can get some some impressive results. So that's what I would say.
Stephen Dufour: As far as the unstructured, yeah, thanks for that. Like I really like seeing whenever people reflect on where they started and where they are and like share those lessons learned. because that is one thing that's valuable. Like it's very much like science, right? Like we have science going back, you know, thousands of years, right? And eventually those scientists started agreeing on a process like okay, if we're gonna like perform science as a function, right, we need to form these steps. And then eventually scientific discovery accelerates because one, you can recreate the the scenarios and environment and the outcomes that a previous scientist did. And you don't have to spend 10 years of research to get to that point. You get to take all of that research and build on top of it. Right. I I really think that sharing some lessons learned here and there. is okay. Like of course you're not going to do any sh trade secrets or or anything like that that you deem valuable. last question. As far as the un unstructured data goes, do you think superintelligence would be better than artificial intelligence?
Walter Haydock: No.
Stephen Dufour: I'd throw that in there. The circle back to the first article.
Walter Haydock: I think we're gonna waste a lot of time control Fing through documents finding and replacing. So I don't think it's a good use of time.
Stephen Dufour: I a hundred percent agree.
Walter Haydock: Yeah. Okay, well, on that note, on that frivolous note, that's I think it's better than a sad note to leave on a fr to end
Stephen Dufour: Yeah, that's right.
Walter Haydock: on a frivolous note. I'll I'll close this out. So to our audience, thank you very much for joining. This has been the Deploy Security Podcast. We will see you next time.
Stephen Dufour: I guess.
Stephen Dufour: You too, sir.
Walter Haydock: So we've got an interesting lineup today. I'm gonna kick things off talking about a recent development from the White House. There's been a lot of movement in the area of AI, but the thing is, we're not calling it AI anymore. It is now called superintelligence because apparently AI makes it seem fake. So the president recently announced that. There is going to be a superintelligence force. So I was I was joking with a friend of mine who's also a Marine Corps veteran that we might be involuntarily recalled to the the superintelligence force or that it was the AI force for for for a hot second there. But looks like it's gonna be a civilian run organization primarily, although there is some representation from the Department of War. It is going to be run by Jay Clayton, who's the director of national intelligence. Although I did serve at the DNI as well, for for one part of my career. So yeah, I'll be interested to see exactly what this is, because there's been a lot of a lot of heat from the White House. I would say not a ton of light when it comes to AI or superintelligence regulation, but you know, there there have been quite a few. announcements from from this administration about AI and you know I look forward to seeing what this what this one means. On a related note, on the on the note of superintelligence, the White House released a accord on superintelligence with the CEOs of five of the hyperscalers and also Nvidia, which doesn't really do much of the compute but does the hardware design. And they Establish what they call a morally binding agreement regarding artificial intelligence controls, such as robust internal controls, creating an internal team to monitor AI-related safety and security issues, external auditing, and then creating an independent committee of the board of directors to oversee any issues and ensure their remediation. So this is a one-page document that is pretty light on details and I would say was written at a pretty pretty basic level. You know, some of these things are are are a little open ended saying, you know, we believe implementing these controls and audits is critical to ensuring a safe future for everyone and each of our companies are committed to doing this. I mean, yeah, that's it's good to hear. But I think I think the biggest news was that it was signed by the president of the United States instead of United States. So
Stephen Dufour: Ha ha ha.
Walter Haydock: That was unfortunate to to see that. And you know, hopefully we'll get some some spell checking on on future versions. They probably should have run this through Chat GPT to I d identify
Stephen Dufour: That's exactly what I was gonna say.
Walter Haydock: any any errors there. also, you know, Google does have a pretty powerful spell checking engine through through docs. But yeah, a lot of a lot of heat coming out of the White House on on this topic. Not a ton of light. And you know, my skepticism has been stated before, so I'll turn it over to you, Steve.
Stephen Dufour: Yeah, what what does it what does your gut tell you about this? Are they setting the stage for like I mean they they renamed artificial intelligence to superintelligence in all the documents, right? And Elon Musk is saying he's gonna rename his his company type type of thing. And it it just it just for me, it just seems like smoke and mirrors like It d it doesn't mean very much. I'm wondering what it's a distraction from. I'm not some big conspiracy theorist or anything like that. It just it just seems rather I guess I guess mundane. It's it's like, we're gonna rename this thing. Politically is this the set things apart from China, Europe, or any any of the con let's let's say the competition to the industry of the United States and artificial intelligence type of thing. It's like, you're doing artificial intelligence. We're doing super intelligence. So I'm waiting for China to come back and say, well, we're doing super duper intelligence, right? Like super intelligence squared, right? Right.
Walter Haydock: S D I S D I
Stephen Dufour: I I am I am I am glad that these frontier model companies like OpenAI and Anthropic sign this document because I I want to say it was two weeks ago or something to that effect, where we've got a lot of testimony in Congress where it sounded like these companies were begging for regulation, right? And I also heard some commentary regarding they're begging for regulation as in you guys regulate us and we can handle the regulation because we're already you know multi-billion dollar companies anyway. And and if the if the United States government enforced some sort of regulation, it would basically squash all the competition because the larger companies are the only ones that can potentially keep up with some sort of regulation. I am glad they signed it, even if it's just more symbolic in nature than anything else. I'd be very interested to see what those what that inventory of controls looks like and also what those committee meetings look like like So have board of directors, and the document that you were show showing earlier said something about a board of directors committee that is in charge of this thing. It's like, okay, so who who's the board of directors committee going to actually report these results to? And is it gonna look like anything we've seen in the past where not I'm not calling open AI out, it's just the first example that popped in my my mind, open AI's model. models break out, which they're supposed to do, right? They break out and gotta have a field day on on the internet. So are we going to see more like that? Because the follow up news to the initial breakout story was I forget who it was at at OpenAI OpenAI. It might might have been the CEO, it might have been somebody else, basically said there's tens of thousands of these types of incidents under investigation right now. So if that's the case, we're gonna get bombarded by all these investigations. Like what what does the output look like? Does it look like marketing or does it look like best practices being pushed throughout the artificial intelligence industry? Those are my questions.
Walter Haydock: Yeah, I think it's all remains to be seen. I mean, there there have been there have been a lot of these types of things under the Biden administration. There was a magnificent seven accord on AI safety and I mean, I'd love to see all this effort directed toward a comprehensive AI governance bill that kind of clarifies everything for the country and get industry involved. Let them you know, make suggestions and then have Congress vote on it and then we can kinda not put this to bed, but we can decide as a country where we're gonna where we're gonna go with with AI governance at the at the federal level. So I'd love to see a lot more effort focused on substantive things. I agree that a a symbolic gesture is better than no gesture, but I would love to see some some actual legislation. Although I mean, frankly having fifty different state level laws, and then frankly, some states have multiple laws and regulations that overlap with each other at the state level, like California. You know, that makes it help stack aware, frankly, like 'cause it creates demand for AI governance services. But, you know, I don't think that's best for our country.
Stephen Dufour: Yeah, hundred percent agree. And just so you know, I verified we're live on YouTube and LinkedIn right now. And we might have had a comment twelve hours ago from Michael Crumley. I d I didn't notice it at first until I came in and verified. And he basically says that yeah, all of all of the task force and things like that. It's just policy theater, right? Like just different wording and this thing that the larger company company sign as far as like the the moral contract or whatever wording you used earlier basically means nothing. Without enforcement from the government and they can only enforce their own laws, then it doesn't mean
Walter Haydock: Mm-hmm.
Stephen Dufour: very much. So thanks for the comment, Michael.
Walter Haydock: Yeah. Awesome. So you wanna take the next one?
Stephen Dufour: Yeah, let's do it. I'll try not to ramble too much. All right, so we're gonna share this window. Let me know if you can see it okay.
Walter Haydock: Yeah, I gotcha.
Stephen Dufour: Can. All right. Awesome. All right. So the next article, and let me copy and paste this into the chat for all the thousands of viewers. All right, awesome. So Florida Woman uses Claude as a diary. I've heard of this before. And if you take advantage of the memory settings within Claude, it can be something really good to keep track of things over time. What I wouldn't recommend is actually putting super personal things in it because these companies do have mechanisms to monitor. And at some point an actual human gets eyes on your chats. Like it's just the way it is. So it if you say that in the terms and conditions at OpenAI or Anthropic, within the terms and conditions, they reserve the right to monitor chats. Like They just do, especially when it comes to threats to human safety. So if you go in and you start using Claude and start putting personal information, and it doesn't have to be identifiable like within the chat, but we're talking about how you feel about a certain political candidate, how you feel about your wife or husband. And in this case, this Florida woman reported that she wanted to kill, let's see here. Yeah, she wanted to kill I wasn't it the CEO of some company? I'm sorry, I for I forgot exactly who she was gonna kill. Like the main point of the article was she had vote voc not vocalized, she had input her desire to murder someone. And then in a follow-up entry in her diary, she said she purchased a gun. Those initial interactions on the back end of Anthropic got flagged somewhere. And when they saw the follow-up, that she had purchased a gun in her diary entries. They contacted law enforcement and law enforcement became involved in in I believe she was arrested. So the lesson here for me is if you're gonna keep a diary, keep it on a bookshelf, like what's over my shoulder, and control access to it, right?
Walter Haydock: That is definitely true. I would say also, you know, don't put homicidal meandering in into something that someone else can can see. although I'm happy if she was serious about it that someone else did see it so that we could you know get some intervention there. Yeah, I mean I think there's been a fair amount of scrutiny of the hyperscalers, OpenAI and Anthropic, specifically because there have been incidents where folks have put this type of information into the chat bots and they've been flagged by the safety classifiers and the Company has not done anything. I know there was a shooting in Canada where some folks did not, or OpenAI did not take action, even though they had some some inklings that that something was going on. So I can expect that they're going to be a little more quick to notify law enforcement about these types of things. And then, you know, you create a whole set of there, there are the practical implications, and then there are the the kind of moral or the the ethical considerations as well, because, you know, some people treat these systems like they are pretty, you know, close confidence. C close confidants. They provide a lot of data to them. but I will say in, you know, people put a lot of pretty sensitive stuff in Google Drive.
Stephen Dufour: Yes.
Walter Haydock: They, you know, put stuff in SMS conversations, not iMessage, but SMS conversations, which are not more more or less not encrypted. you know, they say things on phone lines that are, you know, unencrypted. the president and vice president were the victims of phone-based surveillance from China in 2024. So It's important to understand who can read your content and whatever the medium is. So yeah, I think I think we're gonna see more incidents like this. And this is just kind of the the new wave of of people expressing themselves and expressing themselves in ways that suggest that they might commit crimes. So I I anticipate we're gonna see more more occurrences like this.
Stephen Dufour: Yeah, and I a hundred percent agree with you because it's in these companies' best interests to treat Treat reports that are flagged in the system as seriously as possible and basically like shift the burden of liability, right? Like we did what we were supposed
Walter Haydock: Mm, mm, mm.
Stephen Dufour: to. And from a human standpoint, of course, you want to protect people first and foremost. So
Walter Haydock: Mm-hmm.
Stephen Dufour: if somebody is using your platform, and we've seen over the years Facebook fail numerous times whenever it comes to
Walter Haydock: Mm-hmm.
Stephen Dufour: Let's just say active threats, right? Somebody made a post on on Facebook, it got flagged, but nobody got a got around to reviewing it, or
Walter Haydock: Mm-hmm.
Stephen Dufour: it was themed free speech or whatever, and then somebody
Walter Haydock: Mm.
Stephen Dufour: went and harmed other human beings. So I I believe, now that we're seeing more and more reports of this, that people are gonna realize how not private, like it's not public, but it's definitely not private that the these interactions are being and and you listed off a whole bunch of them where you know sms iMessage so I include iMet iMessage because under the law these these companies if they're subpoenaed for messages during a certain time frame due to court proceedings or anything like that they have a legal ob obligation to give them that data as as long as you know there there's a warrant like like pulling this information. So I hope people keep that in mind whenever they're using it. And then it doesn't have to be for nefarious pe for nefarious reasons as well. Like it can just be like, hey, I'm I'm having issues with my significant other. Can you give me some tips to, you know, like calm things down or how c how do I become a better communicator and stuff like that? Like these are very personal things that I might not want other people to know about, including the people at at open AI or anthropic or the judge, just in case the these records get subpoena for some reason. So yeah, a hundred percent agree.
Walter Haydock: Just quick quick correction on iMessage, if you are using advanced data protection or you are not backing up to the cloud, Apple has said they do not have the ability to retrieve the content of your iMessages.
Stephen Dufour: even even if it's subpoenaed, like completely private?
Walter Haydock: It's not yeah, they Apple doesn't have the ability to decrypt the data. So that was a big issue in the UK because advanced data protection basically makes your entire iCloud account, including any backed up iMessages, unrecoverable to Apple itself. So advanced data protection is actually illegal in the UK.
Stephen Dufour: Okay. Because they want the right to be able to go and subpoena those records. So and in the United States,
Walter Haydock: Yeah. Yeah.
Stephen Dufour: like we're good to go here?
Walter Haydock: It's legal. Yeah. So I don't know what'll happen if I show up in Great Britain with ADP on my iPhone. you know, I doubt I doubt anyone's gonna say anything, but you know, I mean this has been it this has been kind of an ongoing thing. I I I have strong feelings on this. I think it's kind of silly to to talk about you know. mandating limits on on encryption strength and we've kind of w America goes through this debate like once every five or ten years. And, you know, I think I think we there was another episode of this last year when the UK was was was trying to ban advanced data protection. but frankly I I think you're not gonna you're not gonna outrun technology with with with legislation in this specific case and I think there are other ways that law enforcement can can get the data they need, but I do believe that metadata is something that law enforcement could get if if they if they subpoena that in the United States. But content from iMessages I believe is not it's not recoverable, not technically recoverable by Apple.
Stephen Dufour: Yeah, thanks for informing I I had no idea that that was a thing in the United States. So really appreciate it. And so
Walter Haydock: Yeah, yeah, yeah, yeah.
Stephen Dufour: the debate that you mentioned earlier is the debate around dual use. Right. So like if we if we have let's say AES 256 or 512 or whatever it happens to be when it comes to encryption,
Walter Haydock: Mm-hmm. Mm-hmm.
Stephen Dufour: like that will protect all the good guys' messages and data, right? Like this is good stuff,
Walter Haydock: Mm-hmm.
Stephen Dufour: but the bad guys can also use it. Right. So like law enforcement
Walter Haydock: Sure.
Stephen Dufour: governments typically want some kind of workaround or back door. That way they can go back and monitor for previous incidents or occurrences of of a particular event.
Walter Haydock: Yeah, and I and interestingly, that hacking in twenty twenty four that victimized the president and vice president, the Chinese intelligence services used the Kalea Communications Assistance to Law Enforcement interface for nine different mobile providers to get in and retrieve the content from the victims. So the lawful intercept capability of the United States government. Was used by a foreign adversary to spy on mem future members of the United States government. So I would consider that to be kind of a a self-owned and you know, I think I think we're kind of moving past the point where that type of infrastructure really makes sense. And especially because nine different providers were victimized through the same attack vector. I view that as as pretty serious and I think we should reconsider how how how we're doing lawful intercept. yeah, so there's that. And then I forgot my next point, so I'll stop there.
Stephen Dufour: All right. We we could go to the next article. All right. And that one's mine with the healthcare one, right? All right.
Walter Haydock: Yeah. Cool. Cool. All right. Yeah, yeah, yeah, yeah.
Stephen Dufour: All right, here we go. All right. In summary, let me put this in the chat. Boop. It's my verbalization of a copy paste. All right. So a recent report produced by Blue Cross Blue S Blue Shield Association. So that's the top level blue. Then you have all the state level blues underneath it, like Blue Cross Blue Shield of Arkansas or Texas, whatever it happens to be. is claiming that hospitals that are using AI-related tools have already contributed to nearly one billion in additional expenses over two years by claiming patients had more challenging illnesses to treat. And and we were talking about this earlier. And when I read this, I was like, It's a hallucination, some kind of note taking app that the doctor is using, or a medical coding AI related tool that automatically does coding based on the notes that the doctor provides. It hallucinated. Like, it's going in there coding in a particular way. And then I I sat here and thought about it a little bit more. Like that that's the negative of AI, right? Like the probabilistic outcomes. I was thinking about it. I was like, well. What if the AI succeeded? Like it was a true positive where it read the notes, went in and then coded it, coded it, and it was 100% effective in its mandate. Like it did the right thing. This means that people who are visiting their their provider for for whatever reason already had some of these issues, and now they're being coded properly, which is a huge win for. for end users or receivers of medical care because what once the insurance companies like I I think it was Dr. Oz that said something like when these AI tools are gonna be used, there's gonna be this inflationary effect at first. So right now we have this baseline, right? Everything's good. We start using the AI tools, there's gonna be an inflation around the true positives, I I call them. where the AI tool was effective in whatever it was designed to do, either taking notes or medical coding. Right. And so a billion dollars a billion dollars is a lot of money. and I'd be very interested in seeing what the outcomes of these are. Like are the I I seriously doubt that the medical professional is going in and verifying things, but like the the actual like coke medical coding team at the hospitals or or the the provider locations if they're going back and verifying some of these things. And this goes back to something you and I have talked about. And I know it gets overused where you have the human in the loop. So let let's say you have let let's say thousands of medical claims, right? And they're all coded in a specific way. And then you you get an AI tool that's gonna help you deliver faster, deliver better, things like that. But you still have a medical coding team that goes in. And if you have thousands per day, there it's really hard to keep up with that quality assurance process. And so you do what I say like samples. So you have let's say a team of five, and your goal per day is to key a five percent of the total claims from let's say the previous day or something like that. I'd be very interested to see the feedback for these AI related products, either the the note-taking apps that listens to conversations between the provider and the patient, and then and then from the medical coding side, exactly what their rate of success is, right? Like how many of those particular codes that made up the $1 billion that Blue Cross Blue Shield Association is talking about, with what what the provider is seeing. Like where's is is there a disconnect? How much of that disconnect is real and how how much of it is imagined because the association's seeing it in dollars. I I want to know from the provider standpoint, but also the person receiving care from their from their standpoint how how effective these tools actually are.
Walter Haydock: Yeah, I mean, there's certainly the case of this surfacing more severe conditions that were were there. And then also there's the incentive side of things. Like who
Stephen Dufour: Yeah.
Walter Haydock: who who knows what the tuning and the prompting was of these systems, especially because the hospitals and the other providers are gonna want the more severe coding from a reimbursement perspective. So, you know, I was telling someone this earlier today, like you take You take AI and you throw it at something, it's just going to accelerate whatever direction you're headed. So if the incentive structure is all messed up, which I think it is in the US, then you're just gonna throw AI on it. You're just gonna accelerate that screwed up incentive structure. So I think we may be seeing a variety of things going on at the same time here.
Stephen Dufour: Yeah, I absolutely like you you nailed one of the issues being being the incentive structures. Yeah, absolutely.
Walter Haydock: Cool. All right.
Stephen Dufour: I will stop my share.
Walter Haydock: Next one. Yeah. So the last item I want to talk about is the Wall Street Journal reported that OpenAI fired three safety researchers for sharing information with an AI safety group. and it to me it actually wasn't even clear who they shared it with, but I believe it was Metter. the it's the model evaluation and something something organization. And I've I've reviewed quite a few of their reports because they do third party analyses of open AI and anthropic models. And actually they they As I mentioned a couple of weeks ago, they prior to the GPT 5.6 Soul incident with Hugging Face, they predicted or they noted that Five Pox Soul had both expansive cyber offensive cyber capabilities and was excessively aggressive in cheating on the evaluations that it got. And that was before the Hugging Face incident. So Metter has some interesting observations and they've got they've got some good information, but it seems like they You know, it doesn't a hundred percent say exactly what they did, but it looks like they you know, these researchers were providing sensitive data to Metter and this other company, Redwood Research. I mean, really I'd be I'd be interested to see you know, exactly what the data they provided was, because Metter does receive confidential information from OpenAI under NDA, which they they admit in their reports. They say we are bound by certain kind of prepublication. reviews from from OpenAI. So it sounds like they went these researchers went off the reservation a little bit to provide this this data to to Metter. But I'd love to I wish I knew what they provided. I'd love to see what that information was so that I could evaluate, you know, the the severity of it because there's been you know a lot of talk about existential risk. And you know, if if three folks felt like they were justified providing this data to a third party, then I think it's worth discussing what that was. If it was just commercial information that they shouldn't have been giving, then that's one thing. But if it's safety information, then I think OpenAI would be well served by describing at least what what what the offense was. But they've been a little tight lipped here.
Stephen Dufour: Yeah. And I I r I think with so many debates going on around let's say the ethics of artificial intelligence use, both personally and the workforce and then also like globally, with so many data centers being built and so many people becoming aware and just sick of these things. it's probably in open AI's best interest, like you mentioned. to be forthcoming with the information, right? Because it it's one thing where, like you mentioned before, it might be some intellectual property that the company owns or like sales forecasts or leads or or whatever it whatever it happens to be. It's basically secret to the company, right? and divulging that secret wouldn't necessarily harm a human being, right? It's a trade secrets type of thing. And like that's one thing, but like like you mentioned It would be very interesting to know exactly what was disclosed because if it comes from a frontier model that could potentially do harm to a human, right? Like if we turned it on and it took over the entire company. Like, what do we do? And we had to restore from backup, you know. Kind of kind of an over the the top example, but disclosing instead of reactively disclosing how. you know, the the model broke out of jail or something like that. Why don't you proactively disclose some of some of these things that you're investigating and what you're doing to stop them from happening in the future. I look at it as an opportunity from from the marketing standpoint, the reactive one where open AI broke out or anthropic like gave bad advice and like influenced somebody to do something bad or something like that. Like those those are missed opportunities to fix it before it becomes a problem. And it and it's also a missed opportunity from a marketing standpoint, because if you stop something bad from happening, right, like you have a great opportunity, especially in in the world in which we live, where so many people are now starting to be very anti anti-AI from a human standpoint, from data center standpoint, or even from company standpoint, people are a lot of times just sick of seeing what they deem as the AI slop of the world. Right. So like these these companies are I believe are really missing an opportunity to proactively discuss some of the positives, but also some of the negatives that could happen that they're stopping.
Walter Haydock: Yep, I agree. They could have turned it into you know, a marketing marketing event by saying, Hey, you know, here here's what we found, here's here's what we did about it. but I guess we we may never know what happened.
Stephen Dufour: Know anybody at Open AI we can contact and be like, Hey, what was this thing?
Walter Haydock: I do know some people at OpenAI.
Stephen Dufour: Mm.
Walter Haydock: I I probably won't ask about that.
Stephen Dufour: Okay.
Walter Haydock: What about this public embarrassment that just happened? So yeah, I won't yeah, I I
Stephen Dufour: Can I get a quote? Like you're a journalist?
Walter Haydock: yeah, I I I won't I won't be asking them that. I've I've asked enough out of OpenAI. They they admitted us into the the Daybreak Blue cyber program. So I'll I'm gonna I'm gonna hold hold off on my asks for now.
Stephen Dufour: You want to go to the last article?
Walter Haydock: Yeah, yeah, yeah. You go for it.
Stephen Dufour: Let's see if I can actually find it. Alright, we're gonna do this share, this article, and I'm gonna grab it and put it into our chat. Millions of viewers. There we go. Alright. This one is a Tennessee woman is seeking $10 million after being jailed for nearly six months. So what happened was I believe her name is Angela Lips. She is a grandmother. She was babysitting like a friend or a neighbor's child and she got arrested. And she spent some amount of time in the Tennessee jail before she could be extradited over to, I believe, South Dakota. And what and then she contacted a lawyer. She eventually got out. She spent nearly six months there and she's suing. So what happened on the back end was there was a fraud that happened in Let me let me make sure that was South Dakota. North Dakota. Sorry about that. City of Fargo, North Dakota. There was an incident of fraud, and they use facial recognition software and social media to face match. And she came came up in the results, like this is a potential hit. So they issued a warrant based on her online information that that is publicly available, she got arrested. And now the city of Fargo, North Dakota potentially has to pay $10 million, right? This is this is probably like a worst case scenario for anybody, right? Where you have law enforcement performing an active, realistic and might be everyday investigation using a particular tool that uses artificial intelligence, facial recognition, and you get dinged. Right. Like you're just living your best life. And the article doesn't go into her background if she has any priors or anything like that. But regardless of that, she's just living her life. She's got a house, car, children, grandchildren, and from all accounts, she's just living her life. And then because of a let's say a governance failure, right? So the technology is meant to do one thing it is to provide facial recognition matches. And then for somebody to review those matches. Right. And I really think it's up to governments to put on their scrutiny hat, right? Come up with the process around human review, multiple human reviews, especially if it especially if it comes to taking somebody's rights away. Right. And then like where in the process does it say she gets to spend six months in jail? Right. Like there are probably so many failures here. at a personal level, I hope she gets her money. And I hope that the city of Fargo improves their process, right? and and I'm not I'm not too sure about the interstate requirements behind, I guess, information sharing, but also like let's say the state of Tennessee making sure that the state of North Dakota did their due diligence and everything is good on their end because on their side in Tennessee, they're detaining somebody on good faith that North Dakota did all that they were supposed to. So I I think this is a case of multiple failures, both at the let's say at the police department level, there was a warrant issued so judiciary there as well, state, State of North Dakota, the state of Tennessee. And then yeah, I'm I'm really wondering why she spent so much time in jail, especially if she had a lawyer involved at some point. Was that just toward the end? And that's why she was released. Because according to her, and and to multiple people in this article, she looks nothing like the photo that was flagged as her. So I'm I'm really curious about more. I personal level, I hope she gets paid.
Walter Haydock: Yeah, I mean, I think it's unfortunate in our criminal justice system that it's not automatic that like if if you're not found guilty, I feel like it should be automatic that you get some sort of payment from from the government. Because if it's like, oops, we got the wrong woman or the wrong guy and you go through a trial and your name is dragged through the mud and you spend all this time and money on lawyers, at the end of the day, it's just like, sorry. You know. So I I could see I could see certainly some sort of like automatic payment if you're if you're arrested and not charged. If you're charged and you're not convicted, like I I I c I think the government should pay you damages for for what happened. And then, you know, it'll make if especially you want to look at incentives like prosecutors. I mean, maybe you might say that then only prosecutors will go after really high likelihood cases if they know that their department is gonna lose money, if they if they need to pay, you know, pay people who they prosecute who who don't get convicted. But, you know, I could see I mean, I think the government should should reimburse people if if it can't convict them, then, you know, then it then I guess it was a mistake. So it it shouldn't have happened in the first place.
Stephen Dufour: Yeah, and for those watching and those that click the link, as you read this, when you when you read the quotes in this article, like here, I'll give you an example. The lawsuit says investigators analyzed an image tied to a fake ID used by a suspect. So that means a suspect of who's suspected of using of committing fraud in the state of North North Dakota had a fake ID. And they used that image on the fake ID to do the facial recognition stuff. So it's like, so you're using a fake piece of like a faux piece of government issued ID. It's already fake to begin with. Why would you use that as your source of truth? Because she might have used a real image that she found from Facebook or something to that effect. So it's like it's almost it's it's very similar to like a what do they call it? The fruit of the poisonous tree. Right. Like in the criminal justice world, like
Walter Haydock: Mm-hmm.
Stephen Dufour: if you go out and you don't follow the process, anything you find or you you gather, such as information or evidence, if you don't follow the process, it's inadmissible. So you're already starting really like you're starting the investigative process very, very poorly because you're using something that can't be verified as accurate, like to start your investigation to begin with. So like Great opportunity for improvement here. I like that they're attempting it, but they need to like pull that from production and go back to beta testing and training and making sure that the entire process is vetted. that way nobody goes to jail unnecessarily. And I think you nailed it. yeah, again, she lost her home, she lost her car. I'm sure her her neighbors and people who knew her. knew of her arrest and extradition over to North Dakota. It's just a freaking nightmare. You spend your entire life building a life and it's just ripped apart by a mistake. And the mistake is at the governance layer.
Walter Haydock: Unfortunate note to to wrap
Stephen Dufour: Ha.
Walter Haydock: things up on, but hopefully we we will see some improvement in in the future. You got anything else before we close out?
Stephen Dufour: man, I have so many questions I could ask you. I think I think it was a year ago I asked you a unpr like a unscripted question. You wanna you wan you want another unscripted question?
Walter Haydock: Well, I know we're gonna do predictions later this year. So if you're if it was predictions, then let's hold off because we're gonna do that. Okay.
Stephen Dufour: Okay. It is not predictions. So
Walter Haydock: Okay.
Stephen Dufour: I love your company, Stack Aware. I've been following you on LinkedIn since you did vulnerability management. and now that we're a couple of years in with let's say AI being deployed to the masses, right? Like it's in your third parties, whether or not you ask for it, it's on your laptops and you can't turn it off. Like it is freaking every it's in law enforcement. Right.
Walter Haydock: Mm-hmm.
Stephen Dufour: It is literally everywhere. It's even in cases, case studies and law summaries, like it's it's literally everywhere. And What If you could go back three years and give yourself some advice regarding AI governance, knowing everything you know now, what would it be? This is Walter thinking, by the way, audience.
Walter Haydock: If I could give myself some advice on AI governance. You should start with structured data for everything and make that the source of truth. And then you can do AI inference on top of it. And that's should be true for basically anything. Wherever possible, have structured, validated data, and then you can interpret that using AI tools or, you know, just traditional analysis techniques. That's that's what I would say. I for a period of time thought that there was going to be basically like structuring data wouldn't matter at all because you could you're just eventually going to be able to have everything could be unstructured and you could just ask AI to extract it. But we're certainly not there yet. And AI is not perfect at pulling pulling unstructured data. I think combining the two capabilities is very powerful. And, you know. I mean, I've actually been kind of impressed at some of the recent advancements. Like, you know, I'm using the MCP for Xano, which is my no code development tool. And, you know, what what we do is we allow read access for the MCP so that it can pull down the architecture of all the functions, it can see all the functions, how they, you know, essentially build on each other, but any changes to the code are approved by a human before going into production.
Stephen Dufour: There you go.
Walter Haydock: But you know, it's a it I think that's the right combination because we a lot of our data processing is just standard deterministic, you know, operations on on on data. But AI can accelerate that. And when you inference on top of that structured data, you can get some some impressive results. So that's what I would say.
Stephen Dufour: As far as the unstructured, yeah, thanks for that. Like I really like seeing whenever people reflect on where they started and where they are and like share those lessons learned. because that is one thing that's valuable. Like it's very much like science, right? Like we have science going back, you know, thousands of years, right? And eventually those scientists started agreeing on a process like okay, if we're gonna like perform science as a function, right, we need to form these steps. And then eventually scientific discovery accelerates because one, you can recreate the the scenarios and environment and the outcomes that a previous scientist did. And you don't have to spend 10 years of research to get to that point. You get to take all of that research and build on top of it. Right. I I really think that sharing some lessons learned here and there. is okay. Like of course you're not going to do any sh trade secrets or or anything like that that you deem valuable. last question. As far as the un unstructured data goes, do you think superintelligence would be better than artificial intelligence?
Walter Haydock: No.
Stephen Dufour: I'd throw that in there. The circle back to the first article.
Walter Haydock: I think we're gonna waste a lot of time control Fing through documents finding and replacing. So I don't think it's a good use of time.
Stephen Dufour: I a hundred percent agree.
Walter Haydock: Yeah. Okay, well, on that note, on that frivolous note, that's I think it's better than a sad note to leave on a fr to end
Stephen Dufour: Yeah, that's right.
Walter Haydock: on a frivolous note. I'll I'll close this out. So to our audience, thank you very much for joining. This has been the Deploy Security Podcast. We will see you next time.
Stephen Dufour: I guess.
