Cameron Gaudet: All right, well, welcome back to the Wear AI Brief. This is our monthly video series focused on real-world questions that are shaping AI governance today. So, per usual, each episode we're kind of exploring practical issues, perspectives, and feedback for questions, you know, discussion points submitted by you folks that are listening or interacting online. And so I appreciate those and and definitely keep those coming. I am Cameron Godet. As always, I'm joined by Walter Hadock of Stack Aware. Walter, how are we doing today?
Walter Haydock: Doing well, doing well. Getting it's it's basically fall here in New Hampshire, so
Cameron Gaudet: Is it? Okay.
Walter Haydock: yeah, going back to school and all that. is it
Cameron Gaudet: Yeah.
Walter Haydock: are you are your kids back in school?
Cameron Gaudet: They are. Yep. So my my oldest just started kindergarten, so it was kind of a big deal. but
Walter Haydock: Congrats, congrats.
Cameron Gaudet: but still waiting for that fall weather. I've I've got
Walter Haydock: Yeah.
Cameron Gaudet: what I refer to as a lot of northern blood in me and
Walter Haydock: Yeah.
Cameron Gaudet: and living in a a hotter, hotter state temperature wise is taking some adjustment. People locals always tell me, hey, you'll get used to it. And to be a percent honest, I'm waiting for that to kick in, but I I have not. I've not experienced those those benefits yet, but desperately, desperately looking forward to fall weather, albeit probably not quite as as beautiful as as New Hampshire.
Walter Haydock: Awesome.
Cameron Gaudet: Yeah, yeah, very good. Very good. all right, so let's we can jump into some of the submissions from this past month. thank you again for for those of you that sent things in and have continued the discussion, as it were, around AI governance or AI governance topics. So the first one, Walter, I'll grab is organizations are creating AI inventories everywhere. And the question is what distinguishes a living Decision useful AI inventory from a spreadsheet that quickly becomes obsolete.
Walter Haydock: An AI inventory that's useful is plugged into and syncing with your source of truth or multiple sources of truth as they may be. So that means cloud environments, it could be with CMDB, it could be with a governance, risk, and compliance platform. As long as you're tracking all of the data in one place, that's the key thing. And then also if you're making changes. In an automated fashion, that is the optimal way to do it. If you're not doing it automatically, then you definitely run the risk of staleness. And this can cascade throughout your program because if you are building your disclosures on using a certain version of a model and then engineering shifts that model when a new release comes out, it's going to be a long time if you have a manual review process to. Get that reflected in your disclosure documentation. So you could be giving customers old information. So my recommendation is whenever possible, automate that inventory. And if you want to use spreadsheets, that's fine. What Stack Aware does is for many of our customers, we are tracking the inventory in a spreadsheet, but it's syncing live via API. So if you're used to seeing a spreadsheet, you can still see a spreadsheet, but on the back end, there is live syncing going on. Keeping that up to date.
Cameron Gaudet: Mm-hmm. Yeah, yeah, it's kind of it's it's definitely become one of those things that's a little bit of both and if you're gonna still go the spreadsheet, spreadsheet route. So that's that's really really good advice there. on the next question here is when auditing organizations for AI governance, what is the single biggest technical friction point keeping security teams from maintaining a live real time inventory of those integrations? So I guess this like compared to the previous question, like where do you see that that breakdown happening, maybe often or or commonly?
Walter Haydock: If you want to get really technical and really specific, I'll tell you this. In AWS, there's no single endpoint that allows you to inventory or query all the resources in a given environment. Unlike, for example, Azure, which has a single API that lets you do that. And that makes it really challenging because you essentially need to iterate over every type of AWS resource. That exists and and check to see across every region if it's there. Of course, if you're doing an AI inventory, you can limit the types of resources that you're checking for. But that is a major friction point is AWS not having a single resource API. And then also just the AWS API is itself kind of a beast to deal with. So a lot of the work we do is is directly on the command line, just running. scripts because that is the easiest thing to do. And then, you know, taking that information and comparing it to the inventory to do kind of status checks. It's not perfect because you do get a little bit of that staleness. But yeah, the AWS AWS APIs, I know they don't want to break anything and they don't want to ever essentially change their their schema. So so I get that. But you know, would love to get a a resource API where you could interrogate everything in a customer's environment.
Cameron Gaudet: Yeah. Sure. Sure. Yeah, I'd I hadn't realized that that just flat out did not exist for for AWS. And there's a lot of people that are that are utilizing that. So that's yeah, that's that's kind of a gaping gaping hole as it were for for keeping track inventory wise. Yeah.
Walter Haydock: Yeah, and I mean Stack Aware has built obviously a lot of technical infrastructure to navigate this and we've got our fair share of tooling to plug into environments and frankly there are entire companies built on on this, like Wiz, for example, that just kind of navigates all of this in an automated fashion, less focused on the AI piece and more focused on the the general cybersecurity approach. But definitely a problem to be solved and there are companies that are solving it and And getting acquired by Google for twenty three billion dollars or something like that. Indeed. Indeed.
Cameron Gaudet: Right. Getting compensated for it. Yeah, yeah, yeah. Yeah, for sure. Good deal. the next question came from I think a LinkedIn post of your of your own. it was about organizations that switch AI models every couple of weeks. I think that was kind of the beginning of the post was a communication and conversation that you had had similarly. With someone on that. So the question that was included within the comments there was if an organization changes or rotates models every few weeks and only assembles evidence at the next audit to show that its process is operated, are we really certifying the management system in practice or only certifying a description of it?
Walter Haydock: If you are not tracking change control as it's happening or soon thereafter, then you're definitely not having a management system that is describing how you operate. And my recommendation is to again automate as much of this as possible for Stack Aware whenever we detect a new model via the API of a provider like OpenAI. That kicks off a workflow to look into it to see if that's something we want to move the API over to call. The user interface will be automatically upgraded to use that newest model. The ChatGPT user interface will be automatically upgraded. So we actually document that as a risk in our register, is that we can't control the speed of rollout of models, but it's a risk that we necessarily accept. And you know, we have mitigations in place. We could potentially you can roll back to a given model if it's if its successor is not performing very well. And yeah, the key is to track that in real time and and automate it as much as possible. At StackAware, we we have our asset inventory hooked up to Jira so that whenever a new model comes into into play, then we fire a a set of automations that creates tickets to go through and perform the necessary assessments and and then I mentioned we already have that risk that we've accepted of of the user interface just auto-upgrading on its own. But we've documented it and there's nothing we can do about it. So we're gonna we're gonna accept that risk and then the other risks that we identify, then we'll manage them appropriately.
Cameron Gaudet: Yeah. Yeah, and I was I was thinking as you were talking about like accepting that risk. There's also there's also building putting things in place with the anticipation of that specific risk, right? Like to your point of having those having those structures in place, knowing that each of these providers are going to update their model. They're going they're going to continue refining and and sending out, you know, either an update or just a full full scale change. So having things in place, understanding that and making the assumption that that will happen, is wise within the management system there. The the next question here is: how should companies govern third-party AI products when they do not control the training weight, model weights, release schedule, or underlying infrastructure, but are still accountable for how the tool is deployed? So kind of a kind of a foundational, you know, third-party management type of question.
Walter Haydock: If you're using AI models that are hosted by other organizations and are closed source, so you don't have access to the training data, the weights, the infrastructure, things like that, you'll necessarily need to rely on the representations of those vendors. For example, OpenAI and Anthropic, they publish model cards that explain at a high level what these models are trained on, what the data cleaning and processing. Done beforehand was. And oftentimes they have decent information in them that can highlight some key risks. What StackAware does is we use an AI governance standard that says if we're processing this type of data, then we expect this level of granularity of information and this level of risk from a given model. For example, you know, if we can't confirm or deny that the training data is. you know, relatively free of third-party intellectual property claims, then that could potentially trigger a risk. That doesn't mean everything sh grinds to a halt. It just means that this is a risk that we identify because it's fallen below our standard. And we look at that risk, we apply an assessment per ISO 40 2001, and then we decide whether to accept mitigate transfer or avoid that risk. So the approach should be the same from a risk management perspective, whether or not you are hosting the model. The key difference would be how much information you have available if you are not hosting the model and if you're using a closed source model.
Cameron Gaudet: Yeah. Yeah, that makes sense. And I I know that you've you've stated something in in the past that stuck with me, which is everybody at some point in running a business, you have to be able to trust, you know, what what you're given for information. But obviously in this case, you know, there's there could be a difference in how much information you have, but you have to be able to, you know, trust trust those agreements that are put in place. So yeah. It's good good piece of advice there. the next next question's kind of brought up in the news as it were. but as of August 2nd, Article 50 of the EU AI Act made certain AI-facing disclosures and content labeling duties legally applicable, such as requiring disclosures in defined circumstances. So I guess in we can we can touch on the EU AI Act of it all. But More specifically the question that was raised was where do organizations come up short or perhaps a positive spin, have ample room to grow when it comes to aid I AI disclosures and maybe with with the thought in mind of of these new these new requirements being inactive or active.
Walter Haydock: With Article 50 coming into force, we're seeing the technical challenges of watermarking or labeling regimes. And I've long been a skeptic of this type of approach. I think we unfortunately have it backwards here, where the goal should be to label and tag and certify authentic content through digital signatures at the hardware level. rather than flagging AI generated content because you know it gets a little tricky. you know, the EU AI Act has definitions, but it's a little bit it's a little bit blurry and we're still missing all of the harmonized standards on on the EU AI Act that led to the pushback of the high risk AI deadlines to the end of 2027. So there is a lot of gray area. We're seeing technical challenges. I know there were some issues with anthropics. Content labeling approach, the C2PA provenance framework is, you know, it makes sense. But of course, if you change one pixel in an image that's tagged with C2PA, then you remove that that tracker from it, and you can't prove that it was generated with AI. Even unintentionally, for example, if I generate something using Chat GPT, and then I put the stackware logo on it, it's not going to have the C2PA in there. And I would need to figure out some sort of alternative way to mark the document as containing AI generated content. So I don't love the policy approach, and companies are are doing their best to meet the requirements, but I don't see this as being a sustainable practice in the future. And then And then also from a public policy perspective, I think this is causing people to think the wrong way in that, you know, I shouldn't look for a a marker of inauthenticity. I should look for a marker of authenticity.
Cameron Gaudet: Yeah. Yeah, it makes sense. I mean that it may seem it may seem like nuance, but that really changes the mindset the way that the way that you're approaching it of of if you're looking for you're looking for something authentic versus you know, inauthentic or unauthentic. So makes makes a lot of sense for that. And and I echo with the with the EU AI Act that it's there's still a lot of this, you know I guess mi nebulous might be too harsh of a word, but there's there's just still a lot of moving Hearts for it. So but but good question and appreciate appreciate that question for for those that that might be thinking of of kind of seeing how this how this sorts out. So all right, another question that had come in was say you are an AI governance lead. if executives come to you and say they want transparent or trustworthy AI, what questions do you ask to clarify in how to perceive? Or or maybe More so like what would make you skeptical of an organization claiming that it has responsible, safe, or trustworthy AI? Maybe maybe some of the the defining characteristics of those terms.
Walter Haydock: For an organization that's looking to demonstrate transparent or trustworthy AI development or use, I would drill down into exactly what the root of the requirement is. And oftentimes it would be driven by a customer, and in that case, not necessarily be perfectly clear. So there are
Cameron Gaudet: Sure.
Walter Haydock: companies, a lot of companies out there, including very big ones, that. Don't a hundred percent know exactly what they want from AI governance, risk management, things like that. And they will throw a bucket of terminology and contractual requirements at the wall, slash their vendors, and see what sticks. And that can create a challenging environment for both parties because the vendors will try to meet the requirements of the customers, but the customers don't fully know what they want or. Some things they say they want, they might not actually care about, or they don't actually want. So, you know, to tackle this type of situation, what Stack Aware has done recently is we drafted an AI and security addendum that can be added to contractual agreements. And what it does is it maps to existing standards like ISO 42001, ISO 27001, and SOC 2, and says if you cover these criteria, the common criteria, trust services criteria in SOC two, or the NXA controls for the ISO standards, then you get credit under these certain areas of the contractual addenda, addendum for, you know, for example, intended use or for human oversight or for data provenance. And that allows you to get a lot more tactical and specific when it comes to trustworthy, safe, responsible AI.
Cameron Gaudet: Sure. Yeah, yeah, it makes makes a lot of sense. And and I appreciated that question specifically for this for these types of discussions, 'cause as I I feel like I say at the start of every video for for aware AI brief, you know, we're looking for we wanna we wanna talk about AI governance in a very practical, tangible way, right? And and oftentimes to your point, these terminologies that are thrown around are the leading cause of Of non-practical discussions on AI is is really getting down to the the what do you mean what do you mean by that, right? it reminds me reminds me a lot of having small children as I do, which is half the time I spend, you know, they're learning to talk. So half the time I spend figuring out what they're trying to say, and then even from that trying to figure out what that means, right? So like it's it's a multi-layered of of communicating properly and making sure that we're on this. on the same page with terms, so totally totally makes sense. yeah. For sure.
Walter Haydock: I get asked why a lot. So I definitely definitely it's definitely good practice for customer due diligence when you get asked why a lot. And you need to go
Cameron Gaudet: It definitely is. Yeah.
Walter Haydock: down to the kind of quantum particle level explanation
Cameron Gaudet: Ha ha ha.
Walter Haydock: to to to satisfy the the asker. So definitely sympathetic
Cameron Gaudet: Right. Right.
Walter Haydock: there.
Cameron Gaudet: Yeah. Yeah, yeah, for sure. The quantum part. You're absolutely right. It gets it gets real granular. It gets real granular. so we've kind of we've done this in the past. I'm I'm gonna do just kind of throw out some different terms or different topics and in in lieu of of some, you know, football, college football, NFL kicking off, I'm re I'm reminded of my one of my favorite sports media tropes, which is over under properly rated. So I'm gonna pull that back for a second time here and and just throw out a couple couple different topics, discussions that I see happening online, and get your take on over, under, or properly rated for these terms or discussion points. So first up in that category would be prompt injection as the leading enterprise AI security risk.
Walter Haydock: I would say prompt injection is appropriately described as the leading enterprise AI security risk. And that's because with the explosion in agentic AI that has connectivity to lots of different systems and can impact data confidentiality, integrity, and availability, having a malformed or malicious prompt that sends an agent off to do something unintended could Have pretty substantial consequences. And prompt injection is one of those problems that is unfortunately never completely solvable. With SQL injection, because you have deterministic input and structured rules, structured query language, then you can be sure at a certain point that you've checked all your boxes. But that's not true with prompt injection. And when you have million token inputs, There's always something in there that can throw the model for a loop. So I would say prompt injection remains the leading, if not a leading, enterprise security risk.
Cameron Gaudet: deal good deal all right next next one for these these are human in the loop over under properly rated for your take
Walter Haydock: Human in loop is officially overrated at this point. And I've got some data to back that up. Anthropic recently published a study showing that humans caught significantly fewer security and safety issues than Claude's auto review capability. And as a result, they made auto approval of agentic actions in Claude for certain plans and new conversations or new tasks. Made that the default because humans are unfortunately not as good at identifying certain threats. Now you can question the data or the study, but I think that as AI gets better, humans are going to be relatively worse in governing them and telling them what's a correct output, what's an incorrect output. And where humans will add the most value will be saying. This type of thing is a correct output. And here are the parameters, here are the goals we're trying to achieve, but AI will need to figure out how to achieve those parameters and those goals. So if you force a human to review every action, increasingly that's gonna lead to a worse outcome.
Cameron Gaudet: Yeah. Yeah, I'd I'd seen that I I think possibly from yourself or or perhaps others, the the statistics on that you said I think you said anthropic had had listed for that for that measurement with human loop made me think of I don't know if you're a big fan of the office, the American version, but there's a there's a part where they go online and the a website starts taking sales calls and Dwight is trying to trying to beat the website, right? And he's like hanging with them through about five, ten, twelve. And then once it gets past twenty, it just obviously exponentially out does outdoes the person making the sales calls. And and when I saw that I I kept waiting for somebody to post that with like a with a attached meme of of that office scene. Cause as someone that's done human in the loop activities, like absolutely makes sense that it's just it reaches a point where I think you had listed in a in a post about talking about those edge cases and and that being that being where the real value is for for human input and human review. So yeah, good good point.
Walter Haydock: And at Stack Aware, you know, you alluded to the human review that you've done. Where where we're going is humans at the beginning reviewed everything and then reviewed a sample of things and then reviewed questionable cases, then reviewed, you know, a a portion of those questionable cases. And then all along we're we're kind of tweaking the machine, we're refining the machine
Cameron Gaudet: Mm-hmm.
Walter Haydock: to make the machine better at making those types of decisions. But I don't think we'll pull a human entirely out of the loop, we'll have sampling to make sure that we're within our risk tolerances. And as part of our ISO 40 2001 AI management system, we have set thresholds for what's acceptable level of accuracy and what's not. And we'll continue to monitor for that. But as AI gets better, then we may need to sample less often and and you know with less intensity.
Cameron Gaudet: Yeah. Yeah. And and under and like you said, it's the it's the framing of it and and teaching it what to do, maybe more so than just evaluating every single every single output. So next one in this category, we'll get through a couple more here before close, but AI red teaming as proof of governance maturity.
Walter Haydock: AI red teaming, excuse me. AI red teaming is a key part of any risk management program. And you definitely want to continually test your AI systems that are in production and prior to putting them into production. I will say that sometimes organizations jump to that without having architectural reviews and kind of basic security measures in place. And Red teaming identifies some serious architectural flaws. And in that case, it will reveal a lack of maturity rather than proving maturity. So I would say if any organization is considering AI red teaming, that's something that Stack Aware does. Happy to talk to you about it. But first, check out our AI risk readiness kit, which we put together with Sentry Security that highlights the top five. issues that we find during red teaming exercises and that will lay out the things to watch out for before you go into a red team exercise. So you can come out and you can thoroughly say, yes, this is proof of maturity since we have had a red team exercise and no substantial findings.
Cameron Gaudet: Yeah. Yeah, and that's that's an incredible, incredible resource there. So definitely if if if you're someone that that is considering that, looking into that, definitely, definitely check that out. The next one is micro segmentation or neck network segregation for AI agents.
Walter Haydock: I think microsegmentation is appropriately rated and can certainly help in controlling the blast radius of a security event. If you prevent an agent from having access to data that it doesn't need to perform its job, then it's much less likely to negatively impact that data if something goes wrong. So if an agent needs to only write to a database, then Probably doesn't need read capability. It's it's quite possible that it can make blind writes to the database and everything can be fine. And if for some reason an attacker hijacks that agent or it goes rogue, then it's not going to be pulling your entire SQL database out and dumping it into a place where it shouldn't be. So definitely agree that microsub segmentation with AI agents is is a key practice. Being able to achieve it. Potentially more challenging. There's always the engineering/slash IT impulse to just crank all the permissions up to the maximum level every time there's any sort of bug or any sort of error. You know, just just give it read-write all across the entire system. And,
Cameron Gaudet: Yeah.
Walter Haydock: you know, and then we'll we'll fix it later. And then it kind of never gets fixed. So that
Cameron Gaudet: Yeah.
Walter Haydock: would be a cautionary point there. And but I would say micro segmentation is Security.
Cameron Gaudet: Sure, sure. All right, last last couple of these for for overly over under properly rated. this one, the ability to see what a AI agent is doing in real time.
Walter Haydock: The ability to see what an AI agent is doing in real time is overrated, in my opinion, because first of all, an AI agent is probably moving faster than a human would be able to react. I mean, there are some cases I know in the chat GPT models, you can steer them mid mid-prompt or mid-response.
Cameron Gaudet: Mm-hmm.
Walter Haydock: So that could potentially be a useful exception there. And then also just the volume of things that are going on. If you use the open AI API and you send it a detailed query, it's gonna check a hundred websites. So there's no way a human can review every single website that it consulted. And, you know, where you want to put the controls would be before the resulting data has an impact. So, you know, at Stack Aware, we use AI agents to check on information that is out there in, you know, in in the wild. And The first time that we look at a website, a human always reviews the outputs from the agent to make sure there's nothing malicious there. and then we use deterministic checks like using hashes to see if anything on the website has changed. In that case, it's easy. You know, if nothing changed, then there's there's no malicious addition, then you can just update your your entry and move on. And then for existing sites that we've documented, the AI agent goes out and it sees if anything's changed. And You know, it's it's relatively unlikely that a formally trusted site has been hijacked and made malicious, certainly possible, but that's where our sampling would come into play to check to see if anything's changed. And but that's all after the fact. And it's done, you know, with some control gates along the way to make sure that no corrupted data is being pushed into production. But it's not done in real time because even at a small scale, it's not possible.
Cameron Gaudet: Yeah. I I always like when people are like, I get to I get to see it, you know, what it's doing and stuff. And I I think out of curiosity, sometimes it's it's fun to see like what moves it's making. To your point to to be able to you know, direct course and things like that sometimes. especially before any sort of any sort of output that would be that would be detrimental, you know, for for that case that you mentioned. But sometimes it's like, yeah, it the way what it's doing is working so much faster than we'd actually be able to see it all in real time anyway. So it feels feels a little bit like Cliff Notes, Cliff Notes version of operation taking place. But all right, last one for over under properly rated. AI agents should be governed like a service account.
Walter Haydock: Governing AI agents like a service count is a terrible idea, in my opinion, and highly overrated. In general, I don't think service counts should exist. The concept is now so heavily embedded into IT practices that I'm not going to change it. But with AI agents, we have the ability, perhaps, to do things the right way. And we're already seeing a whole wave of startups out there that. Say they govern non-human identities, such as artificial intelligence agents. And I've been critical of the term. I would say, let's just treat an agent like any other asset that has a named owner who is accountable for it, who you can go back to and say, why is this thing doing what it did? And either that person gives you a good answer or the person says, I don't know. And in that case,
Cameron Gaudet: Mm-hmm.
Walter Haydock: then that person needs to. decide to accept, mitigate, transfer, or avoid the risks and either apply some controls to rein in the agent to avoid the risk and shut it down, to accept it because they say, hey, the productivity benefits outweigh the risks. But having this amorphous thing or swarm of things out there where no one is named as the owner of it is a recipe for disaster. And I think that's Gotten a lot of organizations into trouble with service accounts. And I would encourage folks governing AI agents to take a different approach.
Cameron Gaudet: Yeah. Yeah. And and and I know that it I know it's a topic I did definitely throw that one in because I knew it's a topic that you've that you've come at hot before, the the NHI stuff. So if you want more information on that, there's definitely definitely ability to check out posts or or even the the deploy securely blog as well, the the stack aware blog. So definitely definitely check that out if you're looking for more information on that. all right, well that's that's the questions that that came in here this month. again, thank you all for sending questions. If you have an AI governance question or a challenge you're facing or just a topic you'd like us to cover in a future episode, feel free to send that. Send a direct message to myself or Walter on LinkedIn. Leave leave it in the comments of posts or on this YouTube on YouTube videos as well. But but thank you to everyone who submitted questions and thank you Walter for taking the time joining in this conversation here today. appreciate it. Appreciate it very much.
Walter Haydock: Doing well, doing well. Getting it's it's basically fall here in New Hampshire, so
Cameron Gaudet: Is it? Okay.
Walter Haydock: yeah, going back to school and all that. is it
Cameron Gaudet: Yeah.
Walter Haydock: are you are your kids back in school?
Cameron Gaudet: They are. Yep. So my my oldest just started kindergarten, so it was kind of a big deal. but
Walter Haydock: Congrats, congrats.
Cameron Gaudet: but still waiting for that fall weather. I've I've got
Walter Haydock: Yeah.
Cameron Gaudet: what I refer to as a lot of northern blood in me and
Walter Haydock: Yeah.
Cameron Gaudet: and living in a a hotter, hotter state temperature wise is taking some adjustment. People locals always tell me, hey, you'll get used to it. And to be a percent honest, I'm waiting for that to kick in, but I I have not. I've not experienced those those benefits yet, but desperately, desperately looking forward to fall weather, albeit probably not quite as as beautiful as as New Hampshire.
Walter Haydock: Awesome.
Cameron Gaudet: Yeah, yeah, very good. Very good. all right, so let's we can jump into some of the submissions from this past month. thank you again for for those of you that sent things in and have continued the discussion, as it were, around AI governance or AI governance topics. So the first one, Walter, I'll grab is organizations are creating AI inventories everywhere. And the question is what distinguishes a living Decision useful AI inventory from a spreadsheet that quickly becomes obsolete.
Walter Haydock: An AI inventory that's useful is plugged into and syncing with your source of truth or multiple sources of truth as they may be. So that means cloud environments, it could be with CMDB, it could be with a governance, risk, and compliance platform. As long as you're tracking all of the data in one place, that's the key thing. And then also if you're making changes. In an automated fashion, that is the optimal way to do it. If you're not doing it automatically, then you definitely run the risk of staleness. And this can cascade throughout your program because if you are building your disclosures on using a certain version of a model and then engineering shifts that model when a new release comes out, it's going to be a long time if you have a manual review process to. Get that reflected in your disclosure documentation. So you could be giving customers old information. So my recommendation is whenever possible, automate that inventory. And if you want to use spreadsheets, that's fine. What Stack Aware does is for many of our customers, we are tracking the inventory in a spreadsheet, but it's syncing live via API. So if you're used to seeing a spreadsheet, you can still see a spreadsheet, but on the back end, there is live syncing going on. Keeping that up to date.
Cameron Gaudet: Mm-hmm. Yeah, yeah, it's kind of it's it's definitely become one of those things that's a little bit of both and if you're gonna still go the spreadsheet, spreadsheet route. So that's that's really really good advice there. on the next question here is when auditing organizations for AI governance, what is the single biggest technical friction point keeping security teams from maintaining a live real time inventory of those integrations? So I guess this like compared to the previous question, like where do you see that that breakdown happening, maybe often or or commonly?
Walter Haydock: If you want to get really technical and really specific, I'll tell you this. In AWS, there's no single endpoint that allows you to inventory or query all the resources in a given environment. Unlike, for example, Azure, which has a single API that lets you do that. And that makes it really challenging because you essentially need to iterate over every type of AWS resource. That exists and and check to see across every region if it's there. Of course, if you're doing an AI inventory, you can limit the types of resources that you're checking for. But that is a major friction point is AWS not having a single resource API. And then also just the AWS API is itself kind of a beast to deal with. So a lot of the work we do is is directly on the command line, just running. scripts because that is the easiest thing to do. And then, you know, taking that information and comparing it to the inventory to do kind of status checks. It's not perfect because you do get a little bit of that staleness. But yeah, the AWS AWS APIs, I know they don't want to break anything and they don't want to ever essentially change their their schema. So so I get that. But you know, would love to get a a resource API where you could interrogate everything in a customer's environment.
Cameron Gaudet: Yeah. Sure. Sure. Yeah, I'd I hadn't realized that that just flat out did not exist for for AWS. And there's a lot of people that are that are utilizing that. So that's yeah, that's that's kind of a gaping gaping hole as it were for for keeping track inventory wise. Yeah.
Walter Haydock: Yeah, and I mean Stack Aware has built obviously a lot of technical infrastructure to navigate this and we've got our fair share of tooling to plug into environments and frankly there are entire companies built on on this, like Wiz, for example, that just kind of navigates all of this in an automated fashion, less focused on the AI piece and more focused on the the general cybersecurity approach. But definitely a problem to be solved and there are companies that are solving it and And getting acquired by Google for twenty three billion dollars or something like that. Indeed. Indeed.
Cameron Gaudet: Right. Getting compensated for it. Yeah, yeah, yeah. Yeah, for sure. Good deal. the next question came from I think a LinkedIn post of your of your own. it was about organizations that switch AI models every couple of weeks. I think that was kind of the beginning of the post was a communication and conversation that you had had similarly. With someone on that. So the question that was included within the comments there was if an organization changes or rotates models every few weeks and only assembles evidence at the next audit to show that its process is operated, are we really certifying the management system in practice or only certifying a description of it?
Walter Haydock: If you are not tracking change control as it's happening or soon thereafter, then you're definitely not having a management system that is describing how you operate. And my recommendation is to again automate as much of this as possible for Stack Aware whenever we detect a new model via the API of a provider like OpenAI. That kicks off a workflow to look into it to see if that's something we want to move the API over to call. The user interface will be automatically upgraded to use that newest model. The ChatGPT user interface will be automatically upgraded. So we actually document that as a risk in our register, is that we can't control the speed of rollout of models, but it's a risk that we necessarily accept. And you know, we have mitigations in place. We could potentially you can roll back to a given model if it's if its successor is not performing very well. And yeah, the key is to track that in real time and and automate it as much as possible. At StackAware, we we have our asset inventory hooked up to Jira so that whenever a new model comes into into play, then we fire a a set of automations that creates tickets to go through and perform the necessary assessments and and then I mentioned we already have that risk that we've accepted of of the user interface just auto-upgrading on its own. But we've documented it and there's nothing we can do about it. So we're gonna we're gonna accept that risk and then the other risks that we identify, then we'll manage them appropriately.
Cameron Gaudet: Yeah. Yeah, and I was I was thinking as you were talking about like accepting that risk. There's also there's also building putting things in place with the anticipation of that specific risk, right? Like to your point of having those having those structures in place, knowing that each of these providers are going to update their model. They're going they're going to continue refining and and sending out, you know, either an update or just a full full scale change. So having things in place, understanding that and making the assumption that that will happen, is wise within the management system there. The the next question here is: how should companies govern third-party AI products when they do not control the training weight, model weights, release schedule, or underlying infrastructure, but are still accountable for how the tool is deployed? So kind of a kind of a foundational, you know, third-party management type of question.
Walter Haydock: If you're using AI models that are hosted by other organizations and are closed source, so you don't have access to the training data, the weights, the infrastructure, things like that, you'll necessarily need to rely on the representations of those vendors. For example, OpenAI and Anthropic, they publish model cards that explain at a high level what these models are trained on, what the data cleaning and processing. Done beforehand was. And oftentimes they have decent information in them that can highlight some key risks. What StackAware does is we use an AI governance standard that says if we're processing this type of data, then we expect this level of granularity of information and this level of risk from a given model. For example, you know, if we can't confirm or deny that the training data is. you know, relatively free of third-party intellectual property claims, then that could potentially trigger a risk. That doesn't mean everything sh grinds to a halt. It just means that this is a risk that we identify because it's fallen below our standard. And we look at that risk, we apply an assessment per ISO 40 2001, and then we decide whether to accept mitigate transfer or avoid that risk. So the approach should be the same from a risk management perspective, whether or not you are hosting the model. The key difference would be how much information you have available if you are not hosting the model and if you're using a closed source model.
Cameron Gaudet: Yeah. Yeah, that makes sense. And I I know that you've you've stated something in in the past that stuck with me, which is everybody at some point in running a business, you have to be able to trust, you know, what what you're given for information. But obviously in this case, you know, there's there could be a difference in how much information you have, but you have to be able to, you know, trust trust those agreements that are put in place. So yeah. It's good good piece of advice there. the next next question's kind of brought up in the news as it were. but as of August 2nd, Article 50 of the EU AI Act made certain AI-facing disclosures and content labeling duties legally applicable, such as requiring disclosures in defined circumstances. So I guess in we can we can touch on the EU AI Act of it all. But More specifically the question that was raised was where do organizations come up short or perhaps a positive spin, have ample room to grow when it comes to aid I AI disclosures and maybe with with the thought in mind of of these new these new requirements being inactive or active.
Walter Haydock: With Article 50 coming into force, we're seeing the technical challenges of watermarking or labeling regimes. And I've long been a skeptic of this type of approach. I think we unfortunately have it backwards here, where the goal should be to label and tag and certify authentic content through digital signatures at the hardware level. rather than flagging AI generated content because you know it gets a little tricky. you know, the EU AI Act has definitions, but it's a little bit it's a little bit blurry and we're still missing all of the harmonized standards on on the EU AI Act that led to the pushback of the high risk AI deadlines to the end of 2027. So there is a lot of gray area. We're seeing technical challenges. I know there were some issues with anthropics. Content labeling approach, the C2PA provenance framework is, you know, it makes sense. But of course, if you change one pixel in an image that's tagged with C2PA, then you remove that that tracker from it, and you can't prove that it was generated with AI. Even unintentionally, for example, if I generate something using Chat GPT, and then I put the stackware logo on it, it's not going to have the C2PA in there. And I would need to figure out some sort of alternative way to mark the document as containing AI generated content. So I don't love the policy approach, and companies are are doing their best to meet the requirements, but I don't see this as being a sustainable practice in the future. And then And then also from a public policy perspective, I think this is causing people to think the wrong way in that, you know, I shouldn't look for a a marker of inauthenticity. I should look for a marker of authenticity.
Cameron Gaudet: Yeah. Yeah, it makes sense. I mean that it may seem it may seem like nuance, but that really changes the mindset the way that the way that you're approaching it of of if you're looking for you're looking for something authentic versus you know, inauthentic or unauthentic. So makes makes a lot of sense for that. And and I echo with the with the EU AI Act that it's there's still a lot of this, you know I guess mi nebulous might be too harsh of a word, but there's there's just still a lot of moving Hearts for it. So but but good question and appreciate appreciate that question for for those that that might be thinking of of kind of seeing how this how this sorts out. So all right, another question that had come in was say you are an AI governance lead. if executives come to you and say they want transparent or trustworthy AI, what questions do you ask to clarify in how to perceive? Or or maybe More so like what would make you skeptical of an organization claiming that it has responsible, safe, or trustworthy AI? Maybe maybe some of the the defining characteristics of those terms.
Walter Haydock: For an organization that's looking to demonstrate transparent or trustworthy AI development or use, I would drill down into exactly what the root of the requirement is. And oftentimes it would be driven by a customer, and in that case, not necessarily be perfectly clear. So there are
Cameron Gaudet: Sure.
Walter Haydock: companies, a lot of companies out there, including very big ones, that. Don't a hundred percent know exactly what they want from AI governance, risk management, things like that. And they will throw a bucket of terminology and contractual requirements at the wall, slash their vendors, and see what sticks. And that can create a challenging environment for both parties because the vendors will try to meet the requirements of the customers, but the customers don't fully know what they want or. Some things they say they want, they might not actually care about, or they don't actually want. So, you know, to tackle this type of situation, what Stack Aware has done recently is we drafted an AI and security addendum that can be added to contractual agreements. And what it does is it maps to existing standards like ISO 42001, ISO 27001, and SOC 2, and says if you cover these criteria, the common criteria, trust services criteria in SOC two, or the NXA controls for the ISO standards, then you get credit under these certain areas of the contractual addenda, addendum for, you know, for example, intended use or for human oversight or for data provenance. And that allows you to get a lot more tactical and specific when it comes to trustworthy, safe, responsible AI.
Cameron Gaudet: Sure. Yeah, yeah, it makes makes a lot of sense. And and I appreciated that question specifically for this for these types of discussions, 'cause as I I feel like I say at the start of every video for for aware AI brief, you know, we're looking for we wanna we wanna talk about AI governance in a very practical, tangible way, right? And and oftentimes to your point, these terminologies that are thrown around are the leading cause of Of non-practical discussions on AI is is really getting down to the the what do you mean what do you mean by that, right? it reminds me reminds me a lot of having small children as I do, which is half the time I spend, you know, they're learning to talk. So half the time I spend figuring out what they're trying to say, and then even from that trying to figure out what that means, right? So like it's it's a multi-layered of of communicating properly and making sure that we're on this. on the same page with terms, so totally totally makes sense. yeah. For sure.
Walter Haydock: I get asked why a lot. So I definitely definitely it's definitely good practice for customer due diligence when you get asked why a lot. And you need to go
Cameron Gaudet: It definitely is. Yeah.
Walter Haydock: down to the kind of quantum particle level explanation
Cameron Gaudet: Ha ha ha.
Walter Haydock: to to to satisfy the the asker. So definitely sympathetic
Cameron Gaudet: Right. Right.
Walter Haydock: there.
Cameron Gaudet: Yeah. Yeah, yeah, for sure. The quantum part. You're absolutely right. It gets it gets real granular. It gets real granular. so we've kind of we've done this in the past. I'm I'm gonna do just kind of throw out some different terms or different topics and in in lieu of of some, you know, football, college football, NFL kicking off, I'm re I'm reminded of my one of my favorite sports media tropes, which is over under properly rated. So I'm gonna pull that back for a second time here and and just throw out a couple couple different topics, discussions that I see happening online, and get your take on over, under, or properly rated for these terms or discussion points. So first up in that category would be prompt injection as the leading enterprise AI security risk.
Walter Haydock: I would say prompt injection is appropriately described as the leading enterprise AI security risk. And that's because with the explosion in agentic AI that has connectivity to lots of different systems and can impact data confidentiality, integrity, and availability, having a malformed or malicious prompt that sends an agent off to do something unintended could Have pretty substantial consequences. And prompt injection is one of those problems that is unfortunately never completely solvable. With SQL injection, because you have deterministic input and structured rules, structured query language, then you can be sure at a certain point that you've checked all your boxes. But that's not true with prompt injection. And when you have million token inputs, There's always something in there that can throw the model for a loop. So I would say prompt injection remains the leading, if not a leading, enterprise security risk.
Cameron Gaudet: deal good deal all right next next one for these these are human in the loop over under properly rated for your take
Walter Haydock: Human in loop is officially overrated at this point. And I've got some data to back that up. Anthropic recently published a study showing that humans caught significantly fewer security and safety issues than Claude's auto review capability. And as a result, they made auto approval of agentic actions in Claude for certain plans and new conversations or new tasks. Made that the default because humans are unfortunately not as good at identifying certain threats. Now you can question the data or the study, but I think that as AI gets better, humans are going to be relatively worse in governing them and telling them what's a correct output, what's an incorrect output. And where humans will add the most value will be saying. This type of thing is a correct output. And here are the parameters, here are the goals we're trying to achieve, but AI will need to figure out how to achieve those parameters and those goals. So if you force a human to review every action, increasingly that's gonna lead to a worse outcome.
Cameron Gaudet: Yeah. Yeah, I'd I'd seen that I I think possibly from yourself or or perhaps others, the the statistics on that you said I think you said anthropic had had listed for that for that measurement with human loop made me think of I don't know if you're a big fan of the office, the American version, but there's a there's a part where they go online and the a website starts taking sales calls and Dwight is trying to trying to beat the website, right? And he's like hanging with them through about five, ten, twelve. And then once it gets past twenty, it just obviously exponentially out does outdoes the person making the sales calls. And and when I saw that I I kept waiting for somebody to post that with like a with a attached meme of of that office scene. Cause as someone that's done human in the loop activities, like absolutely makes sense that it's just it reaches a point where I think you had listed in a in a post about talking about those edge cases and and that being that being where the real value is for for human input and human review. So yeah, good good point.
Walter Haydock: And at Stack Aware, you know, you alluded to the human review that you've done. Where where we're going is humans at the beginning reviewed everything and then reviewed a sample of things and then reviewed questionable cases, then reviewed, you know, a a portion of those questionable cases. And then all along we're we're kind of tweaking the machine, we're refining the machine
Cameron Gaudet: Mm-hmm.
Walter Haydock: to make the machine better at making those types of decisions. But I don't think we'll pull a human entirely out of the loop, we'll have sampling to make sure that we're within our risk tolerances. And as part of our ISO 40 2001 AI management system, we have set thresholds for what's acceptable level of accuracy and what's not. And we'll continue to monitor for that. But as AI gets better, then we may need to sample less often and and you know with less intensity.
Cameron Gaudet: Yeah. Yeah. And and under and like you said, it's the it's the framing of it and and teaching it what to do, maybe more so than just evaluating every single every single output. So next one in this category, we'll get through a couple more here before close, but AI red teaming as proof of governance maturity.
Walter Haydock: AI red teaming, excuse me. AI red teaming is a key part of any risk management program. And you definitely want to continually test your AI systems that are in production and prior to putting them into production. I will say that sometimes organizations jump to that without having architectural reviews and kind of basic security measures in place. And Red teaming identifies some serious architectural flaws. And in that case, it will reveal a lack of maturity rather than proving maturity. So I would say if any organization is considering AI red teaming, that's something that Stack Aware does. Happy to talk to you about it. But first, check out our AI risk readiness kit, which we put together with Sentry Security that highlights the top five. issues that we find during red teaming exercises and that will lay out the things to watch out for before you go into a red team exercise. So you can come out and you can thoroughly say, yes, this is proof of maturity since we have had a red team exercise and no substantial findings.
Cameron Gaudet: Yeah. Yeah, and that's that's an incredible, incredible resource there. So definitely if if if you're someone that that is considering that, looking into that, definitely, definitely check that out. The next one is micro segmentation or neck network segregation for AI agents.
Walter Haydock: I think microsegmentation is appropriately rated and can certainly help in controlling the blast radius of a security event. If you prevent an agent from having access to data that it doesn't need to perform its job, then it's much less likely to negatively impact that data if something goes wrong. So if an agent needs to only write to a database, then Probably doesn't need read capability. It's it's quite possible that it can make blind writes to the database and everything can be fine. And if for some reason an attacker hijacks that agent or it goes rogue, then it's not going to be pulling your entire SQL database out and dumping it into a place where it shouldn't be. So definitely agree that microsub segmentation with AI agents is is a key practice. Being able to achieve it. Potentially more challenging. There's always the engineering/slash IT impulse to just crank all the permissions up to the maximum level every time there's any sort of bug or any sort of error. You know, just just give it read-write all across the entire system. And,
Cameron Gaudet: Yeah.
Walter Haydock: you know, and then we'll we'll fix it later. And then it kind of never gets fixed. So that
Cameron Gaudet: Yeah.
Walter Haydock: would be a cautionary point there. And but I would say micro segmentation is Security.
Cameron Gaudet: Sure, sure. All right, last last couple of these for for overly over under properly rated. this one, the ability to see what a AI agent is doing in real time.
Walter Haydock: The ability to see what an AI agent is doing in real time is overrated, in my opinion, because first of all, an AI agent is probably moving faster than a human would be able to react. I mean, there are some cases I know in the chat GPT models, you can steer them mid mid-prompt or mid-response.
Cameron Gaudet: Mm-hmm.
Walter Haydock: So that could potentially be a useful exception there. And then also just the volume of things that are going on. If you use the open AI API and you send it a detailed query, it's gonna check a hundred websites. So there's no way a human can review every single website that it consulted. And, you know, where you want to put the controls would be before the resulting data has an impact. So, you know, at Stack Aware, we use AI agents to check on information that is out there in, you know, in in the wild. And The first time that we look at a website, a human always reviews the outputs from the agent to make sure there's nothing malicious there. and then we use deterministic checks like using hashes to see if anything on the website has changed. In that case, it's easy. You know, if nothing changed, then there's there's no malicious addition, then you can just update your your entry and move on. And then for existing sites that we've documented, the AI agent goes out and it sees if anything's changed. And You know, it's it's relatively unlikely that a formally trusted site has been hijacked and made malicious, certainly possible, but that's where our sampling would come into play to check to see if anything's changed. And but that's all after the fact. And it's done, you know, with some control gates along the way to make sure that no corrupted data is being pushed into production. But it's not done in real time because even at a small scale, it's not possible.
Cameron Gaudet: Yeah. I I always like when people are like, I get to I get to see it, you know, what it's doing and stuff. And I I think out of curiosity, sometimes it's it's fun to see like what moves it's making. To your point to to be able to you know, direct course and things like that sometimes. especially before any sort of any sort of output that would be that would be detrimental, you know, for for that case that you mentioned. But sometimes it's like, yeah, it the way what it's doing is working so much faster than we'd actually be able to see it all in real time anyway. So it feels feels a little bit like Cliff Notes, Cliff Notes version of operation taking place. But all right, last one for over under properly rated. AI agents should be governed like a service account.
Walter Haydock: Governing AI agents like a service count is a terrible idea, in my opinion, and highly overrated. In general, I don't think service counts should exist. The concept is now so heavily embedded into IT practices that I'm not going to change it. But with AI agents, we have the ability, perhaps, to do things the right way. And we're already seeing a whole wave of startups out there that. Say they govern non-human identities, such as artificial intelligence agents. And I've been critical of the term. I would say, let's just treat an agent like any other asset that has a named owner who is accountable for it, who you can go back to and say, why is this thing doing what it did? And either that person gives you a good answer or the person says, I don't know. And in that case,
Cameron Gaudet: Mm-hmm.
Walter Haydock: then that person needs to. decide to accept, mitigate, transfer, or avoid the risks and either apply some controls to rein in the agent to avoid the risk and shut it down, to accept it because they say, hey, the productivity benefits outweigh the risks. But having this amorphous thing or swarm of things out there where no one is named as the owner of it is a recipe for disaster. And I think that's Gotten a lot of organizations into trouble with service accounts. And I would encourage folks governing AI agents to take a different approach.
Cameron Gaudet: Yeah. Yeah. And and and I know that it I know it's a topic I did definitely throw that one in because I knew it's a topic that you've that you've come at hot before, the the NHI stuff. So if you want more information on that, there's definitely definitely ability to check out posts or or even the the deploy securely blog as well, the the stack aware blog. So definitely definitely check that out if you're looking for more information on that. all right, well that's that's the questions that that came in here this month. again, thank you all for sending questions. If you have an AI governance question or a challenge you're facing or just a topic you'd like us to cover in a future episode, feel free to send that. Send a direct message to myself or Walter on LinkedIn. Leave leave it in the comments of posts or on this YouTube on YouTube videos as well. But but thank you to everyone who submitted questions and thank you Walter for taking the time joining in this conversation here today. appreciate it. Appreciate it very much.
