Matthew Chiodi: Ken, thanks for coming on the show. Today I'm joined by Ken Huang an AI security researcher, author, and Cloud Security Alliance leader who sits at the intersection of security, agentic AI, and security architecture. In this episode, we're digging into a question every CISO is either about to face or is already dealing with. When AI agents start taking action across enterprise systems, how do we know who they are, what they're allowed to do,
Ken Huang: Thank you. Yeah. Eventually we made it, right? Thanks for your time. Yeah. Many thanks. Right, right. Yeah.
Matthew Chiodi: We did. This took what? How many months? It took about six months. Well, you are in high demand because the thing everybody wants to talk about, even if I got on a call ⁓ it has nothing to do ⁓ ⁓ of course it does, but not directly. This is what people want to talk about. So I'm really grateful that you've made the time to come on the show because I know you're in high demand. who they're acting for and how to stop them. Ken recently co-authored a CSA paper called, Agentic AI Access Management, A New Approach. And we are going to be talking about that paper today. This is foundational research in my view for a CISO or an identity team that's just looking at how they can put a governance framework in place before AI agents go big.
Ken Huang: Thank you Matt for your persistence, also the flexibility. Thanks.
Matthew Chiodi: So before, before we get into the research and you've done a lot of research, just tell us a little bit like how your background in identity and access management, cloud security, web three, like how did that lead you to and more specifically agent identity?
Ken Huang: So yeah, I always look at the technology advancement from security perspective. was started when I was doing a project at Capital One, long time ago, with That ⁓ was management system. We did a next generation.
Matthew Chiodi: from a production perspective. Let's jump into it.
Ken Huang: collection system for them. it's early days with WASP, so I contribute some of this with the validation ⁓ things. at ⁓ one of tests, find the injection where before like ⁓ everyone about that, it interested when Claude start. ⁓ identity become one of the most important thing to take a look. So actually in 2011 I think. I have a presentation about we're still up in the air with Claude. I think people, if you search Ken Huang identity Claude may be able to find my slide. ⁓ So yes, identity, that time like there's Foundations, right? The Federation, Simon coming up. ⁓ There's some Claude.
Matthew Chiodi: Okay. Okay.
Ken Huang: the initiatives or also tools, think. So don't quote me on exact date. yeah, this is how I get interested in the identity space. So your question is, yeah, so recently, the AI, agent AI identity. So we look at the same, they are more for the web area for people, Maybe also for the APIs, the service account. Not really for the agent identity multiple agent system that you need the sum of delegation, sum of dynamics in the identity management. That's why we rotate
Matthew Chiodi: Hmm. Okay.
Ken Huang: research paper, first published in archive, also we go through the formal review process ⁓ cloud security alliance and publish as a white paper.
Matthew Chiodi: I, if I play that back, so, you know, you've been doing this for awhile. You mentioned 2011. So this isn't something that ⁓ you started, in the last year or two. So you've been at this for awhile. ⁓ you mentioned the OWASP foundation, something that ⁓ you had done there. One of the researchers, you know, found the SQL injection right before people were talking about. So you've been at this for awhile. Now you used a couple of the terms that I think some folks may not be as. ⁓ familiar with. talked about MAS, right? The multi-agentic system. Just maybe 30 seconds. What do you mean by a multi-agentic system?
Ken Huang: Yeah, so it's multiple agents. They could be in different kind of architectural pattern. Maybe you have the hierarchical, have a super agent or the manager agent, the work for sub agent. Or maybe there is a swarm of agents communicating with each other. ⁓ the key catalyst is it has a certain level of autonomy. It may use the tools. It's goal oriented like
Matthew Chiodi: Hmm.
Ken Huang: give a task and trying to accomplish your task, right? It may inherit some different set of privileges from the people's identity and also from the service account. And is an indeterminism inside it. That's why we have enlarged the security concerns or attack surface we are not careful. That's why we need a threat model.
Matthew Chiodi: So multi-agent system is something basically, it's, it's agents more than one, multiple agents that are working together. It's, it's as simple as that, but it sounds simple, but it's not simple. Right? So let's, let's look at this from an identity and access management perspective. Um, as I started out the show, right? One of the things I get 90 % of calls, people want to talk about non-human identity. They want to talk about I am with agents. Um, Most CISOs that I talk to, they are being asked to approve AI agents. Let's just back all the way up. What's the core identity problem that they need to understand first? Because they're probably thinking about it from a traditional IAM perspective. What's the core identity problem that most maybe cybersecurity teams need to understand first?
Ken Huang: So in the AI space, at point, I think the core is really depends on what exactly the task you need to give to agent. What kind of level of autonomy you will give to it. What is his context of memory management? And what is runtime? control or intervention that you can do with this and eventually how to fit into the existing platform, how to communicate with other agent. So I call it the AI call. means what kind of autonomy you can use. I ⁓ really intent for this what it can do, right? What tool it can use. And C is context, right? What context it can use. Like if we forget some security rule in the middle, it can delete your email. We do have this use case, right? Yeah, Yeah, it's because for compaction, it's compact as a rule that you should never delete my email. And that through compaction, it's important to security rule, discarded, right? ⁓ context management is important.
Matthew Chiodi: Hmm. We saw that and we saw that a couple of weeks ago, right?
Ken Huang: And then observability, right? So AI call, C-O, right? Observability need to be built in for this identity, what it does, right? So you basically have a trace, you can use in LangSmith or whatever framework you use to trace it. And you have a runtime. Basically you can have an intervention, you can steer it. If you can say, please do not delete my email, you cannot steal it the middle, then it's a problem. So why I think it's most recent codecs as well the cloud code. So it's always a coding agent, I think, ⁓ the pack in of ⁓ innovation. So codecs the middle, you can just write it, steal. So it can steal it towards what you need. with cloud code, you can always say, by the way, like BTW, by the way, this is what you need to consider. ⁓ it's also kind of like a runtime intervention there. I think that's also needed when you're trying to build the system ⁓ ⁓ really the identity in place, Then COR, this And E really... the ecosystems, the evaluation. You have to build the evaluation benchmark or repeatable like, right, to see if it has regression on your whatever or whatever the output, whatever schema you to the model. So ⁓ evaluation is and the ecosystem, how it's fitting to your existing environment. how it can do the A2A calls or MCP, how to use MCP tool. All this, ⁓ think, ⁓ to be thinking like a security consideration needed, both from agent ecosystem and also from the identity perspective, right? This a security consideration, but in order for you to really find what vulnerability you can have, do believe that you can use Maestro framework. Many people use it. Right, right. Yeah, we can, ⁓ we have time, we can dive deeper. But if we don't have time, people can use the tool called maestro-setino.com. It's free. Mostly free, so people can use it to with the tool. Yeah.
Matthew Chiodi: Maestro, the Maestro framework. OK. All right. I will link to that the show notes. ⁓ part of the conversation that I often have is every organization, if it's an enterprise of any scale, they already have all these different identity and access management systems in place. They've got ⁓ for single sign-on. ⁓ got something for identity governance, maybe like a sale point or a saviant, something like that in their environments. Now in the paper, which is called, this came out last year, it's called agentic AI, IAM, a new approach. You call out in there and you and a couple of the other authors call out six where IAM falls short for multi-agent systems. You don't have time to go through all six, but some of them, let's maybe just double click on one is it listed course and static permissions. So predefined roles, the way we do things today are often too broad for AI agent needs leading to over-privileging and operational bottlenecks. Talk a little bit around that because most of the I've spoken with, ⁓ are to fit agents into that existing role-based model. Where does that go wrong?
Ken Huang: Yeah, so you look at the same or author. In most cases, you define the scope during deployment time. You can say like you have access to my Google Drive. You cannot really say you can have access to my Google Drive for this particular file for how much duration during deployment time. Right? You configure it like, like, you configure your ⁓ client application with OAuth, this is what gives the scope, right? So it's a cost-reducing scope. It's static. is a certain new approach to say you have a long time to register, you cannot discover the client metadata to it a little bit. But lots of is patchwork, attribute-based access control using CDAP policy. So it's not defined at the protocol level. So that's why we ⁓ said maybe we need extend. the old author or maybe a new approach. I was invited the National Science Foundations and also ⁓ National Science they have a joint workshop on the agent AI So one of the topic is certainly ⁓ about the agent identity people from Microsoft, Nvidia, OpenID, discussed about that. And I think we're in agreement. that we have not have a standard definition yet. What is agent identity? And also I was with ITU as well, about the potential we can define it. The recent Science Institute was AI-AWASP, AI exchange, was also thinking about if we can define the identity, that would be good. ⁓ But I it's still sometime before we actually
Matthew Chiodi: Hmm.
Ken Huang: it so it's ⁓ ⁓ you say double-click it's how we can dynamically define the scope based on the intent So it's intent-based access control, right? This is actually one of my sub stack article talking about. I think it will be interesting, maybe a little bit of double-click on this is currently what we approach we do is we both overprivileged and underprivileged, right? It's for certain task like to give you the whole HR, for example, human resource privilege to this particular agent, then it really is too over-privileged. The scope is too big. But when it's trying to do something to accomplish the different tasks, the same task, but maybe need different privileges from another human identity, then it's this. So it's also underprivileged, right? That's why intent-based access control with a task scope is important. just a high level, right? What I think is you to constrain the domain with an ontology to... ⁓ understanding the intent because otherwise the intent is really too broad. can say an apple, the apple can be a fruit or apple, iPhone, or maybe a big city. Right. But yeah. So if you just constrain to, okay, I'm talking about this in the mobile phone, right. So apple is iPhone. So that's maybe more constrained. It's similar in the identity or in your business domain.
Matthew Chiodi: That's right.
Ken Huang: you can constrain it to your business domain. You build an ontology and whenever certain words are coming, it will bind into the ontology to understand the intent. Then you can translate the intent through the policy, A CIDA policy is commonly used for that. then you can enforce it. And that's why I actually build a proof of concept. tool in the GitHub. didn't advise it yet, but it's there. People can take a look. basically if you use the open cloud code or client ⁓ tool, you can register against my agent control as one of the agent. And that one intent-based access control in it. ⁓ So you ⁓
Matthew Chiodi: ⁓ interesting.
Ken Huang: like people from HR may not have access to GitHub, right? And your developer may have access to GitHub, but not to other database, some production database. So is sample CIDA policy for the different personality in it. So whenever you run this code, you're using cloud code, right? You do not directly... goes through, it has to say, agent control launch or run this particular tool, a cloud code, then cloud code will always ask the deterministic control layer to do the work. So I think it will be interesting. I just put it there. Just I think only a few stars, I don't really.
Matthew Chiodi: Okay. Well, I will, I will, I will link to that in the show notes. So me play that back to you. Cause there was a lot that you, you, you packaged in there. So we were talking about when you look at how traditional ⁓ works, how it ⁓ with multi-agent systems, ⁓ were talking about specifically OAuth, right? And OAuth and how right now we either, if you look at it, it's very, it's very, not very granular in terms of how you give access, You said oftentimes it's either over or underprivileged. And the way, you look at the current OAuth scope today, there's nothing in there that would allow it to give what you called intent or task-based scope. So I think the example that you give in the paper is like, ⁓ when if you have Claude and you want to give it access to your Google Drive, it's, you know, It's not very granular in terms of it. can say, yes, it's basically read only or full access. That's about as good as you get. Right. And so the point there is, is that those are with existing scopes around OAuth. So it sounds like there needs to be some evolution, but changes to those types of standards, they take time. They take a lot of time for that to happen. Let me ask you this. You know, many companies already have some
Ken Huang: Good night. Yeah.
Matthew Chiodi: non-human identity as part of their identity and access management programs, right? NHI. We talked about this a little bit, but why ⁓ an AI agent call it different from a service account, an API key, or a workload identity? Because again, I think I often see people saying, you know, I already have a context for NHI. I'm just going to put my agents in that. So some of this, think is related to what we were talking about, but maybe just give us a little bit more detail around why someone can't just treat or not that they can't, but maybe they shouldn't treat an agent like just another non-human identity, like a, like a service account. Where does that break?
Ken Huang: Right, so just take example service account for the data access or for database, right? If you give the agent this service account, it's again, it's overprivileged underprivileged, both, right? So that depends on your task again, right? Go back. Your task is to read the example I always quote is the resume database to find the best candidate. So it's really should read the resume database and although it has the service account maybe have the read write access it only can read and if the agent has a certain level of autonomy there's some indirect prompt injection in the resume database when it's read resume, it's actually do the update or it's something else, ⁓ in the table. That's because of the ⁓ two things. One is potential indirect prompt injection coming from all different source from ⁓ the... resume itself or from what it can search, right? Go back to lethal traffic that you're forward talking about, right? And also internal model hallucination also possible with a model misunderstood intent can delete the whole database. That happens as well. ⁓ ⁓ just give a service account. It's a not sufficient. You have to really very well scope it. So that's why coming back to a dynamic policy control always need a thin layer, always real time you can check it. Right? So agent before it do a ⁓ critical or high task, it needed to check. So there's balance. So you, you don't have. don't have to always check. Like if you just do scanning the resume database to find the best candidate, you don't have to check. But if we say it need to delete it, it has to go through it, right? So this is how the intervention is really important in the runtime intervention.
Matthew Chiodi: Hmm. So in the short term, say you were talking about the, the, were talking about like, a service account versus a, an agent that's doing something. And if you just treat them just like an aid, just like a, you would a static service account. Again, it's the same thing. It could be under provisioned. could be over provisioned and use the example of like, Hey, I've got a resume. ⁓ it's going through and its main thing is to go and scan and try to find the best resume. That's a static process that normally happens if it's a scripted, right? Just a traditional script. It's very static. But you're saying now when you add a autonomous or semi-autonomous agent to that, you're now opening it up. It still has the same privileges as that static, you know, service account was running, but on top of it, now you have two additional attack factors, right? You've got indirect prompt injection, which could come from the resume itself. And then the second was just model hallucination. Did I get that right? That's not, is that about right? Perfect. Perfect. Perfect. Now the, the second area that you call out where traditional IAM falls short for multi-agent is what you called dynamic trust and inter-agent authentication. Basically the summary was it lacks native support for peer to peer trust establishment between autonomous agents from different domains. So this is, I think.
Ken Huang: Yeah, perfect.
Matthew Chiodi: Many organizations that I speak to today, they already, they either have agents that they have developed that are customer facing, or they have internal agents, which this is also very likely. They have internal agents that carry out some type of task that maybe a knowledge worker had done in the past. This here is saying that with traditional IAM authentication methods, OAuth, SAML, et cetera, there's no way to establish trust boundaries. If for example, organization A has an agent. that needs to interact with organization B's agent. Talk a little bit more about that. What's the, what is the challenge there? And I guess the really bigger question is, is, you know, this isn't theoretical anymore. What, what do organ, what can organizations do today with the frameworks that exist in that, in those, you know, different across organizations, agents needing to talk to each other.
Ken Huang: Right, so we go back to the web tool or whatever the name we give it, which more like a no agent or human in the web, we're using of ⁓ idea of what the federation is. SAMO is the federation, LIBRA federation, WS, web service federation. So you basically, the corporate level, you can federate your identity into this. So you can have business to business transactions. That... framework is not sufficient. I don't say that it's not needed. It's still needed, but it's not sufficient for the agent-to-agent communications. Precisely because of what we call the non-determinism, right? non-deterministic, it's autonomy. So we do need to really think. ⁓ about how we can re-architect that.
Matthew Chiodi: Ken, of areas that you also call out in the paper is this whole concept ⁓ of dynamic trust or I would call inter-agent authentication. And it's the fact that if you look at how ⁓ our existing IAM frameworks, really fall short around this. Talk with a little bit more around just ⁓ why is this a know, can we use existing federation methods? Where does this fall short? for multi-agent systems.
Ken Huang: Right, yeah, so traditional web tool or whatever name we should give it, basically you have the human involved, the Federation for Business to Business, like if you have a partner application, you can use Insightmo or OAuth. In this case, Federation is still needed, precisely because of the and then determinism, the autonomy involved, need to rethink about the agent to agent identity handoff, right? What kind of privileges? cannot really, like agent A, maybe have a privilege to have access to the database ⁓ and to a message queue, ⁓ or even have to the email box. And when it delegate to another agent, it should not really give this another agent ⁓ of different scope, sub-task scope, ⁓ So in case, the delegation is not defined yet. We don't know how to define it. We just call, right? So the idea is you ⁓ only it to within this particular service task to say, okay, this is the budget that you have, like you can give a budget in terms of a token budget, so it will not do the denial of wallet service. this is the particular tool you can use. You can use the full tools, have file access, and have the access to this particular topic in the message queueing, ⁓ and a particular access to this particular database stable. ⁓ or even to the low, then you can ⁓ do access to certain, another file, or Google Java for this particular file. There's only four tool you can use in this delegation, and you to reply to me if your token budget is over million tokens, example, right? And so is the idea that we to have, but it's still up to discussion.
Matthew Chiodi: So if I hear you correctly, you're saying the scenario here that you're talking about is, again, this is all within the context of a multi-agent system. That first agent is doing something that you've given it to it could have access to a database, could be a message queue, could be an email. And then that agent needs to take what it just did and hand it off to another agent or to say, Hey, go do in these, these other things, right? Cause typically agents trained on a fairly narrow set of tasks, right? It's very rare or the agents don't usually work well when you try to give them many, many things to do. So you're talking about here is the fact that like today, our authentication frameworks don't have a good way of doing that delegated or what you called subtask scope or undefined delegation. So this is just an area today where we're using OAuth grants, things like that, that again are very coarse. They're very static and they don't account for types. ⁓ of granularity. Let me ask you this. Standards like this, take time to evolve. What's maybe like one or two things that if someone's listening today and they're thinking, ⁓ wow, like I didn't even realize that there were these challenges and they were trying to put it into the existing frameworks. What's maybe one or two things that a security team can do today to better secure their agents?
Ken Huang: Yeah, so before they do anything, they need to give a job description to the agent. of like, right, ⁓ to people. ⁓ what this agent could do and we can provision to this particular agent, right? ⁓ And in multiple agent system also, that's important say that when it can hand off to other agent, The current paradigm is really you have a manager agent has access to delegate to the task to specialize the agent, like the sub agent. This is the current paradigm, but you will see in the future, you may have a swarm of agent that can interact with each other. That will happen, then it will become more complicated, right? But simplify little bit, is hierarchical agent. You have the manager agent or supervisor agent hand off to some sub-task agent. So you need to give job description of supervisor agent, it can do. It should do just the planning, it maybe should not have access to everything. Right, it just does the planning and maybe has a good context window. It has access to the best model possible to do best planning. And then when it's a go to the subagent, maybe you have a subagent specialized if ⁓ talking about in the cybersecurity context, you have the triage agent, right? It's just to do the triage. It's ⁓ okay, it's first positive or not. What is my confidence score, right? then it has only have access to CVE database, maybe internal, right? ⁓ also to the incident, see, okay, what is ⁓ source ⁓ address, what the destination, if it's happened ⁓ the past. So has its own kind of privilege, right? Then you have investigation. agent, maybe it's actually investigated. If it happened, it has different privilege. Maybe it has access to sandbox, a sandbox ⁓ VM replicate, right? So yeah, I would say first, give a job description. Second, is do not really put it in the immediately. Maybe you turn it on 10%. Like you have an onboarding process, right? To tune it a little bit. So you do need some kind of policy and the gathering to you are able to gradually relax based on its behavior. It's ⁓ true kind of because of ⁓ it's non-determinism, right? It's like people, person, right? ⁓ And has autonomy. So treat it as this way. So this is one approach is really trying to do ⁓ this ⁓ a traditional way, but I still put the gallery. Another way is really what I have advocated before, it's Intent-based Access Control. So you may put it more dynamic. So ⁓ have ontology, you define the different profile, and then agent need to always check. So this is a non-negotiations check with this dynamic policy, So that, sometimes the agent can hallucinate and also ⁓ back to like ⁓ may violate its principle to right? ⁓ So is why we really have no solution yet, right? So need to careful on this.
Matthew Chiodi: So let me play this back. Let's see if I got this right. So we were talking about like, are, what are one or two things that a security team can do today to better secure their agents? Cause we said a lot of it is just the fundamentals of the authentication protocols themselves. They're just not set up for, for autonomous agents. They're set up for humans, which work at relatively slow speeds and do, do things in a way that's, ⁓ much more deterministic. So you said number one was think about your agents. In terms of what is their job description, right? What do they, what do they do? so part of this goes back to actually, it's just the architecture of the agents, right? Like it's focusing, and we always say people process and technology. Well, in this case here, it's not people, it's an agent. So we have to focus on that. The process is ⁓ process by which these agents are created. And I think you didn't say this directly, but one of the things that I think about that I think we need to do for. ⁓ for agents that we did for cloud. Remember we always talked about shift left, the concept of, Hey, don't, don't be working in the UIs. If you remember in the early days of cloud, the biggest challenges were misconfigurations. And the reason those misconfigurations existed was because people were doing things. Every cloud environment was like a snowflake. was unique. Wasn't until we started to use things like infrastructure as code where we could inspect
Ken Huang: Mm-hmm.
Matthew Chiodi: what was going to happen before it was instantiated. This sounds like there's maybe some correlation there where today we create agents and there's no recipe for these agents. They're all different. And because of that, we struggle with even just the first thing you said, which is giving it a job description. What does it do? Right? And if you can really narrow the scope what that agent does, then at least using our existing frameworks today, ⁓ we can use least privilege, right? So the concept of least privilege still a hundred percent applies. Did I get that kind of right?
Ken Huang: Yeah, yeah, think perfect. I think what you add is really good in terms of ⁓ play, like play it. Like you're using telephone, right? Before you it, you can play, have a playbook, ⁓ To simulate the environment. ⁓ I yeah, you build some of the simulated environment for the agent to play out. It's good, yeah. I think that's the add-on that you just mentioned. I think it's a good add-on.
Matthew Chiodi: So the third thing you mentioned was having an onboarding or really a QA process for your agents, right? So a lot of times what happens is, you know, someone plays with an agent, maybe a developer, or maybe it's not even a developer. It could be somebody in marketing, HR that has developed something with Claude code, right? And this is becoming increasingly more common. I see this in finance departments. They're developing their own agents. And then all of a sudden it just goes out into production. And that is where challenges go on. So again, I think if we're back here, people process and technology or agents process and technology, we're talking about and defining the process by which agents get released into production. So it sounds like some of this is just really basics of IT of how we would do things in the past. And we think because agents are new that a lot of the processes that we've used in the past. go away. But it sounds like, you know, we're kind of back to those fundamentals right now while we wait for the underlying protocols to catch up.
Ken Huang: Yeah, this is what actually in multiple kind of confluence I emphasize that ⁓ AI risk management framework is still applied. It's really good. It has a govern, map, measure, and manage function as for risk management, right? So govern is really say who really is... kind of like racing model, who is responsible for what in this agent AI deployment, right? ⁓ what is the GRC policy you have? And MAP is really MAP, the agent AI system ⁓ to find out the vulnerabilities. So we're using framework as one of the tool. The lethal traffic is actually operated on the layer two of the... a mixture framework on the data layer. The lethal traffic, meaning that you have access to your internal data. You can also have access to malicious with indirect prompt injection possible, right, the external data. And then you can act ⁓ it or do the actions, right, ⁓ effect changes. then it's a result of factor. So you have to limit it to just two, but that's not enough, So that's why we need the whole, like a similar layer, menstrual framework. This is a map. And then the measure is really what kind of risk we're facing. Is this, we need to immediately, or we can wait, or we can observe for it. And this is what actually wasp AI VSS project that I lead, right? We're trying to say for this particular agent AI risk tool misuse, for example, on this particular environment, should we fix it or should we just wait, Observe it. So for the measure part of it. And the management is really what is ⁓ we... talking about onboarding process, job description, then we need to do red teaming, And what you add is simulation, simulated. It's kind of like red teaming, but it's simulation. It's a TEVV in NIST term is a test evaluation, verification, validation, This process. yeah, so it's the process. It's a full function. I think NIST, framework still applies. The traditional shift to left, all these things still apply. would say they are necessary, but not sufficient. ⁓ always right. Same like model alignment, model adversely training. This is necessary for our security, but not sufficient. Currently, missiles coming out. ⁓ They it. ⁓
Matthew Chiodi: Hmm.
Ken Huang: the model using coding, kind of trying to improve the ⁓ capability. But eventually it has emerging capability doing the cyber security offensive and both offensive and defensive, right? So it's offensive capability is so strong that they do not want to release immediately, right? So ⁓ why we basically need go through the...
Matthew Chiodi: Hmm.
Ken Huang: like what you said, the least privilege still applies shift to left, the traditional security people, process, technology still apply, but it's not sufficient. You need argument your security program in the agent of AI. You cannot say we currently security program will still work, right? It's necessary. foundation layer for deterministic software. You need to add on top of it the deterministic autonomy side of it.
Matthew Chiodi: So I appreciate that because I think a lot of times when there's new technology, people send a, know, the, the sky is falling and, certainly there are serious risks that come with ⁓ IK, AI. We're seeing it every day. ⁓ two things that you called out one was the NIST AI risk management framework, right. which covers those key areas, govern map measure. And manage. that's one that our listeners can, if they haven't read it, they can go back and look at that today. that was something that's very practical. The second one you mentioned is the ⁓ framework that's spelled ⁓ A E S T ⁓ O. And that's a specialized, ⁓ threat modeling approach designed for agentic AI systems, focusing on risk, security, and outcomes across seven layers, which we don't have time to get into today, ⁓ those two frameworks. I think are, will give and take a lot of the anxiety out of that whole agentic conversation. So I appreciate you. You appreciate you bringing those up. Can you, ⁓ you are in a lot of different places. You were doing research for multiple different firms from the cloud security lions to multiple other places. We know you have conversations with Nvidia, given that you are at the center of all things AI, how do you, ⁓ How do you stay sharp? What's your process? How do you stay up to date on the advances that are happening seemingly like on an hourly basis? What's your process look like?
Ken Huang: So my doctrine is always health number one. So you have to keep health so you can keep sharp. Second is you never too late to learn. So it's always a lifelong learning process. Third. is I would say you need the family first, right? Everything family first. So you self is first, the family is also first. The second is the work. So ⁓ this Suli I think is very important. That's what actually I use it, right? So how do I learn? ⁓ My learning is I... read fast, I would say. I also listen to some good podcasts. recently I also write some things so it can play back to me. I kind of text to speech. I to my own writing as well. trying to see what needs to be ⁓ updated or it makes sense when I ⁓ to it, reading to it. this is good. I say I think the best way of ⁓ learning writing about it. That's why I write books. I write a sub stack. I reached subscribers a days ago. So thank you, thank you. Yeah, so it's sub stack ⁓ I started.
Matthew Chiodi: Mm-hmm. Wow, congratulations.
Ken Huang: A year ago, kind of hobby. I write ⁓ in old days, it's called the music, so user net. I forgot what it is. I wrote some things. And then a Google blog coming. I write something about that. In very old days, I actually wrote a very popular article to compare the Java with the small talk. on object-oriented programming. It was popular at time, but I don't think people can find it anymore. writing it, ⁓ helped me to understand the topic more, I would say.
Matthew Chiodi: love that. You know, one of my good friends says that writing is thinking and I couldn't agree more. ⁓ if you can't explain something in writing, then you probably don't understand it that way yourself. And it's a great way to, great way to learn. So Ken, you are, you're impressive. If anyone wants to check out Ken's sub stack, it's Ken Huang, us.substack.com. just signed up myself while we were recording. So now you're maybe you're
Ken Huang: Mm-hmm.
Matthew Chiodi: 100,001 subscriber. Congratulations on that. But thanks Ken for coming on the show. This has been a great conversation.
Ken Huang: Thank you very much.
Ken Huang: Thank you. Yeah. Eventually we made it, right? Thanks for your time. Yeah. Many thanks. Right, right. Yeah.
Matthew Chiodi: We did. This took what? How many months? It took about six months. Well, you are in high demand because the thing everybody wants to talk about, even if I got on a call ⁓ it has nothing to do ⁓ ⁓ of course it does, but not directly. This is what people want to talk about. So I'm really grateful that you've made the time to come on the show because I know you're in high demand. who they're acting for and how to stop them. Ken recently co-authored a CSA paper called, Agentic AI Access Management, A New Approach. And we are going to be talking about that paper today. This is foundational research in my view for a CISO or an identity team that's just looking at how they can put a governance framework in place before AI agents go big.
Ken Huang: Thank you Matt for your persistence, also the flexibility. Thanks.
Matthew Chiodi: So before, before we get into the research and you've done a lot of research, just tell us a little bit like how your background in identity and access management, cloud security, web three, like how did that lead you to and more specifically agent identity?
Ken Huang: So yeah, I always look at the technology advancement from security perspective. was started when I was doing a project at Capital One, long time ago, with That ⁓ was management system. We did a next generation.
Matthew Chiodi: from a production perspective. Let's jump into it.
Ken Huang: collection system for them. it's early days with WASP, so I contribute some of this with the validation ⁓ things. at ⁓ one of tests, find the injection where before like ⁓ everyone about that, it interested when Claude start. ⁓ identity become one of the most important thing to take a look. So actually in 2011 I think. I have a presentation about we're still up in the air with Claude. I think people, if you search Ken Huang identity Claude may be able to find my slide. ⁓ So yes, identity, that time like there's Foundations, right? The Federation, Simon coming up. ⁓ There's some Claude.
Matthew Chiodi: Okay. Okay.
Ken Huang: the initiatives or also tools, think. So don't quote me on exact date. yeah, this is how I get interested in the identity space. So your question is, yeah, so recently, the AI, agent AI identity. So we look at the same, they are more for the web area for people, Maybe also for the APIs, the service account. Not really for the agent identity multiple agent system that you need the sum of delegation, sum of dynamics in the identity management. That's why we rotate
Matthew Chiodi: Hmm. Okay.
Ken Huang: research paper, first published in archive, also we go through the formal review process ⁓ cloud security alliance and publish as a white paper.
Matthew Chiodi: I, if I play that back, so, you know, you've been doing this for awhile. You mentioned 2011. So this isn't something that ⁓ you started, in the last year or two. So you've been at this for awhile. ⁓ you mentioned the OWASP foundation, something that ⁓ you had done there. One of the researchers, you know, found the SQL injection right before people were talking about. So you've been at this for awhile. Now you used a couple of the terms that I think some folks may not be as. ⁓ familiar with. talked about MAS, right? The multi-agentic system. Just maybe 30 seconds. What do you mean by a multi-agentic system?
Ken Huang: Yeah, so it's multiple agents. They could be in different kind of architectural pattern. Maybe you have the hierarchical, have a super agent or the manager agent, the work for sub agent. Or maybe there is a swarm of agents communicating with each other. ⁓ the key catalyst is it has a certain level of autonomy. It may use the tools. It's goal oriented like
Matthew Chiodi: Hmm.
Ken Huang: give a task and trying to accomplish your task, right? It may inherit some different set of privileges from the people's identity and also from the service account. And is an indeterminism inside it. That's why we have enlarged the security concerns or attack surface we are not careful. That's why we need a threat model.
Matthew Chiodi: So multi-agent system is something basically, it's, it's agents more than one, multiple agents that are working together. It's, it's as simple as that, but it sounds simple, but it's not simple. Right? So let's, let's look at this from an identity and access management perspective. Um, as I started out the show, right? One of the things I get 90 % of calls, people want to talk about non-human identity. They want to talk about I am with agents. Um, Most CISOs that I talk to, they are being asked to approve AI agents. Let's just back all the way up. What's the core identity problem that they need to understand first? Because they're probably thinking about it from a traditional IAM perspective. What's the core identity problem that most maybe cybersecurity teams need to understand first?
Ken Huang: So in the AI space, at point, I think the core is really depends on what exactly the task you need to give to agent. What kind of level of autonomy you will give to it. What is his context of memory management? And what is runtime? control or intervention that you can do with this and eventually how to fit into the existing platform, how to communicate with other agent. So I call it the AI call. means what kind of autonomy you can use. I ⁓ really intent for this what it can do, right? What tool it can use. And C is context, right? What context it can use. Like if we forget some security rule in the middle, it can delete your email. We do have this use case, right? Yeah, Yeah, it's because for compaction, it's compact as a rule that you should never delete my email. And that through compaction, it's important to security rule, discarded, right? ⁓ context management is important.
Matthew Chiodi: Hmm. We saw that and we saw that a couple of weeks ago, right?
Ken Huang: And then observability, right? So AI call, C-O, right? Observability need to be built in for this identity, what it does, right? So you basically have a trace, you can use in LangSmith or whatever framework you use to trace it. And you have a runtime. Basically you can have an intervention, you can steer it. If you can say, please do not delete my email, you cannot steal it the middle, then it's a problem. So why I think it's most recent codecs as well the cloud code. So it's always a coding agent, I think, ⁓ the pack in of ⁓ innovation. So codecs the middle, you can just write it, steal. So it can steal it towards what you need. with cloud code, you can always say, by the way, like BTW, by the way, this is what you need to consider. ⁓ it's also kind of like a runtime intervention there. I think that's also needed when you're trying to build the system ⁓ ⁓ really the identity in place, Then COR, this And E really... the ecosystems, the evaluation. You have to build the evaluation benchmark or repeatable like, right, to see if it has regression on your whatever or whatever the output, whatever schema you to the model. So ⁓ evaluation is and the ecosystem, how it's fitting to your existing environment. how it can do the A2A calls or MCP, how to use MCP tool. All this, ⁓ think, ⁓ to be thinking like a security consideration needed, both from agent ecosystem and also from the identity perspective, right? This a security consideration, but in order for you to really find what vulnerability you can have, do believe that you can use Maestro framework. Many people use it. Right, right. Yeah, we can, ⁓ we have time, we can dive deeper. But if we don't have time, people can use the tool called maestro-setino.com. It's free. Mostly free, so people can use it to with the tool. Yeah.
Matthew Chiodi: Maestro, the Maestro framework. OK. All right. I will link to that the show notes. ⁓ part of the conversation that I often have is every organization, if it's an enterprise of any scale, they already have all these different identity and access management systems in place. They've got ⁓ for single sign-on. ⁓ got something for identity governance, maybe like a sale point or a saviant, something like that in their environments. Now in the paper, which is called, this came out last year, it's called agentic AI, IAM, a new approach. You call out in there and you and a couple of the other authors call out six where IAM falls short for multi-agent systems. You don't have time to go through all six, but some of them, let's maybe just double click on one is it listed course and static permissions. So predefined roles, the way we do things today are often too broad for AI agent needs leading to over-privileging and operational bottlenecks. Talk a little bit around that because most of the I've spoken with, ⁓ are to fit agents into that existing role-based model. Where does that go wrong?
Ken Huang: Yeah, so you look at the same or author. In most cases, you define the scope during deployment time. You can say like you have access to my Google Drive. You cannot really say you can have access to my Google Drive for this particular file for how much duration during deployment time. Right? You configure it like, like, you configure your ⁓ client application with OAuth, this is what gives the scope, right? So it's a cost-reducing scope. It's static. is a certain new approach to say you have a long time to register, you cannot discover the client metadata to it a little bit. But lots of is patchwork, attribute-based access control using CDAP policy. So it's not defined at the protocol level. So that's why we ⁓ said maybe we need extend. the old author or maybe a new approach. I was invited the National Science Foundations and also ⁓ National Science they have a joint workshop on the agent AI So one of the topic is certainly ⁓ about the agent identity people from Microsoft, Nvidia, OpenID, discussed about that. And I think we're in agreement. that we have not have a standard definition yet. What is agent identity? And also I was with ITU as well, about the potential we can define it. The recent Science Institute was AI-AWASP, AI exchange, was also thinking about if we can define the identity, that would be good. ⁓ But I it's still sometime before we actually
Matthew Chiodi: Hmm.
Ken Huang: it so it's ⁓ ⁓ you say double-click it's how we can dynamically define the scope based on the intent So it's intent-based access control, right? This is actually one of my sub stack article talking about. I think it will be interesting, maybe a little bit of double-click on this is currently what we approach we do is we both overprivileged and underprivileged, right? It's for certain task like to give you the whole HR, for example, human resource privilege to this particular agent, then it really is too over-privileged. The scope is too big. But when it's trying to do something to accomplish the different tasks, the same task, but maybe need different privileges from another human identity, then it's this. So it's also underprivileged, right? That's why intent-based access control with a task scope is important. just a high level, right? What I think is you to constrain the domain with an ontology to... ⁓ understanding the intent because otherwise the intent is really too broad. can say an apple, the apple can be a fruit or apple, iPhone, or maybe a big city. Right. But yeah. So if you just constrain to, okay, I'm talking about this in the mobile phone, right. So apple is iPhone. So that's maybe more constrained. It's similar in the identity or in your business domain.
Matthew Chiodi: That's right.
Ken Huang: you can constrain it to your business domain. You build an ontology and whenever certain words are coming, it will bind into the ontology to understand the intent. Then you can translate the intent through the policy, A CIDA policy is commonly used for that. then you can enforce it. And that's why I actually build a proof of concept. tool in the GitHub. didn't advise it yet, but it's there. People can take a look. basically if you use the open cloud code or client ⁓ tool, you can register against my agent control as one of the agent. And that one intent-based access control in it. ⁓ So you ⁓
Matthew Chiodi: ⁓ interesting.
Ken Huang: like people from HR may not have access to GitHub, right? And your developer may have access to GitHub, but not to other database, some production database. So is sample CIDA policy for the different personality in it. So whenever you run this code, you're using cloud code, right? You do not directly... goes through, it has to say, agent control launch or run this particular tool, a cloud code, then cloud code will always ask the deterministic control layer to do the work. So I think it will be interesting. I just put it there. Just I think only a few stars, I don't really.
Matthew Chiodi: Okay. Well, I will, I will, I will link to that in the show notes. So me play that back to you. Cause there was a lot that you, you, you packaged in there. So we were talking about when you look at how traditional ⁓ works, how it ⁓ with multi-agent systems, ⁓ were talking about specifically OAuth, right? And OAuth and how right now we either, if you look at it, it's very, it's very, not very granular in terms of how you give access, You said oftentimes it's either over or underprivileged. And the way, you look at the current OAuth scope today, there's nothing in there that would allow it to give what you called intent or task-based scope. So I think the example that you give in the paper is like, ⁓ when if you have Claude and you want to give it access to your Google Drive, it's, you know, It's not very granular in terms of it. can say, yes, it's basically read only or full access. That's about as good as you get. Right. And so the point there is, is that those are with existing scopes around OAuth. So it sounds like there needs to be some evolution, but changes to those types of standards, they take time. They take a lot of time for that to happen. Let me ask you this. You know, many companies already have some
Ken Huang: Good night. Yeah.
Matthew Chiodi: non-human identity as part of their identity and access management programs, right? NHI. We talked about this a little bit, but why ⁓ an AI agent call it different from a service account, an API key, or a workload identity? Because again, I think I often see people saying, you know, I already have a context for NHI. I'm just going to put my agents in that. So some of this, think is related to what we were talking about, but maybe just give us a little bit more detail around why someone can't just treat or not that they can't, but maybe they shouldn't treat an agent like just another non-human identity, like a, like a service account. Where does that break?
Ken Huang: Right, so just take example service account for the data access or for database, right? If you give the agent this service account, it's again, it's overprivileged underprivileged, both, right? So that depends on your task again, right? Go back. Your task is to read the example I always quote is the resume database to find the best candidate. So it's really should read the resume database and although it has the service account maybe have the read write access it only can read and if the agent has a certain level of autonomy there's some indirect prompt injection in the resume database when it's read resume, it's actually do the update or it's something else, ⁓ in the table. That's because of the ⁓ two things. One is potential indirect prompt injection coming from all different source from ⁓ the... resume itself or from what it can search, right? Go back to lethal traffic that you're forward talking about, right? And also internal model hallucination also possible with a model misunderstood intent can delete the whole database. That happens as well. ⁓ ⁓ just give a service account. It's a not sufficient. You have to really very well scope it. So that's why coming back to a dynamic policy control always need a thin layer, always real time you can check it. Right? So agent before it do a ⁓ critical or high task, it needed to check. So there's balance. So you, you don't have. don't have to always check. Like if you just do scanning the resume database to find the best candidate, you don't have to check. But if we say it need to delete it, it has to go through it, right? So this is how the intervention is really important in the runtime intervention.
Matthew Chiodi: Hmm. So in the short term, say you were talking about the, the, were talking about like, a service account versus a, an agent that's doing something. And if you just treat them just like an aid, just like a, you would a static service account. Again, it's the same thing. It could be under provisioned. could be over provisioned and use the example of like, Hey, I've got a resume. ⁓ it's going through and its main thing is to go and scan and try to find the best resume. That's a static process that normally happens if it's a scripted, right? Just a traditional script. It's very static. But you're saying now when you add a autonomous or semi-autonomous agent to that, you're now opening it up. It still has the same privileges as that static, you know, service account was running, but on top of it, now you have two additional attack factors, right? You've got indirect prompt injection, which could come from the resume itself. And then the second was just model hallucination. Did I get that right? That's not, is that about right? Perfect. Perfect. Perfect. Now the, the second area that you call out where traditional IAM falls short for multi-agent is what you called dynamic trust and inter-agent authentication. Basically the summary was it lacks native support for peer to peer trust establishment between autonomous agents from different domains. So this is, I think.
Ken Huang: Yeah, perfect.
Matthew Chiodi: Many organizations that I speak to today, they already, they either have agents that they have developed that are customer facing, or they have internal agents, which this is also very likely. They have internal agents that carry out some type of task that maybe a knowledge worker had done in the past. This here is saying that with traditional IAM authentication methods, OAuth, SAML, et cetera, there's no way to establish trust boundaries. If for example, organization A has an agent. that needs to interact with organization B's agent. Talk a little bit more about that. What's the, what is the challenge there? And I guess the really bigger question is, is, you know, this isn't theoretical anymore. What, what do organ, what can organizations do today with the frameworks that exist in that, in those, you know, different across organizations, agents needing to talk to each other.
Ken Huang: Right, so we go back to the web tool or whatever the name we give it, which more like a no agent or human in the web, we're using of ⁓ idea of what the federation is. SAMO is the federation, LIBRA federation, WS, web service federation. So you basically, the corporate level, you can federate your identity into this. So you can have business to business transactions. That... framework is not sufficient. I don't say that it's not needed. It's still needed, but it's not sufficient for the agent-to-agent communications. Precisely because of what we call the non-determinism, right? non-deterministic, it's autonomy. So we do need to really think. ⁓ about how we can re-architect that.
Matthew Chiodi: Ken, of areas that you also call out in the paper is this whole concept ⁓ of dynamic trust or I would call inter-agent authentication. And it's the fact that if you look at how ⁓ our existing IAM frameworks, really fall short around this. Talk with a little bit more around just ⁓ why is this a know, can we use existing federation methods? Where does this fall short? for multi-agent systems.
Ken Huang: Right, yeah, so traditional web tool or whatever name we should give it, basically you have the human involved, the Federation for Business to Business, like if you have a partner application, you can use Insightmo or OAuth. In this case, Federation is still needed, precisely because of the and then determinism, the autonomy involved, need to rethink about the agent to agent identity handoff, right? What kind of privileges? cannot really, like agent A, maybe have a privilege to have access to the database ⁓ and to a message queue, ⁓ or even have to the email box. And when it delegate to another agent, it should not really give this another agent ⁓ of different scope, sub-task scope, ⁓ So in case, the delegation is not defined yet. We don't know how to define it. We just call, right? So the idea is you ⁓ only it to within this particular service task to say, okay, this is the budget that you have, like you can give a budget in terms of a token budget, so it will not do the denial of wallet service. this is the particular tool you can use. You can use the full tools, have file access, and have the access to this particular topic in the message queueing, ⁓ and a particular access to this particular database stable. ⁓ or even to the low, then you can ⁓ do access to certain, another file, or Google Java for this particular file. There's only four tool you can use in this delegation, and you to reply to me if your token budget is over million tokens, example, right? And so is the idea that we to have, but it's still up to discussion.
Matthew Chiodi: So if I hear you correctly, you're saying the scenario here that you're talking about is, again, this is all within the context of a multi-agent system. That first agent is doing something that you've given it to it could have access to a database, could be a message queue, could be an email. And then that agent needs to take what it just did and hand it off to another agent or to say, Hey, go do in these, these other things, right? Cause typically agents trained on a fairly narrow set of tasks, right? It's very rare or the agents don't usually work well when you try to give them many, many things to do. So you're talking about here is the fact that like today, our authentication frameworks don't have a good way of doing that delegated or what you called subtask scope or undefined delegation. So this is just an area today where we're using OAuth grants, things like that, that again are very coarse. They're very static and they don't account for types. ⁓ of granularity. Let me ask you this. Standards like this, take time to evolve. What's maybe like one or two things that if someone's listening today and they're thinking, ⁓ wow, like I didn't even realize that there were these challenges and they were trying to put it into the existing frameworks. What's maybe one or two things that a security team can do today to better secure their agents?
Ken Huang: Yeah, so before they do anything, they need to give a job description to the agent. of like, right, ⁓ to people. ⁓ what this agent could do and we can provision to this particular agent, right? ⁓ And in multiple agent system also, that's important say that when it can hand off to other agent, The current paradigm is really you have a manager agent has access to delegate to the task to specialize the agent, like the sub agent. This is the current paradigm, but you will see in the future, you may have a swarm of agent that can interact with each other. That will happen, then it will become more complicated, right? But simplify little bit, is hierarchical agent. You have the manager agent or supervisor agent hand off to some sub-task agent. So you need to give job description of supervisor agent, it can do. It should do just the planning, it maybe should not have access to everything. Right, it just does the planning and maybe has a good context window. It has access to the best model possible to do best planning. And then when it's a go to the subagent, maybe you have a subagent specialized if ⁓ talking about in the cybersecurity context, you have the triage agent, right? It's just to do the triage. It's ⁓ okay, it's first positive or not. What is my confidence score, right? then it has only have access to CVE database, maybe internal, right? ⁓ also to the incident, see, okay, what is ⁓ source ⁓ address, what the destination, if it's happened ⁓ the past. So has its own kind of privilege, right? Then you have investigation. agent, maybe it's actually investigated. If it happened, it has different privilege. Maybe it has access to sandbox, a sandbox ⁓ VM replicate, right? So yeah, I would say first, give a job description. Second, is do not really put it in the immediately. Maybe you turn it on 10%. Like you have an onboarding process, right? To tune it a little bit. So you do need some kind of policy and the gathering to you are able to gradually relax based on its behavior. It's ⁓ true kind of because of ⁓ it's non-determinism, right? It's like people, person, right? ⁓ And has autonomy. So treat it as this way. So this is one approach is really trying to do ⁓ this ⁓ a traditional way, but I still put the gallery. Another way is really what I have advocated before, it's Intent-based Access Control. So you may put it more dynamic. So ⁓ have ontology, you define the different profile, and then agent need to always check. So this is a non-negotiations check with this dynamic policy, So that, sometimes the agent can hallucinate and also ⁓ back to like ⁓ may violate its principle to right? ⁓ So is why we really have no solution yet, right? So need to careful on this.
Matthew Chiodi: So let me play this back. Let's see if I got this right. So we were talking about like, are, what are one or two things that a security team can do today to better secure their agents? Cause we said a lot of it is just the fundamentals of the authentication protocols themselves. They're just not set up for, for autonomous agents. They're set up for humans, which work at relatively slow speeds and do, do things in a way that's, ⁓ much more deterministic. So you said number one was think about your agents. In terms of what is their job description, right? What do they, what do they do? so part of this goes back to actually, it's just the architecture of the agents, right? Like it's focusing, and we always say people process and technology. Well, in this case here, it's not people, it's an agent. So we have to focus on that. The process is ⁓ process by which these agents are created. And I think you didn't say this directly, but one of the things that I think about that I think we need to do for. ⁓ for agents that we did for cloud. Remember we always talked about shift left, the concept of, Hey, don't, don't be working in the UIs. If you remember in the early days of cloud, the biggest challenges were misconfigurations. And the reason those misconfigurations existed was because people were doing things. Every cloud environment was like a snowflake. was unique. Wasn't until we started to use things like infrastructure as code where we could inspect
Ken Huang: Mm-hmm.
Matthew Chiodi: what was going to happen before it was instantiated. This sounds like there's maybe some correlation there where today we create agents and there's no recipe for these agents. They're all different. And because of that, we struggle with even just the first thing you said, which is giving it a job description. What does it do? Right? And if you can really narrow the scope what that agent does, then at least using our existing frameworks today, ⁓ we can use least privilege, right? So the concept of least privilege still a hundred percent applies. Did I get that kind of right?
Ken Huang: Yeah, yeah, think perfect. I think what you add is really good in terms of ⁓ play, like play it. Like you're using telephone, right? Before you it, you can play, have a playbook, ⁓ To simulate the environment. ⁓ I yeah, you build some of the simulated environment for the agent to play out. It's good, yeah. I think that's the add-on that you just mentioned. I think it's a good add-on.
Matthew Chiodi: So the third thing you mentioned was having an onboarding or really a QA process for your agents, right? So a lot of times what happens is, you know, someone plays with an agent, maybe a developer, or maybe it's not even a developer. It could be somebody in marketing, HR that has developed something with Claude code, right? And this is becoming increasingly more common. I see this in finance departments. They're developing their own agents. And then all of a sudden it just goes out into production. And that is where challenges go on. So again, I think if we're back here, people process and technology or agents process and technology, we're talking about and defining the process by which agents get released into production. So it sounds like some of this is just really basics of IT of how we would do things in the past. And we think because agents are new that a lot of the processes that we've used in the past. go away. But it sounds like, you know, we're kind of back to those fundamentals right now while we wait for the underlying protocols to catch up.
Ken Huang: Yeah, this is what actually in multiple kind of confluence I emphasize that ⁓ AI risk management framework is still applied. It's really good. It has a govern, map, measure, and manage function as for risk management, right? So govern is really say who really is... kind of like racing model, who is responsible for what in this agent AI deployment, right? ⁓ what is the GRC policy you have? And MAP is really MAP, the agent AI system ⁓ to find out the vulnerabilities. So we're using framework as one of the tool. The lethal traffic is actually operated on the layer two of the... a mixture framework on the data layer. The lethal traffic, meaning that you have access to your internal data. You can also have access to malicious with indirect prompt injection possible, right, the external data. And then you can act ⁓ it or do the actions, right, ⁓ effect changes. then it's a result of factor. So you have to limit it to just two, but that's not enough, So that's why we need the whole, like a similar layer, menstrual framework. This is a map. And then the measure is really what kind of risk we're facing. Is this, we need to immediately, or we can wait, or we can observe for it. And this is what actually wasp AI VSS project that I lead, right? We're trying to say for this particular agent AI risk tool misuse, for example, on this particular environment, should we fix it or should we just wait, Observe it. So for the measure part of it. And the management is really what is ⁓ we... talking about onboarding process, job description, then we need to do red teaming, And what you add is simulation, simulated. It's kind of like red teaming, but it's simulation. It's a TEVV in NIST term is a test evaluation, verification, validation, This process. yeah, so it's the process. It's a full function. I think NIST, framework still applies. The traditional shift to left, all these things still apply. would say they are necessary, but not sufficient. ⁓ always right. Same like model alignment, model adversely training. This is necessary for our security, but not sufficient. Currently, missiles coming out. ⁓ They it. ⁓
Matthew Chiodi: Hmm.
Ken Huang: the model using coding, kind of trying to improve the ⁓ capability. But eventually it has emerging capability doing the cyber security offensive and both offensive and defensive, right? So it's offensive capability is so strong that they do not want to release immediately, right? So ⁓ why we basically need go through the...
Matthew Chiodi: Hmm.
Ken Huang: like what you said, the least privilege still applies shift to left, the traditional security people, process, technology still apply, but it's not sufficient. You need argument your security program in the agent of AI. You cannot say we currently security program will still work, right? It's necessary. foundation layer for deterministic software. You need to add on top of it the deterministic autonomy side of it.
Matthew Chiodi: So I appreciate that because I think a lot of times when there's new technology, people send a, know, the, the sky is falling and, certainly there are serious risks that come with ⁓ IK, AI. We're seeing it every day. ⁓ two things that you called out one was the NIST AI risk management framework, right. which covers those key areas, govern map measure. And manage. that's one that our listeners can, if they haven't read it, they can go back and look at that today. that was something that's very practical. The second one you mentioned is the ⁓ framework that's spelled ⁓ A E S T ⁓ O. And that's a specialized, ⁓ threat modeling approach designed for agentic AI systems, focusing on risk, security, and outcomes across seven layers, which we don't have time to get into today, ⁓ those two frameworks. I think are, will give and take a lot of the anxiety out of that whole agentic conversation. So I appreciate you. You appreciate you bringing those up. Can you, ⁓ you are in a lot of different places. You were doing research for multiple different firms from the cloud security lions to multiple other places. We know you have conversations with Nvidia, given that you are at the center of all things AI, how do you, ⁓ How do you stay sharp? What's your process? How do you stay up to date on the advances that are happening seemingly like on an hourly basis? What's your process look like?
Ken Huang: So my doctrine is always health number one. So you have to keep health so you can keep sharp. Second is you never too late to learn. So it's always a lifelong learning process. Third. is I would say you need the family first, right? Everything family first. So you self is first, the family is also first. The second is the work. So ⁓ this Suli I think is very important. That's what actually I use it, right? So how do I learn? ⁓ My learning is I... read fast, I would say. I also listen to some good podcasts. recently I also write some things so it can play back to me. I kind of text to speech. I to my own writing as well. trying to see what needs to be ⁓ updated or it makes sense when I ⁓ to it, reading to it. this is good. I say I think the best way of ⁓ learning writing about it. That's why I write books. I write a sub stack. I reached subscribers a days ago. So thank you, thank you. Yeah, so it's sub stack ⁓ I started.
Matthew Chiodi: Mm-hmm. Wow, congratulations.
Ken Huang: A year ago, kind of hobby. I write ⁓ in old days, it's called the music, so user net. I forgot what it is. I wrote some things. And then a Google blog coming. I write something about that. In very old days, I actually wrote a very popular article to compare the Java with the small talk. on object-oriented programming. It was popular at time, but I don't think people can find it anymore. writing it, ⁓ helped me to understand the topic more, I would say.
Matthew Chiodi: love that. You know, one of my good friends says that writing is thinking and I couldn't agree more. ⁓ if you can't explain something in writing, then you probably don't understand it that way yourself. And it's a great way to, great way to learn. So Ken, you are, you're impressive. If anyone wants to check out Ken's sub stack, it's Ken Huang, us.substack.com. just signed up myself while we were recording. So now you're maybe you're
Ken Huang: Mm-hmm.
Matthew Chiodi: 100,001 subscriber. Congratulations on that. But thanks Ken for coming on the show. This has been a great conversation.
Ken Huang: Thank you very much.
