TRANSCRIPT 🔗
WEBVTT
00:00:00.320 --> 00:00:08.880
One of my uh engineering leaders, uh, she says, and it's very, very true, doing AI is easy.
00:00:09.839 --> 00:00:13.599
Doing AI well is a lot of hard work.
00:00:13.919 --> 00:00:15.439
Doing AI is easy.
00:00:15.679 --> 00:00:17.440
Doing AI well is hard work.
00:00:17.600 --> 00:00:22.079
That may be the clearest description yet for where enterprise AI stands today.
00:00:22.320 --> 00:00:27.839
Because the tools can already write code, build prototypes, and compress weeks of work into just hours.
00:00:28.000 --> 00:00:32.479
But becoming AI native requires much more than handing employees a coding assistant.
00:00:32.640 --> 00:00:41.679
It changes who can build, how teams organize, how quickly ideas reach production, and how much risk an organization can create before anyone realizes what happened.
00:00:41.920 --> 00:00:57.759
So on today's episode of the AI Proving Ground Podcast, we'll be talking with Raj Chopra, Cisco's Chief Product Officer for Security, and two AI leaders here at WWT, Brian Orbald and Joe Berger, about why when code becomes cheap, the scarce resource is no longer the ability to produce it.
00:00:57.920 --> 00:01:04.560
It's the judgment to decide what should be built, the domain expertise to make it useful, and the governance to keep it all secure.
00:01:04.799 --> 00:01:08.719
We recorded this episode live on the show floor at Cisco Live.
00:01:08.799 --> 00:01:23.439
And throughout the conversation, Raj, Brian, and Joe explore what separates organizations merely using AI from those genuinely changing how they operate, and what happens when autonomous agents can reason, access enterprise systems, and take action on their own.
00:01:23.599 --> 00:01:24.959
So let's get to it.
00:01:48.000 --> 00:01:48.879
Oh my gosh, yeah.
00:01:48.959 --> 00:01:52.879
I would say over the past, yeah, over the past six to 12 months, we've just seen a huge uptick.
00:01:52.959 --> 00:02:05.680
I think if you start looking at a lot of the different use cases of AI that people have developing over the past few years, they really honed in on coding assistance because you were able to see immediate impact within your organization.
00:02:05.920 --> 00:02:10.159
And I think we're at that point now where a lot of companies have adopted these tools.
00:02:10.319 --> 00:02:14.400
Now it's around how does this actually change my organization?
00:02:14.479 --> 00:02:19.840
It's not just I have people using the tools, but how does it change my software development lifecycle?
00:02:19.919 --> 00:02:21.520
How do I start shipping code quicker?
00:02:21.680 --> 00:02:23.520
How do I test for bugs faster?
00:02:23.599 --> 00:02:30.080
I know we're going to get into security talk here in a little bit, but it's really creating this big momentum for the software development team.
00:02:30.159 --> 00:02:39.039
And as well, you're even seeing it make its way into sort of your everyday knowledge worker in terms of low-code tools and now everyone can start developing really quickly.
00:02:39.199 --> 00:02:47.280
So just tremendous opportunity out there that I think a lot of organizations are, they've already been using it or they're kicking the tires on, at least for sure.
00:02:47.520 --> 00:02:53.599
Yeah, and Brian, I mean, he's talking about how it's changing not just the developer teams, it's it's changing the whole course of an organization.
00:02:53.759 --> 00:03:03.520
What are you seeing in terms of implications of AI native engineering and what it means for an organization, not just from a development standpoint, but infrastructure and policy and things like that?
00:03:03.759 --> 00:03:10.080
Uh, you know, first thought that I have before infrastructure and policy is just the impact to Joe's point on the on the workforce.
00:03:10.319 --> 00:03:20.479
So you start seeing the tipping point from AI native engineering into workforce AI, and you've got the the entire workforce has the ability to do things that were never possible for them before.
00:03:20.560 --> 00:03:29.759
And I think that's what ultimately has a consequence on behalf of B infrastructure policy, the way you architect things to Raj's point, like everything's changing.
00:03:29.919 --> 00:03:32.639
The problems of yesterday are not the problems of tomorrow.
00:03:32.800 --> 00:03:35.680
And so we have to solve things in a very different manner now.
00:03:35.840 --> 00:03:43.680
And that is just like we've gone through over the last 25 years, it's another iteration of this requires new ways of thinking about infrastructure design.
00:03:43.840 --> 00:03:53.360
You're you're accounting for ways in which you have to connect systems in a different manner, which ultimately requires how you secure them and in just a new frame of mind, new mindset.
00:03:53.599 --> 00:04:08.240
So I'm I'm excited about the fact that we've got this ability to go create new experiences for our user base, but it's also incumbent upon us to make sure that we are empowering them in a way that's responsible, it has an ability to scale and provides a delightful experience for them.
00:04:08.639 --> 00:04:10.639
Certainly a great segue into Raj.
00:04:10.719 --> 00:04:11.199
Yeah, go ahead.
00:04:11.360 --> 00:04:18.160
I mean, hopefully this is not a hot take anymore, but AI is exceptionally good at doing hard things.
00:04:18.480 --> 00:04:18.720
Right.
00:04:18.959 --> 00:04:25.439
For a long period of time, people have this notion, which is like, okay, we'll use AI for as an assistant or so on and so forth.
00:04:25.759 --> 00:04:30.000
Hopefully, again, one more time, this is not like I'm not blowing anybody's mind off.
00:04:30.399 --> 00:04:33.040
AI is very, very good at doing hard things.
00:04:33.360 --> 00:04:40.720
So the real question comes: do you have people that have agency to really challenge what are the big things to do?
00:04:40.879 --> 00:04:41.199
Right?
00:04:41.439 --> 00:04:46.560
If
Load full transcript
writing code is essentially free, what are the big problems to solve?
00:04:46.800 --> 00:05:04.959
And what we are finding from our developer experience is that people have so many high-quality problems to solve that you should expect not just the professional skill set, but the cultural mindset to shift to people who have a lot of agency.
00:05:05.279 --> 00:05:10.240
I think the distance between the end customer and the developer is gonna shrink.
00:05:10.399 --> 00:05:14.879
The surface areas on which people interact are going to change.
00:05:15.279 --> 00:05:20.800
The way uh vendors like us enable partners like you all is going to change.
00:05:20.959 --> 00:05:25.680
It is not gonna be the usual lecture, and here's the document, and so on and so forth.
00:05:26.000 --> 00:05:30.560
But every aspect when you lay it down, think of the hardest problems.
00:05:30.720 --> 00:05:38.000
Those are the first ones that AI is solving, and that is culturally changing the composition of the team, not just skill-wise.
00:05:38.319 --> 00:05:49.759
Okay, so the tools exist, the capability is there, but the shift from using AI to write code to being the kind of organization that's actually AI native, that's a cultural change as much as a technical one.
00:05:49.839 --> 00:05:51.120
And it doesn't happen by accident.
00:05:51.439 --> 00:06:06.800
Yeah, hey Raj to that point, are you noticing because obviously, as as developers move from sort of writing code to now being the orchestrator of stories and a building, are you seeing that shift within Cisco?
00:06:06.879 --> 00:06:16.639
And is it also is it kind of challenging the mindset of the engineer of like my job yesterday was this, I now have to go think like this, and it's different.
00:06:16.720 --> 00:06:18.319
I mean, there's a shift happening.
00:06:18.639 --> 00:06:23.040
It definitely is, and I think these shifts are those of mindsets, not of titles.
00:06:23.680 --> 00:06:32.560
So most people now in the team should think of themselves as member, member of technical stuff.
00:06:32.800 --> 00:06:33.360
Yeah, right.
00:06:33.439 --> 00:06:41.519
Yeah, it is not like you are a developer, you are an you are not a developer, you are a QA, you are a PM, you're a designer, what have you.
00:06:41.759 --> 00:06:44.480
But it really is you're a member of the technical stuff.
00:06:44.560 --> 00:06:49.519
And that technical stuff gets applied to all sorts of different kinds of problems.
00:06:49.759 --> 00:07:05.199
It truly is where you're thinking about how exactly should we solve the most urgent, the biggest problems, and then have the empowerment to go solve it because the cost of failure of trying something and failing at it is small.
00:07:05.360 --> 00:07:05.680
Oh, yeah.
00:07:05.920 --> 00:07:09.759
So if the cost of failure is small, what else would you try?
00:07:10.000 --> 00:07:11.360
I think that's the mindset.
00:07:11.519 --> 00:07:17.199
So it's I think the org structure is evolving more
organically than you would think.
00:07:17.519 --> 00:07:30.720
It is mission-oriented, it is those who take agency, it's where their job becomes more about collapsing whatever the inefficiencies are rather than being told, go do this, go do that.
00:07:30.959 --> 00:07:37.199
And let me ask you, because I know Cisco's got a very large development organization, what did you all do to enable this?
00:07:37.360 --> 00:07:38.879
Was it training programs?
00:07:38.959 --> 00:07:42.480
Or is it, hey, get the tools in everyone's hand, let them try it out?
00:07:42.720 --> 00:07:45.040
Like how how did you get over this adoption curve?
00:07:45.360 --> 00:07:50.959
I think the most effective way that we did, and a bunch of those things were tried and still are tried.
00:07:51.120 --> 00:07:57.600
I think, first of all, making these tools available for people to experiment and sometimes fail.
00:07:57.759 --> 00:07:59.120
I mean, that's totally okay.
00:07:59.279 --> 00:08:02.240
But really, availability of the tools is very, very important.
00:08:02.720 --> 00:08:10.560
The the two big things I would say is expressing the mission of the company in uncomplicated words.
00:08:10.800 --> 00:08:11.839
Very, very important.
00:08:12.160 --> 00:08:16.879
And then second is driving the lived experience of urgency.
00:08:17.040 --> 00:08:19.120
There is no time to waste.
00:08:19.360 --> 00:08:20.240
Again, one more time.
00:08:20.319 --> 00:08:22.720
This is not about like foster, harder, faster.
00:08:23.120 --> 00:08:30.079
It is more about the fact that if we are not solving the big problems, then we are not being responsible.
00:08:30.319 --> 00:08:30.480
Yeah.
00:08:30.720 --> 00:08:31.839
Brian, what do you think about that?
00:08:31.920 --> 00:08:36.000
I mean, you're you're interfacing with clients every day across a wide variety of industries.
00:08:36.080 --> 00:08:39.360
Is that what many organizations are doing, or is there a learning curve there?
00:08:39.519 --> 00:08:41.519
I think there's a little bit of all, right?
00:08:41.600 --> 00:08:44.159
There's you think your experience is gonna vary.
00:08:44.320 --> 00:08:50.320
Cisco's obviously a leading technology solutions manufacturer provider, developer innovator.
00:08:50.480 --> 00:09:09.039
I think organizations that don't necessarily have that kind of a culture built into the fabric of how they operate, that that may represent, that may manifest itself in a development workforce that may be a little bit more hesitant to lean in and understand that it's mission-oriented as opposed to project-oriented or code-oriented.
00:09:09.200 --> 00:09:20.159
And what we've seen with clients that have really, I think, made it past that that made it around that curve is kind of forcing the issue to have the conversation with your workforce that it is gonna behave differently.
00:09:20.320 --> 00:09:24.399
We're asking you to do things differently, and what you were doing yesterday might be a little bit different tomorrow.
00:09:24.480 --> 00:09:26.480
Your skills are still super valuable.
00:09:26.639 --> 00:09:33.840
We still value all of what you've done for the organization or in your career around software, but we're gonna apply it differently now.
00:09:34.000 --> 00:09:54.080
And so I think being forward with that in areas where maybe the culture isn't as innovation-leaning is important, but I think it's a great question from Joe just around being thoughtful around the cultural shift and the adoption curve matters deeply in this stuff because it's not just about acquiring a tool and giving it to your workforce and hoping great things happen.
00:09:54.240 --> 00:09:56.000
There has to be a purpose and a plan behind it.
00:09:56.080 --> 00:09:59.679
And I think the mission way you say the standard diraj is a great way to frame it.
00:10:00.559 --> 00:10:24.240
If I may, one of the other things, Brian, is what we internally have found is I think the showcasing the failures, yeah, and not in a way like you're calling somebody out, but celebrating that this was tried and this was a learning from it is is very helpful.
00:10:24.559 --> 00:10:33.919
And then the other part that we again live through talk rather showing rather than talking.
00:10:34.559 --> 00:10:34.799
Right.
00:10:35.039 --> 00:10:46.159
So instead of a very elaborate sort of spec or this or that or the other PRD specification, why don't you just give it a lived experience through a prototype?
00:10:46.399 --> 00:10:47.120
Prototype it, yeah.
00:10:47.360 --> 00:10:47.600
Right.
00:10:47.759 --> 00:10:57.360
And I'm telling you, like 10 out of 10 times, once people have used the tool, they feel so much of empowerment.
00:10:57.519 --> 00:10:57.840
Yeah.
00:10:58.080 --> 00:10:58.639
Right.
00:10:59.440 --> 00:11:06.879
One other additional thing, and this goes back to your your skills, are really important, is AI is generic, right?
00:11:07.039 --> 00:11:11.360
And it will give generic answers if you give it generic prompts and generic everything.
00:11:11.679 --> 00:11:17.840
What makes a product really hum and really sing is that domain expertise that needs to come.
00:11:18.080 --> 00:11:28.960
So it is not that a generic AI is gonna solve bisco problems, domain expertise amplified through AI is what solves real world problems.
00:11:29.120 --> 00:11:32.159
So that skill set stuff is very, very important.
00:11:32.399 --> 00:11:34.159
Code is just a means to express it.
00:11:34.320 --> 00:11:34.559
Yeah.
00:11:34.639 --> 00:11:43.519
I I've I've been using kind of a framework of you've got to create a mindset around adoption to build confidence that leads to having, and ultimately you're gonna see change.
00:11:43.679 --> 00:11:44.000
Yep.
00:11:44.159 --> 00:11:46.879
And you know, it's it's being deliberate about those things.
00:11:46.960 --> 00:11:49.919
Again, not just hoping for the best because you've provided a tool.
00:11:50.000 --> 00:11:53.200
We've given tools
to people many, many times with mixed results.
00:11:53.279 --> 00:11:57.679
And I think in this case, we're being at least internal to worldwide, but very, very deliberate.
00:11:57.759 --> 00:12:03.759
It sounds like Cisco is as well, but you're you're focusing on the how and the why, not just the what.
00:12:03.919 --> 00:12:04.639
Yep, absolutely.
00:12:06.080 --> 00:12:11.600
I I would also add, and I think Brian can attest to this, and Raj, I think Cisco's probably in the same boat.
00:12:11.759 --> 00:12:12.720
This starts from the top.
00:12:12.960 --> 00:12:13.759
Top down, 100%.
00:12:14.080 --> 00:12:19.279
And you know, if you know anything about our CEO, Jim Kavanaugh, I mean, he's very passionate around AI.
00:12:19.360 --> 00:12:25.360
And so every Monday on our AI calls throughout the week, he's attending, he's asking the questions.
00:12:25.519 --> 00:12:37.440
But but when we've seen customers who have really dived into AI over the past couple of years, it really is because they've got someone at the top driving it and making sure the success and pushing it down.
00:12:37.600 --> 00:12:43.600
To your point around the mission, it's not just a product set, it's really how do I change the organization with these tools?
00:12:43.759 --> 00:12:45.279
But that's driven from the top down.
00:12:45.360 --> 00:12:46.559
And that's really how you can moment.
00:12:46.879 --> 00:12:47.360
I think sending us.
00:13:17.440 --> 00:13:17.600
Yeah.
00:13:18.240 --> 00:13:24.080
Because we want to see proliferation of our user base uh really taking full advantage of these capabilities.
00:13:24.399 --> 00:13:31.840
One of my engineering leaders, she says, and it's very, very true, doing AI is easy.
00:13:32.799 --> 00:13:35.919
Doing AI well is a lot of hard work.
00:13:36.080 --> 00:13:36.480
Oh, yeah.
00:13:36.559 --> 00:13:43.679
But I I I think it matters to get people off the starting block because when they realize what's possible, then the curiosity kicks in.
00:13:43.840 --> 00:13:52.000
And that's when I think real change can happen, and that unlocks all sorts of new possibilities that we weren't really able to tap into before it weren't even in our vision before.
00:13:52.320 --> 00:13:55.519
So the adoption curve is real, but it cuts both ways.
00:13:55.679 --> 00:14:04.240
As tools proliferate and everyone from developers to business leaders starts building with AI, a new risk surfaces that most organizations aren't ready for.
00:14:04.720 --> 00:14:06.960
This episode is supported by cognition.
00:14:07.200 --> 00:14:20.080
Cognition's AI powered developer tools, windsurf and dev and deliver deep code-based understanding, intelligent code completion, and the autonomous engineer capabilities to help teams embrace the future of software engineering.
00:14:20.480 --> 00:14:22.960
Supercharge your development team with a cognition platform.
00:14:23.279 --> 00:14:32.639
Well, Joe, I mean, you know, Brian's talking about how you know the
unlimited possibilities once it kind of clicks for people, but are we also kind of entering an era where you know we've all heard of shadow AI?
00:14:32.720 --> 00:14:37.840
Are we gonna have shadow engineering where you know Brian's making that that's already out there, Brian?
00:14:38.000 --> 00:14:45.600
If he's writing code now and well, if he's prompting for code, well, that's where kind of Brian talked a little bit about the governance.
00:14:45.679 --> 00:14:57.440
And you know, you hear more and more people talking about the notion of the harness where you've got to make sure your data sets are in place, your governance is there, you've got security behind it because it does start spreading fast.
00:14:57.600 --> 00:15:02.159
If you don't have these other things already in place, that's where you're gonna get yourself into trouble.
00:15:02.320 --> 00:15:04.639
Yeah, and so just having that team.
00:15:04.720 --> 00:15:15.919
I know we talked a little bit about you know the COEs getting formed in a lot of the organizations, just having that attention on these things now before your departments go out and just start buying stuff off the shelf.
00:15:16.080 --> 00:15:23.679
You don't know where your data is going, having that COE and governance now is really gonna be a tremendous asset as this gets bigger.
00:15:23.919 --> 00:15:30.399
Yeah, Raj, I mean, we're talking about code being created faster than ever and tools in everybody's hands.
00:15:30.720 --> 00:15:35.919
What other risks do we have to be concerned about as we enter this era where things are gonna be popping up all the time?
00:15:36.240 --> 00:15:39.600
Yeah, so I think the biggest risk, there are two parts to it.
00:15:39.919 --> 00:15:44.639
One is agents have reasoning capability.
00:15:44.879 --> 00:15:46.720
What does that mean in normal English?
00:15:47.039 --> 00:15:49.440
We used to write, I've written software, right?
00:15:49.600 --> 00:15:50.960
You're writing software.
00:15:52.080 --> 00:15:52.720
Sort of.
00:15:53.519 --> 00:15:55.360
Is we used to write code, right?
00:15:55.440 --> 00:15:57.360
It would be if this happens, do this.
00:15:57.600 --> 00:15:59.519
If then else, so on and so forth, right?
00:15:59.600 --> 00:16:01.200
So there was a codified logic.
00:16:01.360 --> 00:16:06.320
You give the same input ten times, you're gonna get the exact same output, right?
00:16:07.120 --> 00:16:11.840
These agents do not have that kind of methodical thing.
00:16:11.919 --> 00:16:13.360
It is what is called reasoning.
00:16:13.519 --> 00:16:16.159
Depending on the input, I might take a different action.
00:16:16.639 --> 00:16:19.519
So that is number one, the risk part.
00:16:19.759 --> 00:16:23.519
The other thing is agents take action.
00:16:24.559 --> 00:16:29.360
APIs don't take actions by themselves, devices don't take actions by themselves, right?
00:16:29.519 --> 00:16:33.600
There is, those are all in some way waiting for instruction.
00:16:33.840 -->
00:16:35.919
But agents take action.
00:16:36.159 --> 00:16:44.240
And in that world, there are those are the, I would say, from a technical perspective, those are the two biggest risks.
00:16:44.559 --> 00:16:58.159
Because between reasoning and the ability to take action, what really becomes important is yeah, accessing that Salesforce.com or your Jira instance or whatever, that may be perfectly okay, right?
00:16:58.320 --> 00:16:59.440
For an agent.
00:16:59.679 --> 00:17:05.200
But the intent behind that access becomes very important, a very simplistic example.
00:17:05.359 --> 00:17:15.680
It's one thing to say, I'm asking my agent to have access to email so it can summarize this email that Brian has sent, because Brian sent me very long emails.
00:17:16.480 --> 00:17:17.680
He's excited about the software.
00:17:19.440 --> 00:17:23.279
So it it's one thing to say, I'm gonna summarize an email.
00:17:23.519 --> 00:17:29.039
But that same access, if you're scraping my entire mailbox, very different intent.
00:17:29.440 --> 00:17:40.000
So codifying that reasoning along with the ability to take action action and putting it behind an intent, that is the key.
00:17:40.160 --> 00:18:03.519
Now, there is no protocol for intent, there is no spec that is out there in the internet, but that is what we as an industry and we as Cisco are trying very hard to distill intent into both visibility as well or observability as well as policy so that you can match up the intent of the agent because these are reasoning and they take action.
00:18:03.759 --> 00:18:08.160
So, how do you allow looking at one email to summarize?
00:18:08.400 --> 00:18:09.119
Perfect.
00:18:09.359 --> 00:18:17.920
Looking at my entire mailbox, same function, but calling my entire mailbox to scrape every single email, probably not okay.
00:18:18.319 --> 00:18:19.519
How do you see, Raj?
00:18:19.680 --> 00:18:20.160
Stick with you.
00:18:20.240 --> 00:18:25.039
How do you see identity and access evolving as as AI becomes more autonomous?
00:18:25.200 --> 00:18:34.960
I mean, you're the way you would talk about it previously would be for a person or for team, yeah, or for an application that didn't have agentic reasoning.
00:18:35.119 --> 00:18:37.680
So how is it shifting over time and where is it going?
00:18:37.920 --> 00:18:39.519
Yeah, so two-step process.
00:18:39.599 --> 00:18:41.200
One is authentication.
00:18:41.440 --> 00:18:48.240
Authentication, which is just giving an agent an identity, just like back in the old days, right?
00:18:48.400 --> 00:18:56.880
No, you wouldn't let somebody come in, plug a device into the wall socket, right, whatever, and say, now you are on the internet or you have a DHCP address.
00:18:56.960 --> 00:18:57.119
Yeah.
00:18:57.279 --> 00:18:57.440
Right?
00:18:57.519 --> 00:18:58.079
We wouldn't do that.
00:18:58.160 --> 00:18:58.319
Yeah.
00:18:58.480 --> 00:19:00.480
We do a posture check and all that kind of stuff.
00:19:00.720 --> 00:19:08.720
So when somebody brings an agent, before you give it an identity, you've got to make sure that this agent is worthy to be on your environment.
00:19:08.960 --> 00:19:11.279
Because this agent is going to take action all around.
00:19:11.440 --> 00:19:12.400
So how do you do that?
00:19:12.559 --> 00:19:20.960
You basically posture check that agent, the skills that it has, does it uh have access to MCP servers that might be corrupted, etc., etc.
00:19:21.279 --> 00:19:24.240
So you're basically checking the posture of that device.
00:19:25.279 --> 00:19:29.039
Once you've given it an identity, you want to classify these, right?
00:19:29.119 --> 00:19:32.799
I mean, they're what, I don't know, 100 billion URLs on the internet.
00:19:33.279 --> 00:19:35.599
We don't manage these one URL at a time.
00:19:35.839 --> 00:19:39.519
We're not going to manage 10 million agents one agent at a time.
00:19:39.680 --> 00:19:47.039
So you classify and say, this is an HR, this is a this is a benefit, so this is whatever, travel expense, something like that.
00:19:47.279 --> 00:19:49.440
So that you can put appropriate policy.
00:19:49.519 --> 00:19:53.759
So that's step one, which is literally the act of giving it identity.
00:19:54.240 --> 00:19:56.720
The second part is authorization.
00:19:57.279 --> 00:20:02.799
Authorization is this is the scope on which you can access whatever you're trying to access.
00:20:03.279 --> 00:20:19.519
The key part that we this is the lesson that we've learned building agents ourselves, and then how we are helping customers together working with WWT is that authorization needs to be scoped appropriately based on the intent.
00:20:20.799 --> 00:20:21.279
Right?
00:20:22.000 --> 00:20:27.519
Ideal case, and these are the solutions we're building, the agent never holds an authorization token.
00:20:27.759 --> 00:20:34.319
There is an entity, whether it's a sandbox, whether it's a gateway, whatever, something else is holding that authorization.
00:20:34.400 --> 00:20:40.960
So IT and security can determine this is too much, this is too little, this is appropriate, not appropriate.
00:20:41.119 --> 00:20:48.880
But you ideal state, you never want to give the authorization token directly to the agent because it will go hay.
00:20:49.680 --> 00:20:54.400
So Raj, to that point, what happens when the agent isn't within your organization?
00:20:54.559 --> 00:21:01.920
Let's let's say a consumer tool where I'm using my agent on my behalf, it's calling my bank or it's ordering something.
00:21:02.559 --> 00:21:10.720
Yeah, so third-party agents in that authorization becomes even more pronounced and even more important.
00:21:11.759 --> 00:21:15.839
And the way to work this is what is called self-attenuation.
00:21:16.000 --> 00:21:19.359
I don't want to go technical on you guys, but just very briefly.
00:21:19.680 --> 00:21:20.400
Well, you got Brian here.
00:21:20.480 --> 00:21:20.559
Yeah.
00:21:21.519 --> 00:21:22.240
So
maybe for Brian.
00:21:22.480 --> 00:21:24.240
I slept at Holiday in Express last night.
00:21:24.799 --> 00:21:28.880
Very briefly, you it let's say you have a.
00:21:58.240 --> 00:21:59.759
So now it's using an authorized.
00:22:00.400 --> 00:22:03.920
Topen on my behalf to go to Amex.
00:22:04.559 --> 00:22:04.880
Yeah.
00:22:05.039 --> 00:22:05.519
Yeah?
00:22:06.319 --> 00:22:10.240
So every two weeks, expenses get filed.
00:22:10.400 --> 00:22:12.160
Four, six, eight, ten, whatever.
00:22:12.400 --> 00:22:18.799
Six months into it, there is a new credit card that is issued in my name to send to a different address.
00:22:20.319 --> 00:22:20.960
Is it right?
00:22:21.119 --> 00:22:21.680
Is it wrong?
00:22:21.759 --> 00:22:22.559
I don't know.
00:22:22.720 --> 00:22:22.960
Right?
00:22:23.039 --> 00:22:24.319
I might have changed addresses.
00:22:24.400 --> 00:22:25.519
Maybe I lost my wallet.
00:22:25.680 --> 00:22:28.400
Maybe the card expired, whatever the story may be.
00:22:28.799 --> 00:22:31.440
But but this is what goes haywire.
00:22:31.519 --> 00:22:33.279
I don't know whether that card is okay or not.
00:22:33.680 --> 00:22:43.519
But when you take this read-write permission for my scope that I've given in Conquer and take that read-write permission to your Amex, that becomes a problem.
00:22:43.680 --> 00:22:48.799
So you want to trim that down to just read permission when it goes to MX.
00:22:49.519 --> 00:22:49.759
Okay?
00:22:49.920 --> 00:22:59.839
And when MX were to call USPS to look up if this address is really true, you'd trim it down even further to say you only have view capability.
00:23:00.079 --> 00:23:00.480
Right?
00:23:00.799 --> 00:23:05.279
So trimming down the scope is how you keep these things in check.
00:23:05.359 --> 00:23:10.799
Otherwise, they have absolute sort of agency to go in any direction.
00:23:11.599 --> 00:23:16.000
Just another thought on the on the topic of policy and risk.
00:23:16.400 --> 00:23:32.160
Maybe a a different angle on it than the traditional secure, you know, security and compliance side is there was a story this week around an organization that unleashed their enterprise AI tooling to their workforce without caps, without with no metering.
00:23:32.640 --> 00:23:35.200
And they've got a$500 million bill.
00:23:35.440 --> 00:23:35.599
Yeah.
00:23:35.839 --> 00:23:36.400
Is it true?
00:23:37.440 --> 00:23:39.200
That wasn't that was in the news this week, right?
00:23:39.359 --> 00:23:45.839
Now it might be marketing fluff, but it I think what it represents is FinOps starts becoming a very meaningful part of this.
00:23:45.920 --> 00:23:49.759
And a lot of the tools that we're deploying today, they're SaaS.
00:23:50.160 --> 00:23:56.640
In many cases, if you're building your own, you might be deploying that in a NeoCloud and a hyperscaler.
00:23:56.720 --> 00:23:58.240
It's off-prem, it's somewhere else.
00:23:58.480 --> 00:24:01.759
But when it becomes a bill, it becomes very real.
00:24:02.240 --> 00:24:28.799
And to your point, whether it's assigning an identity or a policy to that identity for an agent or to the actual workforce at large, the user base, the real people, and how they're building and deploying new capabilities using these tools, there has to be a mind's eye towards the not only the cost, the you know, the token at the tokenomics associated with it, but also there is a right size for right fit type of an approach here as well.
00:24:28.880 --> 00:24:30.640
And that's again back to infrastructure.
00:24:30.799 --> 00:24:43.920
You you're likely going to see a significant shift towards a hybrid IT, hybrid AI model that is a balance between I'm consuming some from the public, private, as a service, et cetera.
00:24:44.160 --> 00:24:52.240
But I'm gonna I'm gonna have applications and workloads and data access and security and governance controls of the right order for the right purpose.
00:24:52.400 --> 00:24:56.720
And I think that's, you know, maybe a few steps downstream from where we are today.
00:24:56.960 --> 00:25:02.559
But that that first article of a half a billion dollars in expense wakes everybody up.
00:25:02.720 --> 00:25:08.480
And so now there's a, you know, it's at least an acknowledgement as of today, you have to think about this.
00:25:08.559 --> 00:25:08.799
Yeah.
00:25:08.960 --> 00:25:24.160
And even though you're gonna go this path now, because it's the path either of least resistance or speed, you have to be mindful of not only the expense, the risk, and the security, but how you architect systems for the future to again enable the right fit for the right, the right purpose.
00:25:24.319 --> 00:25:28.400
Yeah, and locally deployed models are definitely gonna be there.
00:25:28.640 --> 00:25:40.319
Domain-specific models, I mean, it's a little bit of an oxymoron, but small language models, all of these are very much part of the techniques for development at scale.
00:25:40.480 --> 00:25:42.880
It is happening now, it's not future tasks, right?
00:25:42.960 --> 00:25:48.079
It is because you're not gonna go to the frontier model for every single question.
00:25:48.319 --> 00:25:56.480
That should be your, I don't want to say last resort, but that should be for the most sophisticated of the things that the local models cannot adequately answer.
00:25:56.720 --> 00:25:59.119
This is why e-vals become really important to you.
00:25:59.359 --> 00:26:11.920
This is why we we're really happy to have something like a Galileo as part of our mix because how do you make sure that when an agent fails at a certain task, for you to know what why did it fail?
00:26:12.720 --> 00:26:13.200
Right?
00:26:13.519 --> 00:26:22.960
So tying exactly to the spend and everything else that is happening, there is going to be specialization of these the way models are.
00:26:23.119 --> 00:26:30.240
And I, to anybody who disagrees with me, I tell them just look at how many, how much of specialization there is
in databases, right?
00:26:30.319 --> 00:26:31.359
That's right, right?
00:26:31.759 --> 00:26:41.119
Just look at how many bespoke variations of databases because at scale, you cannot not have that kind of specialization.
00:26:41.200 --> 00:26:42.960
So absolutely spot on.
00:26:43.279 --> 00:26:52.960
And so really honing in on what outcomes you're focused on and what is the true cost to get to that outcome, I think is becoming a lot more important these days.
00:26:53.279 --> 00:26:56.559
Accomplishing the task, yeah, which is what EBAs measure.
00:26:56.799 --> 00:26:57.519
Yeah, I think 100%.
00:26:57.759 --> 00:26:57.839
Yeah.
00:26:58.000 --> 00:27:02.799
It's a perfect way of so not just not just token spend, but but execution.
00:27:02.880 --> 00:27:05.519
And to build on your news article, I'll offer you another one.
00:27:05.599 --> 00:27:13.359
There was a Wall Street Journal article that that found only 18% of spending on AI coding tokens translated into shipped products.
00:27:13.440 --> 00:27:15.839
So, Joe, I mean, what's what do we think the gap is there?
00:27:15.920 --> 00:27:17.759
Is it just not knowing how to use the tools?
00:27:17.839 --> 00:27:18.720
Is it training?
00:27:18.880 --> 00:27:22.240
Is I I think a lot of people are playing with it.
00:27:22.319 --> 00:27:23.839
Yeah, and they're creating stuff.
00:27:23.920 --> 00:27:28.240
And to them, maybe it is providing value in their own day-to-day.
00:27:28.400 --> 00:27:33.440
But as you start looking, this goes back to sort of that outcome-based or kind of business value.
00:27:33.680 --> 00:27:35.279
What are you really trying to achieve?
00:27:35.359 --> 00:27:38.640
And maybe that's more of a team or department structure.
00:27:38.960 --> 00:27:42.240
Is it producing the outcome that group needs, not just the individual?
00:27:42.480 --> 00:27:44.160
I think I think we're still really early.
00:27:44.400 --> 00:27:44.720
Yeah.
00:27:45.599 --> 00:27:49.839
The shift is underway, but it's you know, it's top of the first right now for a baseball analogy.
00:27:49.920 --> 00:28:00.319
I think there's still a long way to go for an organization to see meaningful gains from a growth, an operational efficiency and or innovation standpoint.
00:28:00.400 --> 00:28:20.640
But there are absolutely whether it's 18% or 5% or 30%, I'd say instead of worldwide, we're seeing very meaningful progress through all of our departments, whether it's finance, it's HR, it's ops, it's global supply chain, it's our technology teams, all of us, our API, we're all gaining ground, becoming more operationally efficient.
00:28:20.880 --> 00:28:27.519
I think we're actually seeing a path to revenue gains as a result of a lot of work that we're doing and the way that we show up with our clients.
00:28:27.680 --> 00:28:32.400
Now, the question is, does that efficiency translate into meaningful progress?
00:28:32.640 --> 00:28:36.640
Or did I just give somebody back 20 minutes and an hour and they're gonna go screw around?
00:28:36.720 --> 00:28:36.960
Yeah.
00:28:37.119 --> 00:28:37.279
Right.
00:28:37.440 --> 00:28:43.039
Is that actually translating into that 20 minutes gets applied to a more meaningful task now that they couldn't otherwise get to?
00:28:43.200 --> 00:28:49.359
I think for us and for most of the clients that we encounter, there's more work to be done than they have people and resources.
00:28:49.519 --> 00:28:51.759
So there's a backlog that's a million miles long.
00:28:52.000 --> 00:28:58.240
Ideally, because we're able to get through more tasks faster, we're able to conquer that backlog more quickly.
00:28:58.319 --> 00:29:05.119
And again, turn your mind's eye, your best people, towards what are the biggest problems you have to solve and how do we start making progress against those?
00:29:05.440 --> 00:29:19.440
Talking about big problems and how tokens get utilized in the eight odd weeks, eight some weeks that we've had access to mythos, we have gone through nearly two billion lines of code around 50 plus products.
00:29:19.680 --> 00:29:21.119
All of that consumes tokens.
00:29:21.279 --> 00:29:21.599
Yeah.
00:29:21.759 --> 00:29:28.960
So it's not the producing more code, but all of the analysis that goes into it so that you're fixing it.
00:29:29.200 --> 00:29:32.880
Just give you one example because Mythos is on everybody's mind.
00:29:33.119 --> 00:29:40.880
But there are many, many such examples where there is, of course, there is some leakage where people are filipping with their use of tokens.
00:29:41.039 --> 00:29:48.240
But I would say by and large, it is used for whether it's for research, competitive Intel, digital twinning.
00:29:48.400 --> 00:29:56.480
There are thousands of other things that software requires other than just a writing code uh for it to go into production.
00:29:56.799 --> 00:29:58.400
So the measurement question matters.
00:29:58.559 --> 00:30:02.640
How do you know AI is actually delivering value and not just consuming tokens?
00:30:02.799 --> 00:30:11.519
That's a real challenge, but there's a harder challenge sitting underneath it because the same capabilities that make AI coding assistance powerful also make them dangerous in the wrong hands.
00:30:11.839 --> 00:30:18.400
I think Mythos came out and it either I think a third of the group got completely scared and goes, oh crap, what are we gonna do with this?
00:30:18.640 --> 00:30:23.599
A third thinks, eh, I don't know how it's gonna affect me yet, and a third just don't know how it's gonna affect them.
00:30:23.839 --> 00:30:27.440
So there's definitely still a lot of confusion about it.
00:30:27.680 --> 00:30:30.880
I know our security team acted very quickly on it.
00:30:30.960 --> 00:30:37.839
We've we've created workshops and briefings on in the past six weeks just because of the amount of customers asking us about it.
00:30:38.319 --> 00:30:41.599
But we see we see it being real.
00:30:41.680 --> 00:30:46.960
I
mean, this thing, it the cat's out of the bag once again, and this thing's got some real power behind it.
00:30:47.119 --> 00:30:58.000
I think kind of how you guys are using it for good of hey, I can look at all my code bases and quickly figure out what gaps I might have and remediate them faster than maybe you ever had before.
00:30:58.240 --> 00:31:07.279
I think everyone's just scared of what happens when mythos starts getting in the hands of the bad actors at some point, and what's that gonna do to everyone's infrastructure and their in their in their code?
00:31:07.519 --> 00:31:14.160
But there's there's just a lot of conversations being had about it and people need to take it seriously right now, for sure.
00:31:14.400 --> 00:31:23.519
And I think the just to tie it back to native engineering, there's there is a path here where you're using it to scan your code base or your infrastructure and look for vulnerabilities.
00:31:23.759 --> 00:31:29.519
To be able to patch those at the speed that you need to using legacy approaches might just not be fast enough.
00:31:29.599 --> 00:31:29.759
Yeah.
00:31:29.920 --> 00:31:42.160
And so to have coding assistants at your disposal that are proficient and actually tailor-mage for that type of a use case to quickly remediate your code and get yourself into a position where you're no longer at risk.
00:31:42.319 --> 00:31:44.000
That's actually where these two worlds collide.
00:31:44.079 --> 00:31:46.559
So now I've got tooling, whether it's mythos or other.
00:31:46.640 --> 00:31:51.279
There are plenty of other tools out there that will help evaluate the infrastructure and code for vulnerabilities.
00:31:51.359 --> 00:31:56.079
And now I've got assistance to help me patch or remediate really quickly.
00:31:56.240 --> 00:31:58.319
I think that's again, it's changing the game.
00:31:58.400 --> 00:32:03.440
These are not the same problems that we were dealing with before, and the solutions aren't going to be the same either.
00:32:03.599 --> 00:32:08.319
And we're we're coming up with new ways and new innovations to ensure that we're staying a step ahead.
00:32:08.559 --> 00:32:08.960
Yeah.
00:32:09.759 --> 00:32:11.200
I mean, you guys will see.
00:32:11.359 --> 00:32:14.480
We've blogged about this, we've taken some of these things to open source.
00:32:14.559 --> 00:32:15.920
It's called Foundry Spec.
00:32:16.000 --> 00:32:18.160
What is the harness that we built around?
00:32:18.480 --> 00:32:23.599
Because I mean, 1.8, whatever that number is, 1.8 billion lines of code.
00:32:23.759 --> 00:32:27.359
You don't go to a chatbot and say, here is a GitHub repo, tell me the button.
00:32:27.440 --> 00:32:28.880
That's not how it works, right?
00:32:28.960 --> 00:32:33.200
So there is a lot that goes into the harness to make sure that it is accurate.
00:32:33.440 --> 00:32:37.759
Cisco's taken that, what we used internally, and open sourced it, right?
00:32:37.920 --> 00:32:39.279
So everybody can use it.
00:32:39.440 --> 00:32:48.319
And I know I've talked to to Brian and a couple of other folks within your organization of how best to use it.
00:32:48.559 --> 00:33:03.920
The thing that I wanted to say again there is yes, finding bugs, yes, fixing code, but the biggest benefit that we get out of the AI model is the richness of the test cases that are built.
00:33:04.400 --> 00:33:11.440
Because it's those test cases that help us make sure that this fix is a durable, nothing more, nothing less.
00:33:11.599 --> 00:33:15.759
The performance is accurate, and that we have genuinely plugged it.
00:33:15.839 --> 00:33:24.480
It's not like we've just taken care of one aspect of it, but this is going to be effective in many different ways that we are going to see the attack form.
00:33:24.880 --> 00:33:41.519
So, from my perspective, not even just a practitioner's perspective, my perspective, it is the ability to write a very, very rich library of use cases where AI is a huge asset, more than even fixing the code.
00:33:41.839 --> 00:33:43.200
So we'll close out with this question.
00:33:43.279 --> 00:33:48.720
Um, I'm gonna ask each of you, you know, just in general, what do you think a priority or a you know a tip or piece of advice is?
00:33:48.960 --> 00:33:51.440
Joe, I'm gonna ask you from the perspective of a user.
00:33:51.599 --> 00:33:54.799
Brian, you'll be from the perspective of an organization or an executive.
00:33:54.880 --> 00:34:05.519
And then Raj will ask you from the security standpoint, what do we need to be doing to make sure that the back half of 2026 we're set up for success with coding agents and AI native engineering?
00:34:05.759 --> 00:34:13.360
Yeah, I'd say from the user perspective, it's be curious, start testing out these tools, play with them, become familiar with go get training.
00:34:13.440 --> 00:34:20.639
If your organization isn't providing these yet, you know, start fighting for those licenses because this is the new way.
00:34:20.719 --> 00:34:25.039
And if you're not doing it, your competition's coming up on you because they're using the tools too.
00:34:25.119 --> 00:34:29.119
So yeah, get used to it, don't be scared and just dive into it.
00:34:29.440 --> 00:34:31.039
Okay, so go go rack up the tokens.
00:34:31.199 --> 00:34:32.880
So go rack up the tokens.
00:34:33.039 --> 00:34:34.960
Okay, Brian might have something to combat that with here.
00:34:35.280 --> 00:34:35.840
Yeah, maybe.
00:34:36.000 --> 00:34:48.639
No, I think you know, if I'm an organizational leader or an enterprise and a business leader, I think one of the things that I'm as I just reflect on our conversation and all the conversations that we have, events like this and others, it's a lot.
00:34:48.880 --> 00:34:50.159
There's an awful lot to deal with.
00:34:50.239 --> 00:34:57.039
Whether you're in purchasing, you're a finance leader, you're a CIO, CTO, CISO, chief data officer, CEO, board.
00:34:57.280 --> 00:35:04.480
The the amount
of demands and things to unpack all at once, it's never been at this kind of pace.
00:35:04.960 --> 00:35:08.239
And the margin for error is just too narrow, right?
00:35:08.320 --> 00:35:19.599
It's just it's a combination of factors that make it very difficult for an enterprise leader to be able to move at the pace that they'd like to keep up with or even advance themselves.
00:35:19.840 --> 00:35:27.599
I I think the most important thing is, you know, being in an event like this with Raj with Cisco, choosing the right partners matters.
00:35:28.000 --> 00:35:43.280
And it makes an just an immediate impact when you look at having the right organization at your side to help you navigate the complexity of all these things, learn from each other, benefit from the experiences that have been getting elsewhere, tackle new problems in new ways.
00:35:43.440 --> 00:35:59.920
But I think, you know, organizations like Cisco, like worldwide, I think we we we pack the goods to really help navigate the complexity of all the challenges that exist out there and do so in a way that we're gonna fail quick, but keep moving forward and and ultimately find a really positive outcome in the end.
00:36:00.079 --> 00:36:07.679
Yeah, so Raj, I mean, if if we want everybody to get our hands on this, but also the margin for error is thin and there's some risk here, what do we need to know from a security perspective?
00:36:07.920 --> 00:36:09.840
I would say three or four things.
00:36:10.000 --> 00:36:13.199
Number one, you guys run capture the flag exercise.
00:36:13.519 --> 00:36:14.800
Yes, take advantage of it.
00:36:14.880 --> 00:36:15.280
Yeah, right.
00:36:15.440 --> 00:36:21.280
There's never been a better time using the sort of open source sort of tooling that is out there.
00:36:21.519 --> 00:36:22.960
Capture the flag, really important.
00:36:23.119 --> 00:36:24.239
Run a hackathon.
00:36:24.400 --> 00:36:26.559
Run a hackathon like you actually mean it.
00:36:26.880 --> 00:36:33.360
Give people truly two days, whatever the time frame is, and and give them the ambition to be bold.
00:36:33.679 --> 00:36:37.280
What is the biggest problem you would solve if cost was not an issue?
00:36:37.599 --> 00:36:38.079
Right?
00:36:39.119 --> 00:36:46.159
The third thing I would say is you've got to celebrate that moment where something happens.
00:36:47.039 --> 00:36:52.639
In if you use general terms, things become like, yeah, okay, we'll get to it one day.
00:36:52.880 --> 00:36:55.519
Here was how I inspired our team.
00:36:55.760 --> 00:37:07.280
We were at a customer advisory board, we ran a session, I told everybody explicitly from 3 30 to 5 30, we were gonna say, What is the most important new feature you require?
00:37:07.599 --> 00:37:08.079
Right?
00:37:08.320 --> 00:37:15.920
And the goal was that by the time CAB was over, which was a day and a half later, we will have that production ready feature.
00:37:16.719 --> 00:37:18.320
That was an explicit goal.
00:37:18.480 --> 00:37:23.280
You're gonna get something that you don't know about between 330, between 3:30 and 5.30.
00:37:23.440 --> 00:37:29.280
And by the time they leave the day after next at 11 or 12, it is has to be in production.
00:37:29.519 --> 00:37:30.800
And we did it.
00:37:31.679 --> 00:37:37.840
By making it concrete, people know what is the what is it that that makes it real.
00:37:38.000 --> 00:37:40.880
Otherwise, like code, we could go faster, blah blah blah.
00:37:41.039 --> 00:37:44.079
Like those things are good, but they don't have the impact.
00:37:44.239 --> 00:37:47.920
Yeah, give a goal that is concrete and then celebrate it.
00:37:48.159 --> 00:37:48.960
Really important.
00:37:49.119 --> 00:37:51.519
And then if something fails, yeah.
00:37:51.760 --> 00:37:52.400
Whatever.
00:37:52.559 --> 00:37:53.440
Fix fast.
00:37:53.599 --> 00:37:55.199
Uh failure will happen.
00:37:55.440 --> 00:37:56.239
Move on.
00:37:56.480 --> 00:38:00.559
Um but this is the moment time to be precise and auto.
00:38:01.519 --> 00:38:06.480
All right, thanks to Raz, Joe, and Brian for joining us during a very busy week at Cisco Live.
00:38:06.639 --> 00:38:11.840
The key lesson here: AI native engineering begins with experimentation, but it can't end there.
00:38:12.000 --> 00:38:29.760
The organizations that gain the most from these tools will not simply generate more code or consume more tokens, they'll be the ones turning domain expertise into working solutions, measuring whether those solutions produce meaningful outcomes, and building the security controls required for systems that can reason and act independently.
00:38:29.920 --> 00:38:32.159
That means giving people room to experiment.
00:38:32.400 --> 00:38:39.760
It also means giving agents identities, limiting their authority, understanding their intent, and knowing when their behavior begins to drift.
00:38:39.920 --> 00:38:44.000
And perhaps more importantly, it means choosing problems worthy of this moment.
00:38:44.239 --> 00:38:49.920
This episode of the AI Proving Ground Podcast was co-produced by Nas Baker, Kara Kuhn, and Sarah Chiadini.
00:38:50.159 --> 00:38:53.760
Our audio and video producers were John Knock and Brian Gagliano.
00:38:53.920 --> 00:38:55.199
My name is Brian Phelps.
00:38:55.280 --> 00:38:55.920
Thanks for listening.
00:38:56.159 --> 00:38:57.199
See you next time.