Strands Agents: A Model-Driven Approach to AI Agents with Clare Liguori (Senior Principal Engineer at AWS)

WEBVTT
00:00:05.700 --> 00:00:07.565
Welcome to What's New in Data.
00:00:07.565 --> 00:00:09.465
I'm your host, John Kutay.
00:00:09.465 --> 00:00:13.865
Today we're joined by Clare Liguori, senior principal engineer at AWS.
00:00:14.333 --> 00:00:25.466
Clare recently announced Strands Agents, a lightweight open source SDK that's already powering agent workloads at Amazon Q developer, Glue, and much more.
00:00:25.500 --> 00:00:33.265
And now that it's open source, the whole community gets to benefit from the simple abstractions that it gives us for building agents like applications.
00:00:33.633 --> 00:00:45.299
We're going to dive into the core principles behind Strands Agents, and also some practical advice for software engineers to break into AI engineering and how it's a little bit different from what they're used to.
00:00:45.365 --> 00:00:52.832
But more than that, Clare shows us what it means to think big about AI and how powerful it can be when you just start hacking on real problems.
00:00:53.033 --> 00:00:53.966
Let's dive right in.
00:00:55.033 --> 00:00:57.966
everybody, thank you for tuning in to this episode of What's New in Data.
00:00:57.966 --> 00:01:08.299
Super excited to be chatting here with Clare Liguori, Senior Principal Engineer at AWS, working on Strands, among many other exciting projects.
00:01:08.299 --> 00:01:10.299
Clare, how are you doing today? Good.
00:01:10.299 --> 00:01:12.500
Thanks for having me on. Absolutely.
00:01:12.500 --> 00:01:16.200
Clare, tell the listeners about yourself and your journey into this space.
00:01:16.465 --> 00:01:27.033
Yeah, I've been at AWS for almost 11 years now, and throughout that whole journey, I've really been focused on developers and developer experience.
00:01:27.566 --> 00:01:38.132
And, let's say two over two years ago, you know, generative AI as a real application for users was exploding.
00:01:38.132 --> 00:01:40.400
There was ChatGPT going on.
00:01:40.400 --> 00:01:45.000
Lots of new models were coming out, lots of new projects were coming out.
00:01:45.533 --> 00:01:56.266
And, I started to think, as well as others at AWS, about how generative AI could change, how we develop code, how we create applications as developers.
00:01:56.733 --> 00:02:14.699
And that really led me to start thinking about agenetic AI as well, when you can let the AI write an entire application for you and interact with the world for you, to pull in information and manage your, your AWS resources and things like that.
00:02:14.866 --> 00:02:23.532
So over the last couple of years, I've been so excited to see how powerful models have become, to enable building AI agents.
00:02:23.832 --> 00:02:24.566
Absolutely.
00:02:24.566 --> 00:02:32.400
AI agents are already proving their value, both for the indie developer and the and the enterprise.
00:02:32.400 --> 00:02:42.500
And I think one of the cool things about AWS is it's, you know, it has that range of, you know, being, valuable to, to, to all those, types of developers.
00:02:42.765 --> 00:02:48.800
Would love to hear, about your, your launch with Strands and what the product is.
00:02:48.800 --> 00:02:55.400
Strands agents is a open source library for developing AI agents.
00:02:56.099 --> 00:03:05.032
One of the key, design choices that we made with Strands is that it's what we call model driven, a model driven approach.
00:03:05.265 --> 00:03:15.066
When I started building AI agents back in early 2023 with, anthropic cloud v1, I think 1.3.
00:03:15.265 --> 00:03:17.000
I was looking at some of the code today.
00:03:18.032 --> 00:03:23.532
It was we were just starting to build agents in the industry.
00:03:23.532 --> 00:03:32.133
I think the, the original react paper that showed that that LLMS could interact with their environment through tools had just come out.
00:03:32.900 --> 00:03:35.532
But it was super hard at that time.
00:03:35.532 --> 00:03:47.733
You would I remember at the time, using Claude, you had to put all of these instructions and examples and XML tags and things like that, and you couldn't really get it to do much.
00:03:47.733 --> 00:03:50.099
That was complex.
00:03:50.099 --> 00:03:58.866
So one of the first agents that I tried to write was, troubleshooting problems with your AWS resource configuration.
00:03:59.133 --> 00:04:06.066
Things like, you know, you get this very opaque error message that something's broken with your application, but you don't know what.
00:04:06.300 --> 00:04:13.633
And so I was trying to figure out, well, could agents just kind of explore my AWS account and figure it out for me?
00:04:14.000 --> 00:04:18.632
And at the time, they just couldn't deal with that much complexity.
00:04:18.632 --> 00:04:33.733
And so you had to have kind of fallback to, prompt chaining, agent chaining, very explicit agent workflow for very explicit, specific use cases that those agents would be able to complete tasks for.
00:04:34.365 --> 00:04:47.132
But over time, now that we have this sort of latest crop of, of large language models, there's so much more capable of driving agents themselves.
00:04:47.132 --> 00:04:55.000
And that's really where we came to this model driven approach, because we were seeing that actually, we didn't need a lot of complexity.
00:04:55.000 --> 00:05:06.932
And some of the internal agents we were building at AWS with all of this chaining and workflows and things that you could really throw the tools directly at the model, and the model can figure out which tools to run.
00:05:07.233 --> 00:05:11.399
And it's actually a lot better at complex use cases now.
00:05:11.800 --> 00:05:21.100
And so you don't necessarily have to create a bunch of individual agents that are, sort of domain experts.
00:05:21.100 --> 00:05:24.966
You can have actually have one very generic agent with a set of tools.
00:05:25.533 --> 00:05:32.432
So that was really the, the beginning of Strands agents with us internally within one of the products I work on.
00:05:32.432 --> 00:05:44.199
Q developer realizing that there was a simpler way and there was actually was actually even, more higher quality because this is now how these models are being trained.
00:05:44.199 --> 00:05:46.500
They're being trained to use tools.
00:05:46.500 --> 00:05:54.932
So we, started to re-implement, throw away some of our agents internally and rebuild on this model driven approach.
00:05:55.365 --> 00:06:02.766
And we were seeing it was a better user experience, it was more accurate, and it was actually faster for us.
00:06:02.766 --> 00:06:03.000
Right.
00:06:03.000 --> 00:06:09.632
It's obviously faster to develop something where you write a prompt and you give it some tools and off you go.
00:06:09.632 --> 00:06:24.533
So that was we started developing it internally for us at AWS, and then, you know, we started seeing such great, impact internally for us that we, we wanted to release it for the, for the community.
00:06:24.766 --> 00:06:44.932
And so this model driven approach is really at the core of it, where it's really focused on let's leverage the power of how powerful these models have become, especially with tool use, and, and get rid of a lot of the complexity that that came from, you know, using earlier models like Claude V1.
00:06:44.932 --> 00:06:45.333
Yeah.
00:06:45.333 --> 00:06:50.199
And that's a great call out that, these lines are now being trained on using tools.
00:06:50.632 --> 00:07:02.300
You know, we'll get into this, but some of the protocols that are coming out, like MCP and A2A lots of acronyms, you know, these are all sort of materialize and they're productive.
00:07:02.699 --> 00:07:09.266
So, you know, there's certainly a lot of interesting directions it can go in, but it all starts with the model.
00:07:09.266 --> 00:07:21.266
Like you said, and having, you know, the, the, the models, the the prompts, the the evals, the tools, like as you're kind of, core building blocks.
00:07:21.266 --> 00:07:35.300
And then from anything there, you could kind of, from those abstractions, you can build applications on top, like what are the like, what are the other core architectural principles behind Strands, and how do they reflect a shift in how these agent systems are being built?
00:07:35.500 --> 00:07:48.699
Well, a couple are one, it's not tied to any the particular model, just that your model have your reasoning capabilities and tool use capabilities.
00:07:48.699 --> 00:07:52.065
And what we're seeing is that lots of models are now being trained that way.
00:07:52.365 --> 00:07:57.466
And lots of models either have sort of a messages API or a tools API.
00:07:57.466 --> 00:08:00.466
In the case of, anthropic APIs.
00:08:01.165 --> 00:08:10.533
And so we were really cognizant early on of what are the interfaces that we're providing for you to be able to plug in your favorite model.
00:08:10.533 --> 00:08:14.399
That could be something that's running locally on your laptop with a llama.
00:08:14.865 --> 00:08:17.865
It could be something that's running on AWS with bedrock.
00:08:18.100 --> 00:08:29.199
But we also partnered with anthropic to provide, anthropic API support, meta for, for their new, llama API.
00:08:29.233 --> 00:08:32.500
And we're working with a lot of other partners to provide additional APIs.
00:08:32.799 --> 00:08:46.432
So we wanted it to be something very, flexible in that in that case of, you know, regardless of what provider you're using, we're finding that there's a lot, more consistency in interfaces to models these days.
00:08:46.899 --> 00:08:53.932
And then we're also seeing more consistency, of course, in, interfaces to tools, like you said with MCP.
00:08:54.265 --> 00:09:13.500
That's been amazing for, having a more standard interface across whatever agent framework you might be happening to use, but then also for the tool author to be able to provide MCP and, servers for, for anybody to use, regardless of their agent framework.
00:09:14.033 --> 00:09:19.365
And so it's been amazing to see the growth in the community of those MCP servers.
00:09:19.365 --> 00:09:29.332
I mean, there's thousands now out on GitHub and, and on, you know, PyPI and and npm, and you can bring any of those to Strands.
00:09:29.332 --> 00:09:29.600
Right.
00:09:29.600 --> 00:09:35.466
So we're trying to support standard interfaces as much as we can.
00:09:35.466 --> 00:09:36.932
It supports open telemetry.
00:09:36.932 --> 00:09:40.432
So you can bring in whatever telemetry provider you prefer.
00:09:41.000 --> 00:09:42.566
And then you can run it anywhere.
00:09:42.566 --> 00:09:43.765
It doesn't need to be.
00:09:43.765 --> 00:09:48.666
It can run on AWS or it can run on, you know, your laptop or anywhere else you need to run it.
00:09:49.265 --> 00:10:00.899
So we wanted to provide something that wasn't tied necessarily to any individual provider, but really instead showcase this, this model driven approach that's been so powerful for us.
00:10:01.265 --> 00:10:23.000
I and that's a that's a very astute point that, you know, you have the model and you can have multiple models and you can have multiple runtimes and, you know, having the power to sort of develop locally and then scale in the cloud, and not be restricted, you know, based on, you know, your code shouldn't be tied to, you know, one specific environment.
00:10:23.100 --> 00:11:00.966
So it's taking a lot like when I looked at when I looked at it, I mean, it seems like it's taking a lot of just software engineering, you know, both, you know, scalable principles and practical, like, easy to develop with and, and I, and I like the other part like you mentioned, it's, it's, it's model driven, but but it can work with multiple models, which is becoming pretty standard for AI application development, even in the if you're doing things like, as a judge or, model cascading and you want to test, you know, cheaper models through some workflows and then, you know, potentially have to fall back to more, expensive ones.
00:11:02.000 --> 00:11:04.899
You know, if the cheaper, fast model is not working as well.
00:11:04.899 --> 00:11:21.765
So it's super flexible and I think it's, it's, it's a really nice reflection of where, AI applications are standardizing their approaches, diving in deeper into MCP and in a to a and these types of things.
00:11:21.765 --> 00:11:23.899
I, I would love to see your take on it.
00:11:23.899 --> 00:11:26.765
Like, where do these like emerging protocols.
00:11:26.765 --> 00:11:33.666
Like I mentioned, MCP and A2A sort of redefine the communication between intelligence systems.
00:11:33.832 --> 00:11:48.299
one of the specific, use cases that we've seen for multi-agent because I think, I think there's, what we've seen is that, agent to agent communication is not always need to be something special.
00:11:48.732 --> 00:12:03.932
So we've had a lot of success with internally with, the agent as tool model where, the agent is, a bit more of a sort of a request response where the sub agent, you're calling it as a normal tool.
00:12:03.932 --> 00:12:08.299
You could put it behind MCP, you give it a prompt, it comes back with a response.
00:12:09.100 --> 00:12:12.299
We have multiple systems internally that, that run that way.
00:12:12.633 --> 00:12:21.133
And some of that is because there are, agents that have specialized knowledge or which really ends up being specialized access to data.
00:12:21.700 --> 00:12:22.100
Right.
00:12:22.100 --> 00:12:33.299
But often what we see is it's also a way, organizationally to, to split up a very complex system into multiple teams.
00:12:33.765 --> 00:12:34.033
Right?
00:12:34.033 --> 00:12:36.899
So at Amazon, we're kind of famous for the two pizza team.
00:12:36.899 --> 00:12:42.133
And, you know, letting teams own, their, their whole system.
00:12:42.432 --> 00:12:55.966
And so in some ways, multi-agent is sometimes more of an organizational, solution where, you know, you have five different teams that want to own some experience or domain expertise in a, in an overall agent system.
00:12:57.000 --> 00:13:03.500
But we do see some more complex, use cases for agent to agent communication.
00:13:03.765 --> 00:13:14.533
So one challenge that we've had with MCP is with MCP being able to model many, many tools.
00:13:15.033 --> 00:13:22.566
So for example, we have an agent internally that models, more than 6000 tools.
00:13:22.566 --> 00:13:27.500
So there was those mapped to a huge number of, of AWS APIs.
00:13:28.000 --> 00:13:31.399
And that's you can't put that in your in your prompt context.
00:13:31.399 --> 00:13:31.832
Right.
00:13:31.832 --> 00:13:40.066
If you have an MCP, server that exposes 6000 tools, you'll probably blow out your context window.
00:13:40.066 --> 00:13:46.566
And if not, your requests are going to be very slow and very inaccurate in terms of what tool it chooses.
00:13:47.100 --> 00:14:04.265
And so we've done a lot of work around agents where, inside of this sort of sub agent, it does a lot of work around selecting the particular tools that you should, exposed to the LLM based on the customer's input.
00:14:05.133 --> 00:14:09.600
So, you know, if they asked about S3, it's probably going to be an S3 API.
00:14:09.600 --> 00:14:18.232
If they asked about EC2, it's going to be an EC2 API, and sometimes it's a little bit more, ambiguous.
00:14:18.232 --> 00:14:21.232
Like they might ask about a function and not say lambda.
00:14:21.533 --> 00:14:43.033
So, being able to figure out, you know, what are those maybe top five most relevant tools for the LLM is something that that agent to agent communication has been really helpful for, for us, and being able to, maintain state across a session in that sub agent in the agent as tool use case.
00:14:43.033 --> 00:14:47.399
Often you don't necessarily save state in that subnets that request response.
00:14:47.399 --> 00:14:48.466
And I'm done.
00:14:48.466 --> 00:14:58.100
But we have some of those use cases especially around that, that selection of within 6000 tools for the session, becomes very important.
00:14:58.500 --> 00:15:02.799
We're doing a lot of work with, we're doing some work with A2A.
00:15:02.832 --> 00:15:06.932
We have an example of using Strands with A2A.
00:15:07.732 --> 00:15:18.133
And that enables you to do, you know, to kind of hook into that entire ecosystem of having, agents talk with agents that could be implemented in any agent framework.
00:15:18.133 --> 00:15:18.432
Right?
00:15:18.432 --> 00:15:23.966
You might have some link chain, you might have some crew, you might have some Strands, and they can all communicate together.
00:15:24.265 --> 00:15:38.365
But we're also doing a lot of work, in the MCP community upstream to, try to evolve that, to be able to support agents, community over the MCP protocol, which is super interesting.
00:15:38.799 --> 00:15:59.732
So one example is, we a few weeks ago now, I think we got merged a new feature in the MTP spec called elicitation, where the MCP server can respond back with and elicitation along the lines of I need this information from the end user.
00:16:00.732 --> 00:16:05.865
And the the core agent, you know, goes off and and returns control back to the human user.
00:16:06.633 --> 00:16:09.799
And that's going to be super important, of course, for agent agent communication.
00:16:10.332 --> 00:16:15.765
So I think it's very interesting to see, you know, both how these both evolve, right.
00:16:15.765 --> 00:16:25.133
But also how we start to think about the lines blurring between, communicating with agents and communicating with tools if they're in the same interface.
00:16:25.865 --> 00:16:42.732
So at a high level and, and tell me if I'm oversimplifying, but, would you categorize MCP as the best way to, for, for agents to communicate with tools and then who is the best way to have, like these multi-agent communication today?
00:16:42.732 --> 00:16:55.899
Yeah, that's that's the, the pattern that I tend to see, is where, you know, it's really simple to wrap, MCP the interface around an agent.
00:16:55.899 --> 00:16:59.666
You know, it's, it's an API, that you can stand up.
00:16:59.666 --> 00:17:10.865
So we've had customers have success with, for example, putting an agent inside of a Lambda function and then just putting, you know, making that an API.
00:17:11.000 --> 00:17:13.133
That is MCP compliant.
00:17:13.133 --> 00:17:18.365
Some of the work that's been done upstream and MCP, I think has more enabled this.
00:17:18.365 --> 00:17:29.032
So in the original spec, it was just, mostly local MCP servers, which was great for, you know, Ides and chat clients.
00:17:29.032 --> 00:17:34.965
And if you look at those thousands of MCP servers out there, they're not they're not actually servers.
00:17:34.965 --> 00:17:41.900
They're really just local process with, you know, standard IO streams to your, your chat client locally.
00:17:42.232 --> 00:17:50.000
But where I see the industry going is really embracing, what we call the remote MC server.
00:17:50.000 --> 00:17:50.232
Right.
00:17:50.232 --> 00:17:53.099
The MCP server that's just behind an API.
00:17:53.099 --> 00:18:17.766
And that's really just become possible, I think, in the last couple of months, even, as the upstream community has been, adding new ways of, of building scalable MCP servers that are remote, with, something called a streamable Http transport, which basically just means, you know, putting it behind, more familiar Http APIs.
00:18:18.965 --> 00:18:19.532
Yeah.
00:18:19.532 --> 00:18:27.732
And that's definitely a big boost for, performance and just some other things like that that you would expect.
00:18:27.900 --> 00:18:29.799
An API to, to, to deliver.
00:18:29.799 --> 00:18:33.465
So that's a really exciting, drop for sure.
00:18:33.465 --> 00:18:38.465
And, and like you mentioned, you know, it just became available in the last few months every time.
00:18:38.700 --> 00:18:57.566
And just listening to these episodes back in January, like, there's there's been so much, innovation even since then, I think, like the rate of development, in AI obviously correlated with the, you know, I code gen tools, but, you know, and then, of course, the excitement and the value, it's bringing and I.
00:18:57.766 --> 00:18:57.965
Yeah.
00:18:57.965 --> 00:19:36.633
And, and when you look at, the frameworks that, that now, you know, companies like AWS are delivering, like you said, you've been working on this since, 2023, but it's on the, on the, on the Q developer side and the, the the amount of applications you've seen built then versus now, obviously a huge change in the, in the amount, but also the requirements around them, the, you know, being able to handle the scale of these data processing engines and databases, and, and making it streamlined for, enterprise application development.
00:19:36.633 --> 00:19:41.900
And then of course, like even even the SMB and mid-market ones as well that have large data volumes.
00:19:42.200 --> 00:19:44.766
So that's been just really fun to see.
00:19:44.766 --> 00:19:58.766
And I felt like it was a great turning point in the industry when you saw, Strands agents come out because it's just sort of this kind of light layer of abstraction to build abstractions to build these applications.
00:19:58.766 --> 00:20:05.165
It's, and it's really grounded in this, this idea, like you said, that, you know, we start with the models, right?
00:20:05.165 --> 00:20:08.000
Which is a powerful way to look at it.
00:20:08.000 --> 00:20:08.700
Definitely.
00:20:08.700 --> 00:20:23.066
And I think one of the interesting things that I've seen over the last, I would say six months maybe is, how differently we think about this model driven approach in terms of even just information, right?
00:20:23.066 --> 00:20:25.766
When we we all kind of started out building agents.
00:20:25.766 --> 00:20:35.633
It was all about rag, you know, right before you called to the model, you always supplemented your context with pulling from some vector DB, right.
00:20:36.266 --> 00:20:47.066
And one of the things I'm starting to see now is, what, internally, one of my colleagues calls retrieval as a tool as opposed to retrieval, augmented generation.
00:20:47.700 --> 00:20:52.599
I don't love the acronym Rat, but but there it is.
00:20:53.133 --> 00:21:05.965
So, you know, we're seeing a lot more of, you know, don't actively give information to the model that might be too much information or even information that it doesn't need.
00:21:05.965 --> 00:21:12.865
Like if it has enough information in its training data, you might actually make things worse by proactively giving it information.
00:21:13.299 --> 00:21:17.799
Instead, create a tool that that can retrieve information for it.
00:21:18.232 --> 00:21:29.965
So as opposed to proactively having kind of a prompt pipeline where you always augment that prompt, instead provide that vector DB as a tool where it can retrieve documentation.
00:21:31.099 --> 00:21:40.165
And I think one of the what are the places where we're seeing this that's interesting is in actually a lot of the development tools with MCP.
00:21:40.599 --> 00:22:01.299
So we're seeing a lot more of what I call documentation MCP servers, where, you know, in the agent space especially I was looking at the training dates recently of the latest Claud models and since, Claude 3.7 came out, there's been like ten new agent frameworks.
00:22:01.732 --> 00:22:09.266
And then even since sonnet four came out, I think I read that their training data ended in, in March of this year.
00:22:09.465 --> 00:22:12.865
Strands came out in May and so it doesn't even know about Stands yet.
00:22:13.232 --> 00:22:25.465
And so, we're starting to see a lot of these, you know, projects provide LLM tags so you can retrieve those through an MCP server and again, the MCP server is a tool.
00:22:25.465 --> 00:22:36.633
So it's letting the LLM decide, when do I actually need this information for my task versus have proactively throwing a bunch of information at it.
00:22:36.633 --> 00:22:50.465
So that that's been a big change in sort of those data access patterns that we've seen from moving from that prompt pipeline where you would try to get it every time, and then letting the model decide when it needs that information.
00:22:50.465 --> 00:22:52.865
Absolutely. And timely.
00:22:52.865 --> 00:23:01.965
Because just, at least from the time we're recording this episode, last week, in the Strands GitHub repo.
00:23:01.965 --> 00:23:03.566
And that's one of the other great things about Strands.
00:23:03.566 --> 00:23:07.865
There's so many great actionable, easy to run, examples on the GitHub.
00:23:07.865 --> 00:23:10.900
I'll have that down in the, the link to that in the show notes.
00:23:10.900 --> 00:23:23.266
But there was just recently one that came out with, Amazon Aurora, de SQL, running through Strands agents and just like you mentioned, it's, it's retrieving data with these tool calls.
00:23:23.599 --> 00:23:23.965
Right?
00:23:23.965 --> 00:23:30.165
So you look at the architecture diagram and it's and it's reading read only query, it's doing transactions.
00:23:30.165 --> 00:23:37.232
There's get schema and different just high level tool call APIs that interact with the database.
00:23:37.665 --> 00:23:44.365
And when, when Mark Broker he was he was on What's New in Data last season.
00:23:44.400 --> 00:23:53.865
And, you know, we were chatting about how agents and, serverless databases actually are very complementary technologies.
00:23:54.000 --> 00:23:56.799
So it's very cool to see sort of the materialization here.
00:23:56.799 --> 00:24:09.465
And, and like you said, it's it's it's super powerful too, rather than having a rag and trying to like populate your, your, your prompts with, with contacts kind of manually or, or overloading it.
00:24:10.165 --> 00:24:21.299
The, the agent can kind of decide when it wants to make a tool call, to, to retrieve very specific data with specific parameters and it makes the application more deterministic.
00:24:21.700 --> 00:24:26.333
And we've also seen that, it also makes it much more generic.
00:24:26.665 --> 00:24:27.232
Right.
00:24:27.232 --> 00:24:40.965
You don't have to have a very specific prompt pipeline where you determine the, the query to the database ahead of time and then add the data to the, to the prompt.
00:24:41.365 --> 00:24:57.200
You're able to apply these agents to much more generic problems than you could before by effectively teaching the model how to query, and then giving it some, some boundaries, right, giving it some guidelines.
00:24:57.900 --> 00:25:04.833
But generally it's, it's much more, flexible for, you know, a lot of different tasks that you could throw at it.
00:25:05.200 --> 00:25:06.400
Yeah, absolutely.
00:25:06.400 --> 00:25:12.500
And, and it gives a good guardrails for, for things like data access.
00:25:12.900 --> 00:25:28.299
You know, even in that example, there's, you know, it specifies, you know, read-only queries and, retrieving schemas, but then it bubbles up into this powerful experience where, you know, the agent can, you know, get all the tables and do some operations.
00:25:28.500 --> 00:25:47.932
But but a smart, engineer who, who understands, you know, how to set up database applications can still predefine those tool calls in a way that's that's safe and scalable, and you can even do your, your explain and analyze to to to make sure those are efficient queries.
00:25:48.232 --> 00:25:55.000
Or you have you might have agents interacting with systems that that with databases that are trafficked on other applications as well.
00:25:55.266 --> 00:25:57.665
So it's a very powerful concept.
00:25:57.665 --> 00:26:11.465
And it's so nice to see in very, succinct and well-organized lines of code, how it can be implemented, in that, in that GitHub repo, along with, with a lot of other fun examples.
00:26:11.465 --> 00:26:21.133
And since, you know, we're on that topic, like, what's one of your favorite tutorials, in that, in that GitHub repo that, that you like to call out?
00:26:21.266 --> 00:26:23.299
It's actually not in the GitHub repo.
00:26:23.299 --> 00:26:29.000
I put it in the, in our blog post, about the Strands launch.
00:26:29.432 --> 00:26:36.266
When we were, we decided to open source Strands.
00:26:36.266 --> 00:26:44.865
It had a different code name internally, right, that we, and so we were trying to come up with, okay, what are we going to call this, this open source project?
00:26:44.865 --> 00:26:51.766
We need to come up with a name and, and, you know, the lawyers have to sign off on it, and it, you know, it can't be too similar to any other companies or whatever.
00:26:52.333 --> 00:26:58.532
And, as it turns out, in the AI space, all words are taken.
00:26:58.865 --> 00:27:01.799
Everybody's got, you know, blank AI.
00:27:01.799 --> 00:27:02.500
Right?
00:27:02.500 --> 00:27:11.432
So, so we were having this trouble where, you know, we do a brainstorming session where we come up with, oh, this would be a really cool name.
00:27:11.432 --> 00:27:13.133
That would be a really cool name.
00:27:13.133 --> 00:27:18.665
And then we'd go and search on it online and it would say, oh, of course, of course it's taken already. Right?
00:27:18.665 --> 00:27:30.900
Or we find one that didn't have a very, very clear company that was already using it, but someone had already registered the domain that we would want to use, or the GitHub org that we would want to use.
00:27:31.365 --> 00:27:40.299
And so I wrote, with Strands a little bot that would, help me to come up with names.
00:27:40.299 --> 00:27:49.965
So in the prompt I would give it some like some guardrails around, you know, how we name things and then, some ideas of themes that I really liked.
00:27:50.133 --> 00:27:52.865
Like we had, you know, a DNA theme going.
00:27:52.865 --> 00:27:56.732
We had a, like a sewing theme going as well for a while.
00:27:57.032 --> 00:28:05.900
We had a, like a car theme going for a while where you would think about, like a motor going around a root or an axle going around and around.
00:28:06.432 --> 00:28:10.566
Anyway, so I gave out all this ideas and then I gave it a few tools.
00:28:10.566 --> 00:28:20.865
I wrote an MCP server, for my favorite thesaurus website, because I always felt that helps when I'm brainstorming.
00:28:21.333 --> 00:28:29.232
I wrote a, a, domain name search, you know, figure out whether it was registered or not.
00:28:29.665 --> 00:28:40.133
I discovered that, through this, I didn't know this, who is is no longer the hot thing for figuring out for domain name registration.
00:28:40.566 --> 00:28:45.165
It's now Rdapper or RDAP.
00:28:45.500 --> 00:28:52.965
And as it turns out, the.dev TLD on my Google doesn't support who is, it only supports RDAP.
00:28:52.965 --> 00:28:57.532
And so I was using this who is MCP server that I found, you know, on GitHub.
00:28:57.900 --> 00:29:04.032
And it nothing.dev was coming back and of course we wanted to use it.
00:29:04.299 --> 00:29:07.266
We wanted to to, you know, explore using that .Dev.
00:29:07.266 --> 00:29:09.932
This was a very developer focused project, right.
00:29:09.932 --> 00:29:17.000
And then I also had like the GitHub MCP server where it could look up, possible org names.
00:29:17.532 --> 00:29:22.665
And that helped us just churn through ideas, which was really cool.
00:29:22.799 --> 00:29:25.365
And we were able to prove like, hey, this one doesn't work.
00:29:25.365 --> 00:29:28.965
This one doesn't work because it's got these, conflicts already.
00:29:29.465 --> 00:29:31.299
And that was just very exciting.
00:29:31.299 --> 00:29:34.465
You know, one, it was super easy to write with Strands.
00:29:34.465 --> 00:29:37.465
It was, I don't know how many lines of code, but really small.
00:29:37.665 --> 00:29:53.432
And then also being able to take advantage of, some of the MCP servers that are already out there, like, I was able to pull the GitHub one off the shelf, and then it was really easy to write up one where I was, web scraping.
00:29:53.432 --> 00:29:56.299
My favorite this the source website to be unnamed.
00:29:56.299 --> 00:30:02.900
And then, you know, adding, this domain name lookup, one that I found.
00:30:02.900 --> 00:30:08.099
So, you know, it was it was just so easy to kind of pull these things together.
00:30:08.532 --> 00:30:15.766
And the the most work that I was really doing was thinking about what are the the real kind of business value.
00:30:15.766 --> 00:30:16.099
Right.
00:30:16.099 --> 00:30:26.266
I was thinking about what are the, themes that we want to explore, what are the names that we really like or don't like, as opposed to doing all this manual lookup.
00:30:26.266 --> 00:30:28.133
So it's really, really fun for me.
00:30:28.133 --> 00:30:37.032
And that that also speaks to, the simplicity and how it was just able to aid your creative process.
00:30:37.799 --> 00:30:51.365
Well, while you were working through a higher level challenge was like naming something and that's, you know, and then, for, for, for AWS, the the scope of naming something obviously has even more, challenges to it.
00:30:51.365 --> 00:31:09.432
Which, which, which, you know, you're able to, to navigate things to, you know, some, some clever, both brand brand thinking and also tooling, but, you know, and it's not going to be we're not far away from a future where, like you said it, you just you you have this tactical thing you have to get done, and.
00:31:09.432 --> 00:31:12.700
Okay, let's let's outsource it to some agents.
00:31:13.133 --> 00:31:16.766
That can do a lot of the, hard work for me.
00:31:16.766 --> 00:31:21.833
And, you know, I didn't know about the RDAPR whois thing.
00:31:22.066 --> 00:31:23.766
And so, yeah, I find it.
00:31:23.766 --> 00:31:28.232
But, you know, it sounds like it's something that, you know, you discovered in the process.
00:31:28.232 --> 00:31:32.766
Maybe your agents discovered in the process and told you, Yeah.
00:31:32.766 --> 00:31:34.566
No, exactly.
00:31:34.566 --> 00:31:38.833
And, and, you know, there's one end where.
00:31:38.932 --> 00:31:52.500
Yeah, you'll you'll use these products which sort of wrap the agent experience and make it easy for you for, for for folks who are, you know, non-technical and, you know, they'll just type in some, some instruction and it's going to deploy a fleet of agents to get it done.
00:31:52.900 --> 00:32:12.200
And then, on the other hand, you're going to have developers who either through hand coding or, AI code gen, can can use like a nice wrapper, framework to, to, to build this stuff for themselves, at a, at a rate which was just not possible before.
00:32:12.566 --> 00:32:25.833
And it's from that perspective, it's, it's super exciting both like as for, for like the non-technical consumers of AI and, and the builders who, who, who want AI at their, at their fingertips.
00:32:26.865 --> 00:32:32.900
So, I, I totally recommend, we'll have a link to that, to your, to your blog in the show notes as well.
00:32:33.599 --> 00:32:35.432
And it's in to the, to the GitHub repo.
00:32:35.432 --> 00:32:44.566
But I always just encourage people like the people who are creating these frameworks, including yourself, are making it easy to run locally and then deploy in the cloud.
00:32:44.566 --> 00:32:50.965
So start with your your laptop, your laptop is, most people's laptops are pretty powerful these days.
00:32:50.965 --> 00:33:09.333
There's a lot of great, tooling, but especially with Python, to build really nice applications that are portable, that, you know, you can you can really get far on your laptop and then still be in a state where you can deploy to the cloud without changing too much, in terms of, you know, just just environment variables.
00:33:09.333 --> 00:33:21.700
So I do totally recommend everyone to just just get, get hacking and, you know, hack around and play with these frameworks and, and try it out because you, because people don't even understand what they can do.
00:33:22.099 --> 00:33:25.766
You know, just because it just seems impossible.
00:33:25.766 --> 00:33:30.566
Like, even if you've been in software engineering for 25 years, I hear that a lot.
00:33:30.566 --> 00:33:32.732
People just didn't understand what was possible.
00:33:32.732 --> 00:33:34.299
And it's it's so powerful.
00:33:34.299 --> 00:33:44.766
Like, you know, you kind of just have to go, go do it and play around and get that, that empirical knowledge, yourself and Strands is a great, great framework for that.
00:33:45.333 --> 00:33:50.165
You know, looking ahead like, you know what foundational.
00:33:50.165 --> 00:33:57.633
But so we're talking about all the amazing stuff that's there, but what are the foundational building blocks that are still missing from the agent ecosystem?
00:33:58.000 --> 00:34:12.900
Well, I think one is that I'm kind of excited about is thinking about how can we reuse these agents across, other developers like, you know, we think about we have libraries, right.
00:34:12.900 --> 00:34:19.166
And, and we've created this amazing ecosystem of libraries across all these package managers.
00:34:19.166 --> 00:34:25.900
But I also think that agents are kind of the new, you know, MCP library and city servers are the new libraries.
00:34:25.900 --> 00:34:28.032
Agents are going to be the new libraries.
00:34:28.032 --> 00:34:45.300
And so how do we share these or, you know, make them available over APIs or something where, you know, as a developer creating an application, maybe what I'm doing is pulling together a set of MCP servers and pulling together a set of agents.
00:34:45.932 --> 00:35:14.166
You know, I've seen that, you know, even just with my little naming bot, I was seeing that the power of being able to just pull in these, these MCP servers, whereas before, you know, I think about API integration work using some like a, like the GitHub library, for example, would still take me a long time to build to write all that client code and, MCP servers, I just pulled it in and, and, you know, wrote a prompt.
00:35:14.565 --> 00:35:19.400
And I think that sharing agents could potentially be even more powerful.
00:35:19.965 --> 00:35:27.432
So I'm, I'm looking forward to that world where, you know, we we start to share them as well, like libraries.
00:35:27.599 --> 00:35:36.365
And then I think also, the it's still very unsolved about building up entire applications around agents.
00:35:37.500 --> 00:35:41.733
You know, you still have to do a lot of work to, to put it behind an API somewhere.
00:35:41.733 --> 00:35:45.432
You have to, you know, figure out authentication.
00:35:45.833 --> 00:35:52.865
You have to figure out how does my my react code in the front end interact with this agent?
00:35:52.865 --> 00:35:54.900
What does it look like in the UX?
00:35:54.900 --> 00:35:58.666
I think we're still very stuck in the, generative.
00:35:58.666 --> 00:36:01.032
The AI means chat world.
00:36:01.032 --> 00:36:05.632
So I'm looking forward to, you know, the UX around agents evolving as well.
00:36:06.000 --> 00:36:13.733
Where we start to think about, how could agents help in my work other than having a conversation with it?
00:36:13.865 --> 00:36:17.199
So I think there's just so much space.
00:36:17.199 --> 00:36:25.333
It's and it's like you said, this space is changing so fast that, you know, you can't even imagine what's going to come in three months, right?
00:36:25.965 --> 00:36:26.565
Yeah.
00:36:26.565 --> 00:36:54.632
And I, I really like your point about just being able to share agents sort of like sharing a, an API, but you have everything, ready for you to use there to execute that agent and, and also, one of the things that's really, it seems critical for AI applications is, you know, the traces, the evals, the, you know, you're kind of your integration tests for, for the AI, like, changes to the, to the prompts.
00:36:54.632 --> 00:36:58.365
How does that actually impact the, the the outputs?
00:36:58.365 --> 00:37:10.065
So when I come back to, you know, your idea of, of sharing agents, doing that in a way that also like, delivers like the trust that it's that it's working the right way.
00:37:10.300 --> 00:37:13.865
And I think we're all as an industry trying to figure out like, what are that?
00:37:13.865 --> 00:37:16.099
What are those sort of units of trust?
00:37:16.099 --> 00:37:24.900
And of course, you know, you have LLMs as a judge and in confidence scores and all these things that are really useful and, you know, just borrowed from, from machine learning.
00:37:25.000 --> 00:37:30.900
I always when I'm working with software engineers who haven't done agentic or AI development before, I'm like that.
00:37:31.032 --> 00:37:32.365
That's where you should really start.
00:37:32.365 --> 00:37:35.800
Like how we come up with your framework for deciding like what's correct.
00:37:36.065 --> 00:37:53.166
One of the things I've noticed working with teams at AWS who are starting to build generative AI capabilities is exactly that hurdle that, us as engineers expect testing to look like a define an input, I define an output.
00:37:53.266 --> 00:37:55.565
I get that output when I provide the input.
00:37:55.565 --> 00:37:57.266
Then things are working.
00:37:57.266 --> 00:38:00.666
And obviously generative AI is not like that.
00:38:00.666 --> 00:38:07.166
You have to you can't rely on it providing you ever the same answer as what you got before.
00:38:07.599 --> 00:38:21.266
And so thinking about, what success criteria looks like, what good looks like is it know 95% accuracy of tool selection.
00:38:21.266 --> 00:38:24.699
Is it, 95%.
00:38:24.932 --> 00:38:30.233
You know, LLM judge saying this is close enough to your ground truth answer.
00:38:30.900 --> 00:38:34.365
How do you even think about creating test sets and ground truth?
00:38:34.565 --> 00:38:35.932
Answers.
00:38:35.932 --> 00:38:40.032
These are problems that, as an industry software engineers have never had to deal with.
00:38:40.032 --> 00:38:42.900
This has been scientist ML scientist stuff.
00:38:42.900 --> 00:38:43.800
Right.
00:38:43.800 --> 00:38:49.300
And so we're all kind of needing to become scientists a little bit.
00:38:49.300 --> 00:38:49.632
Right?
00:38:49.632 --> 00:38:55.333
And we, we're not used to, using the scientific method we used in, learned in grade school at work.
00:38:55.333 --> 00:38:56.065
Right.
00:38:56.065 --> 00:39:01.365
Having a hypothesis and coming up with a test and, and executing that test.
00:39:01.900 --> 00:39:14.266
And so, I think that, evaluation tools today are still really hard for engineers to grok in a lot of ways.
00:39:14.266 --> 00:39:21.465
You know, even that question of, well, how do I come up with the test set that this evaluation tool is going to go through is really hard?
00:39:21.599 --> 00:39:29.900
I think there's so much room for improvement there that speaks to, you know, the fact that we are complete amateurs in space.
00:39:29.932 --> 00:39:32.233
You know, I don't have a I don't have a science PhD.
00:39:32.233 --> 00:39:45.800
So, and then I think also mapping it to, the reality of the, you know, the way that your users are actually interacting with your system.
00:39:45.800 --> 00:39:46.565
Right?
00:39:46.565 --> 00:39:53.965
One thing that we say at AWS, for any service we launch, they're going to be customers that use it in ways you never expected.
00:39:54.365 --> 00:39:57.166
And I think that's probably true of any application.
00:39:57.166 --> 00:40:04.099
And so you have before you launch something, you have this guess as to how people are going to use your system.
00:40:04.099 --> 00:40:23.065
And you create a test set based on that guess, but then you have to follow back and you have to validate, okay, what kinds of questions are people actually asking of my of my agent, or what kind of inputs are they actually providing and try to match that distribution in your test set?
00:40:23.965 --> 00:40:25.965
And that can also be really hard, right?
00:40:25.965 --> 00:40:32.699
How do you how do you think about that kind of matching that data, the live data set with what you're testing against?
00:40:33.266 --> 00:40:33.900
Yeah.
00:40:33.900 --> 00:40:36.465
And we're sort of scraping the surface of this.
00:40:36.465 --> 00:40:49.199
But you know, if I were to ask you generally like what's your advice for software engineers, whether they're new or experienced or just fundamental computer science background, what's your advice to software engineers who are breaking into AI engineering?
00:40:49.900 --> 00:40:53.465
One is, try Jupyter notebooks.
00:40:55.500 --> 00:41:02.032
That's something I had never I had never heard of, honestly, coming out of, you know, a computer science program.
00:41:02.400 --> 00:41:10.166
And then all of a sudden, I'm working with these, you know, ML and AI scientists, and they're all sending me these Jupyter notebook files, you know?
00:41:10.565 --> 00:41:17.500
So, we do actually have a lot of, Jupyter notebook examples in the Strands examples repo.
00:41:17.500 --> 00:41:19.300
So that's a good place to start.
00:41:19.300 --> 00:41:40.965
But I think also thinking about, again, kind of how, how testing is going to look like one thing that I often see from myself as well as other engineers is, what I started call vibe checking, which is, you know, we know about vibe coding, but how do you know when your thing works?
00:41:40.965 --> 00:41:42.699
How do you know when your agent works?
00:41:42.699 --> 00:41:49.733
You just kind of give it a couple of inputs, and then you personally like the outputs, and then you send it on its way.
00:41:49.733 --> 00:41:50.500
Right.
00:41:50.500 --> 00:41:59.266
Starting to think about, okay, how can I actually write these down and think about what do I expect the answer to be?
00:41:59.266 --> 00:42:03.532
And thinking about how you would compare, you know, good versus not good.
00:42:03.766 --> 00:42:10.666
So I think that there is a lot of kind of intuition that you have about what is good and what's hard is writing that down.
00:42:11.132 --> 00:42:16.766
And so trying to kind of look through what, what you would think of as good.
00:42:16.766 --> 00:42:24.266
We actually have a, we have another product called I'm not going to get it right, but it's something like bedrock automated reasoning.
00:42:24.266 --> 00:42:34.599
And one of the challenges that we have with that product is basically looks for hallucinations in, large language model outputs.
00:42:35.065 --> 00:42:39.099
But one of the things you have to do ahead of time is codify.
00:42:39.099 --> 00:42:40.500
What is the truth?
00:42:40.500 --> 00:42:44.532
It can only tell you it's wrong if you've already told it the truth.
00:42:44.800 --> 00:42:49.233
And then it uses mathematical proofs and automated reasoning to determine whether it is true or not.
00:42:49.233 --> 00:42:51.166
When it comes out of the model.
00:42:51.166 --> 00:42:54.132
And one of the things that we found is people don't know it's true.
00:42:54.132 --> 00:43:01.800
And so what's really interesting is, you start with here's what I think is true.
00:43:02.032 --> 00:43:12.900
And then it kind of takes you through some scenarios and says, okay, if this is the case and this is the case based on what you've told me, this would be the result.
00:43:12.900 --> 00:43:15.032
And then you say, no, no, no, no, that's not right.
00:43:15.032 --> 00:43:25.733
So one example is, if you give it a tax code, and it does it best to take a tax code and turn it into mathematical models, then it'll give you.
00:43:25.733 --> 00:43:38.099
Okay, if you are, a married couple with two dependents, then your deduction is ex and someone who's a subject matter expert on on actually doing that math can come in and say, no, no.
00:43:38.099 --> 00:43:39.000
And this is why.
00:43:39.000 --> 00:43:44.800
And it can go back and, and, regenerate those, those models.
00:43:44.800 --> 00:43:50.333
And I think that's a very interesting for how we think about what good looks like in our application.
00:43:50.766 --> 00:43:54.599
You know, play, play kind of a game with yourself.
00:43:54.599 --> 00:43:55.199
Okay.
00:43:55.199 --> 00:43:59.565
If this was the input and that was the output, what would I like about it or not like about it?
00:43:59.565 --> 00:44:02.565
And then that effectively is you're oh I'm judge prompt.
00:44:02.800 --> 00:44:03.333
Right.
00:44:03.333 --> 00:44:33.333
And so if you're able to give those kind of self introspect and come up with some of those, those guidelines and that can really help you to build the application you want to build, that, that follows what, what you, you know, what you want to put out into the world, Yeah, I love the I love the, term vibe checking, you know, equally as important as the, the the vibe coding and, in the AI engineering process and, you know, we.
00:44:33.333 --> 00:44:39.766
Yeah, you know, we we as humans, we have this natural reflex to be really good at kind of critiquing, critiquing what we don't like.
00:44:39.833 --> 00:44:50.333
But, you know, it's funny how just having, like, a positive mindset and defining what's good, in a very structured way, is actually very productive for AI, right?
00:44:50.532 --> 00:44:58.233
So, you know, when you're coming up with your, your ground truth and you're coming up with, just ways to, to to define that clearly.
00:44:58.233 --> 00:45:06.266
And like you said, it takes it takes intuition about the core problems that your AI application, is solving. Right.
00:45:06.266 --> 00:45:12.599
And I think that kind of forces everyone to be, you know, even on the engineering side, better, better product thinkers.
00:45:12.632 --> 00:45:22.965
Because ultimately, yes, having that that domain expert, who can define what's good, and do that up front and think about the thousand scenarios of generating good outputs.
00:45:23.400 --> 00:45:30.365
That's, that's ultimately going to be super helpful, and make the AI application that much better.
00:45:30.532 --> 00:45:45.400
So yes, great, great advice to software engineers who typically think the other way, which is like, hey, I'm going to develop all the code first and then test. When it's yes, I know, it's almost like test driven development could make a comeback.
00:45:45.400 --> 00:45:49.266
It could it could make a big comeback with with AI applications.
00:45:49.333 --> 00:45:54.132
Clare, it was super great having you on this episode of What's New and Data.
00:45:54.300 --> 00:45:57.000
Where can people continue to follow along with your work?
00:45:57.000 --> 00:46:00.000
I'm on LinkedIn and I'm on blue Sky.
00:46:00.465 --> 00:46:01.132
Excellent.
00:46:01.132 --> 00:46:12.965
We'll have links to, Clare Liguori's Blue Sky and her LinkedIn below, along with, her awesome blog post going through and demonstrating Strands agents.
00:46:13.400 --> 00:46:17.132
And, thank you to the listeners for for tuning in today.
00:46:17.132 --> 00:46:18.532
This is super fun episode.
00:46:18.532 --> 00:46:21.500
A lot of great insights. Clare.
00:46:18.532 --> 00:46:21.500
Thank you again for joining.
00:46:21.500 --> 00:46:24.500
Thanks for having me.
00:54:06.400 --> 00:54:09.333
everybody, thank you for tuning in to this episode of What's New in Data.
00:54:09.333 --> 00:54:19.666
Super excited to be chatting here with Clare Liguori, Senior Principal Engineer at AWS, working on Strands, among many other exciting projects.
00:54:19.666 --> 00:54:21.666
Clare, how are you doing today? Good.
00:54:21.666 --> 00:54:23.865
Thanks for having me on. Absolutely.
00:54:23.865 --> 00:54:27.565
Clare, tell the listeners about yourself and your journey into this space.
00:54:27.833 --> 00:54:38.400
Yeah, I've been at AWS for almost 11 years now, and throughout that whole journey, I've really been focused on developers and developer experience.
00:54:38.932 --> 00:54:49.500
And, let's say two over two years ago, you know, generative AI as a real application for users was exploding.
00:54:49.500 --> 00:54:51.766
There was ChatGPT going on.
00:54:51.766 --> 00:54:56.365
Lots of new models were coming out, lots of new projects were coming out.
00:54:56.900 --> 00:55:07.632
And, I started to think, as well as others at AWS, about how generative AI could change, how we develop code, how we create applications as developers.
00:55:08.099 --> 00:55:26.065
And that really led me to start thinking about agenetic AI as well, when you can let the AI write an entire application for you and interact with the world for you, to pull in information and manage your, your AWS resources and things like that.
00:55:26.233 --> 00:55:34.900
So over the last couple of years, I've been so excited to see how powerful models have become, to enable building AI agents.
00:55:35.199 --> 00:55:35.932
Absolutely.
00:55:35.932 --> 00:55:43.766
AI agents are already proving their value, both for the indie developer and the and the enterprise.
00:55:43.766 --> 00:55:53.865
And I think one of the cool things about AWS is it's, you know, it has that range of, you know, being, valuable to, to, to all those, types of developers.
00:55:54.132 --> 00:56:00.166
Would love to hear, about your, your launch with Strands and what the product is.
00:56:00.166 --> 00:56:06.766
Strands agents is a open source library for developing AI agents.
00:56:07.465 --> 00:56:16.400
One of the key, design choices that we made with Strands is that it's what we call model driven, a model driven approach.
00:56:16.632 --> 00:56:26.432
When I started building AI agents back in early 2023 with, anthropic cloud v1, I think 1.3.
00:56:26.632 --> 00:56:28.365
I was looking at some of the code today.
00:56:29.400 --> 00:56:34.900
It was we were just starting to build agents in the industry.
00:56:34.900 --> 00:56:43.500
I think the, the original react paper that showed that that LLMS could interact with their environment through tools had just come out.
00:56:44.266 --> 00:56:46.900
But it was super hard at that time.
00:56:46.900 --> 00:56:59.099
You would I remember at the time, using Claude, you had to put all of these instructions and examples and XML tags and things like that, and you couldn't really get it to do much.
00:56:59.099 --> 00:57:01.465
That was complex.
00:57:01.465 --> 00:57:10.233
So one of the first agents that I tried to write was, troubleshooting problems with your AWS resource configuration.
00:57:10.500 --> 00:57:17.432
Things like, you know, you get this very opaque error message that something's broken with your application, but you don't know what.
00:57:17.666 --> 00:57:25.000
And so I was trying to figure out, well, could agents just kind of explore my AWS account and figure it out for me?
00:57:25.365 --> 00:57:30.000
And at the time, they just couldn't deal with that much complexity.
00:57:30.000 --> 00:57:45.099
And so you had to have kind of fallback to, prompt chaining, agent chaining, very explicit agent workflow for very explicit, specific use cases that those agents would be able to complete tasks for.
00:57:45.733 --> 00:57:58.500
But over time, now that we have this sort of latest crop of, of large language models, there's so much more capable of driving agents themselves.
00:57:58.500 --> 00:58:06.365
And that's really where we came to this model driven approach, because we were seeing that actually, we didn't need a lot of complexity.
00:58:06.365 --> 00:58:18.300
And some of the internal agents we were building at AWS with all of this chaining and workflows and things that you could really throw the tools directly at the model, and the model can figure out which tools to run.
00:58:18.599 --> 00:58:22.766
And it's actually a lot better at complex use cases now.
00:58:23.166 --> 00:58:32.465
And so you don't necessarily have to create a bunch of individual agents that are, sort of domain experts.
00:58:32.465 --> 00:58:36.333
You can have actually have one very generic agent with a set of tools.
00:58:36.900 --> 00:58:43.800
So that was really the, the beginning of Strands agents with us internally within one of the products I work on.
00:58:43.800 --> 00:58:55.565
Q developer realizing that there was a simpler way and there was actually was actually even, more higher quality because this is now how these models are being trained.
00:58:55.565 --> 00:58:57.865
They're being trained to use tools.
00:58:57.865 --> 00:59:06.300
So we, started to re-implement, throw away some of our agents internally and rebuild on this model driven approach.
00:59:06.733 --> 00:59:14.132
And we were seeing it was a better user experience, it was more accurate, and it was actually faster for us.
00:59:14.132 --> 00:59:14.365
Right.
00:59:14.365 --> 00:59:21.000
It's obviously faster to develop something where you write a prompt and you give it some tools and off you go.
00:59:21.000 --> 00:59:35.900
So that was we started developing it internally for us at AWS, and then, you know, we started seeing such great, impact internally for us that we, we wanted to release it for the, for the community.
00:59:36.132 --> 00:59:56.300
And so this model driven approach is really at the core of it, where it's really focused on let's leverage the power of how powerful these models have become, especially with tool use, and, and get rid of a lot of the complexity that that came from, you know, using earlier models like Claude V1.
00:59:56.300 --> 00:59:56.699
Yeah.
00:59:56.699 --> 01:00:01.565
And that's a great call out that, these lines are now being trained on using tools.
01:00:02.000 --> 01:00:13.666
You know, we'll get into this, but some of the protocols that are coming out, like MCP and A2A lots of acronyms, you know, these are all sort of materialize and they're productive.
01:00:14.065 --> 01:00:20.632
So, you know, there's certainly a lot of interesting directions it can go in, but it all starts with the model.
01:00:20.632 --> 01:00:32.632
Like you said, and having, you know, the, the, the models, the the prompts, the the evals, the tools, like as you're kind of, core building blocks.
01:00:32.632 --> 01:00:46.666
And then from anything there, you could kind of, from those abstractions, you can build applications on top, like what are the like, what are the other core architectural principles behind Strands, and how do they reflect a shift in how these agent systems are being built?
01:00:46.865 --> 01:01:00.065
Well, a couple are one, it's not tied to any the particular model, just that your model have your reasoning capabilities and tool use capabilities.
01:01:00.065 --> 01:01:03.432
And what we're seeing is that lots of models are now being trained that way.
01:01:03.733 --> 01:01:08.833
And lots of models either have sort of a messages API or a tools API.
01:01:08.833 --> 01:01:11.833
In the case of, anthropic APIs.
01:01:12.532 --> 01:01:21.900
And so we were really cognizant early on of what are the interfaces that we're providing for you to be able to plug in your favorite model.
01:01:21.900 --> 01:01:25.766
That could be something that's running locally on your laptop with a llama.
01:01:26.233 --> 01:01:29.233
It could be something that's running on AWS with bedrock.
01:01:29.465 --> 01:01:40.565
But we also partnered with anthropic to provide, anthropic API support, meta for, for their new, llama API.
01:01:40.599 --> 01:01:43.865
And we're working with a lot of other partners to provide additional APIs.
01:01:44.166 --> 01:01:57.800
So we wanted it to be something very, flexible in that in that case of, you know, regardless of what provider you're using, we're finding that there's a lot, more consistency in interfaces to models these days.
01:01:58.266 --> 01:02:05.300
And then we're also seeing more consistency, of course, in, interfaces to tools, like you said with MCP.
01:02:05.632 --> 01:02:24.865
That's been amazing for, having a more standard interface across whatever agent framework you might be happening to use, but then also for the tool author to be able to provide MCP and, servers for, for anybody to use, regardless of their agent framework.
01:02:25.400 --> 01:02:30.733
And so it's been amazing to see the growth in the community of those MCP servers.
01:02:30.733 --> 01:02:40.699
I mean, there's thousands now out on GitHub and, and on, you know, PyPI and and npm, and you can bring any of those to Strands.
01:02:40.699 --> 01:02:40.965
Right.
01:02:40.965 --> 01:02:46.833
So we're trying to support standard interfaces as much as we can.
01:02:46.833 --> 01:02:48.300
It supports open telemetry.
01:02:48.300 --> 01:02:51.800
So you can bring in whatever telemetry provider you prefer.
01:02:52.365 --> 01:02:53.932
And then you can run it anywhere.
01:02:53.932 --> 01:02:55.132
It doesn't need to be.
01:02:55.132 --> 01:03:00.032
It can run on AWS or it can run on, you know, your laptop or anywhere else you need to run it.
01:03:00.632 --> 01:03:12.266
So we wanted to provide something that wasn't tied necessarily to any individual provider, but really instead showcase this, this model driven approach that's been so powerful for us.
01:03:12.632 --> 01:03:34.365
I and that's a that's a very astute point that, you know, you have the model and you can have multiple models and you can have multiple runtimes and, you know, having the power to sort of develop locally and then scale in the cloud, and not be restricted, you know, based on, you know, your code shouldn't be tied to, you know, one specific environment.
01:03:34.465 --> 01:04:12.333
So it's taking a lot like when I looked at when I looked at it, I mean, it seems like it's taking a lot of just software engineering, you know, both, you know, scalable principles and practical, like, easy to develop with and, and I, and I like the other part like you mentioned, it's, it's, it's model driven, but but it can work with multiple models, which is becoming pretty standard for AI application development, even in the if you're doing things like, as a judge or, model cascading and you want to test, you know, cheaper models through some workflows and then, you know, potentially have to fall back to more, expensive ones.
01:04:13.365 --> 01:04:16.266
You know, if the cheaper, fast model is not working as well.
01:04:16.266 --> 01:04:33.132
So it's super flexible and I think it's, it's, it's a really nice reflection of where, AI applications are standardizing their approaches, diving in deeper into MCP and in a to a and these types of things.
01:04:33.132 --> 01:04:35.266
I, I would love to see your take on it.
01:04:35.266 --> 01:04:38.132
Like, where do these like emerging protocols.
01:04:38.132 --> 01:04:45.032
Like I mentioned, MCP and A2A sort of redefine the communication between intelligence systems.
01:04:45.199 --> 01:04:59.666
one of the specific, use cases that we've seen for multi-agent because I think, I think there's, what we've seen is that, agent to agent communication is not always need to be something special.
01:05:00.099 --> 01:05:15.300
So we've had a lot of success with internally with, the agent as tool model where, the agent is, a bit more of a sort of a request response where the sub agent, you're calling it as a normal tool.
01:05:15.300 --> 01:05:19.666
You could put it behind MCP, you give it a prompt, it comes back with a response.
01:05:20.465 --> 01:05:23.666
We have multiple systems internally that, that run that way.
01:05:24.000 --> 01:05:32.500
And some of that is because there are, agents that have specialized knowledge or which really ends up being specialized access to data.
01:05:33.065 --> 01:05:33.465
Right.
01:05:33.465 --> 01:05:44.666
But often what we see is it's also a way, organizationally to, to split up a very complex system into multiple teams.
01:05:45.132 --> 01:05:45.400
Right?
01:05:45.400 --> 01:05:48.266
So at Amazon, we're kind of famous for the two pizza team.
01:05:48.266 --> 01:05:53.500
And, you know, letting teams own, their, their whole system.
01:05:53.800 --> 01:06:07.333
And so in some ways, multi-agent is sometimes more of an organizational, solution where, you know, you have five different teams that want to own some experience or domain expertise in a, in an overall agent system.
01:06:08.365 --> 01:06:14.865
But we do see some more complex, use cases for agent to agent communication.
01:06:15.132 --> 01:06:25.900
So one challenge that we've had with MCP is with MCP being able to model many, many tools.
01:06:26.400 --> 01:06:33.932
So for example, we have an agent internally that models, more than 6000 tools.
01:06:33.932 --> 01:06:38.865
So there was those mapped to a huge number of, of AWS APIs.
01:06:39.365 --> 01:06:42.766
And that's you can't put that in your in your prompt context.
01:06:42.766 --> 01:06:43.199
Right.
01:06:43.199 --> 01:06:51.432
If you have an MCP, server that exposes 6000 tools, you'll probably blow out your context window.
01:06:51.432 --> 01:06:57.932
And if not, your requests are going to be very slow and very inaccurate in terms of what tool it chooses.
01:06:58.465 --> 01:07:15.632
And so we've done a lot of work around agents where, inside of this sort of sub agent, it does a lot of work around selecting the particular tools that you should, exposed to the LLM based on the customer's input.
01:07:16.500 --> 01:07:20.965
So, you know, if they asked about S3, it's probably going to be an S3 API.
01:07:20.965 --> 01:07:29.599
If they asked about EC2, it's going to be an EC2 API, and sometimes it's a little bit more, ambiguous.
01:07:29.599 --> 01:07:32.599
Like they might ask about a function and not say lambda.
01:07:32.900 --> 01:07:54.400
So, being able to figure out, you know, what are those maybe top five most relevant tools for the LLM is something that that agent to agent communication has been really helpful for, for us, and being able to, maintain state across a session in that sub agent in the agent as tool use case.
01:07:54.400 --> 01:07:58.766
Often you don't necessarily save state in that subnets that request response.
01:07:58.766 --> 01:07:59.833
And I'm done.
01:07:59.833 --> 01:08:09.465
But we have some of those use cases especially around that, that selection of within 6000 tools for the session, becomes very important.
01:08:09.865 --> 01:08:14.166
We're doing a lot of work with, we're doing some work with A2A.
01:08:14.199 --> 01:08:18.300
We have an example of using Strands with A2A.
01:08:19.100 --> 01:08:29.500
And that enables you to do, you know, to kind of hook into that entire ecosystem of having, agents talk with agents that could be implemented in any agent framework.
01:08:29.500 --> 01:08:29.800
Right?
01:08:29.800 --> 01:08:35.332
You might have some link chain, you might have some crew, you might have some Strands, and they can all communicate together.
01:08:35.632 --> 01:08:49.733
But we're also doing a lot of work, in the MCP community upstream to, try to evolve that, to be able to support agents, community over the MCP protocol, which is super interesting.
01:08:50.166 --> 01:09:11.100
So one example is, we a few weeks ago now, I think we got merged a new feature in the MTP spec called elicitation, where the MCP server can respond back with and elicitation along the lines of I need this information from the end user.
01:09:12.100 --> 01:09:17.233
And the the core agent, you know, goes off and and returns control back to the human user.
01:09:18.000 --> 01:09:21.166
And that's going to be super important, of course, for agent agent communication.
01:09:21.699 --> 01:09:27.132
So I think it's very interesting to see, you know, both how these both evolve, right.
01:09:27.132 --> 01:09:36.500
But also how we start to think about the lines blurring between, communicating with agents and communicating with tools if they're in the same interface.
01:09:37.233 --> 01:09:54.100
So at a high level and, and tell me if I'm oversimplifying, but, would you categorize MCP as the best way to, for, for agents to communicate with tools and then who is the best way to have, like these multi-agent communication today?
01:09:54.100 --> 01:10:07.265
Yeah, that's that's the, the pattern that I tend to see, is where, you know, it's really simple to wrap, MCP the interface around an agent.
01:10:07.265 --> 01:10:11.033
You know, it's, it's an API, that you can stand up.
01:10:11.033 --> 01:10:22.233
So we've had customers have success with, for example, putting an agent inside of a Lambda function and then just putting, you know, making that an API.
01:10:22.365 --> 01:10:24.500
That is MCP compliant.
01:10:24.500 --> 01:10:29.733
Some of the work that's been done upstream and MCP, I think has more enabled this.
01:10:29.733 --> 01:10:40.399
So in the original spec, it was just, mostly local MCP servers, which was great for, you know, Ides and chat clients.
01:10:40.399 --> 01:10:46.332
And if you look at those thousands of MCP servers out there, they're not they're not actually servers.
01:10:46.332 --> 01:10:53.265
They're really just local process with, you know, standard IO streams to your, your chat client locally.
01:10:53.600 --> 01:11:01.365
But where I see the industry going is really embracing, what we call the remote MC server.
01:11:01.365 --> 01:11:01.600
Right.
01:11:01.600 --> 01:11:04.466
The MCP server that's just behind an API.
01:11:04.466 --> 01:11:29.132
And that's really just become possible, I think, in the last couple of months, even, as the upstream community has been, adding new ways of, of building scalable MCP servers that are remote, with, something called a streamable Http transport, which basically just means, you know, putting it behind, more familiar Http APIs.
01:11:30.332 --> 01:11:30.899
Yeah.
01:11:30.899 --> 01:11:39.100
And that's definitely a big boost for, performance and just some other things like that that you would expect.
01:11:39.265 --> 01:11:41.166
An API to, to, to deliver.
01:11:41.166 --> 01:11:44.832
So that's a really exciting, drop for sure.
01:11:44.832 --> 01:11:49.832
And, and like you mentioned, you know, it just became available in the last few months every time.
01:11:50.065 --> 01:12:08.932
And just listening to these episodes back in January, like, there's there's been so much, innovation even since then, I think, like the rate of development, in AI obviously correlated with the, you know, I code gen tools, but, you know, and then, of course, the excitement and the value, it's bringing and I.
01:12:09.132 --> 01:12:09.332
Yeah.
01:12:09.332 --> 01:12:48.000
And, and when you look at, the frameworks that, that now, you know, companies like AWS are delivering, like you said, you've been working on this since, 2023, but it's on the, on the, on the Q developer side and the, the the amount of applications you've seen built then versus now, obviously a huge change in the, in the amount, but also the requirements around them, the, you know, being able to handle the scale of these data processing engines and databases, and, and making it streamlined for, enterprise application development.
01:12:48.000 --> 01:12:53.265
And then of course, like even even the SMB and mid-market ones as well that have large data volumes.
01:12:53.565 --> 01:12:56.132
So that's been just really fun to see.
01:12:56.132 --> 01:13:10.132
And I felt like it was a great turning point in the industry when you saw, Strands agents come out because it's just sort of this kind of light layer of abstraction to build abstractions to build these applications.
01:13:10.132 --> 01:13:16.533
It's, and it's really grounded in this, this idea, like you said, that, you know, we start with the models, right?
01:13:16.533 --> 01:13:19.365
Which is a powerful way to look at it.
01:13:19.365 --> 01:13:20.065
Definitely.
01:13:20.065 --> 01:13:34.432
And I think one of the interesting things that I've seen over the last, I would say six months maybe is, how differently we think about this model driven approach in terms of even just information, right?
01:13:34.432 --> 01:13:37.132
When we we all kind of started out building agents.
01:13:37.132 --> 01:13:47.000
It was all about rag, you know, right before you called to the model, you always supplemented your context with pulling from some vector DB, right.
01:13:47.632 --> 01:13:58.432
And one of the things I'm starting to see now is, what, internally, one of my colleagues calls retrieval as a tool as opposed to retrieval, augmented generation.
01:13:59.065 --> 01:14:03.966
I don't love the acronym Rat, but but there it is.
01:14:04.500 --> 01:14:17.332
So, you know, we're seeing a lot more of, you know, don't actively give information to the model that might be too much information or even information that it doesn't need.
01:14:17.332 --> 01:14:24.233
Like if it has enough information in its training data, you might actually make things worse by proactively giving it information.
01:14:24.666 --> 01:14:29.166
Instead, create a tool that that can retrieve information for it.
01:14:29.600 --> 01:14:41.332
So as opposed to proactively having kind of a prompt pipeline where you always augment that prompt, instead provide that vector DB as a tool where it can retrieve documentation.
01:14:42.466 --> 01:14:51.533
And I think one of the what are the places where we're seeing this that's interesting is in actually a lot of the development tools with MCP.
01:14:51.966 --> 01:15:12.666
So we're seeing a lot more of what I call documentation MCP servers, where, you know, in the agent space especially I was looking at the training dates recently of the latest Claud models and since, Claude 3.7 came out, there's been like ten new agent frameworks.
01:15:13.100 --> 01:15:20.632
And then even since sonnet four came out, I think I read that their training data ended in, in March of this year.
01:15:20.832 --> 01:15:24.233
Strands came out in May and so it doesn't even know about Stands yet.
01:15:24.600 --> 01:15:36.832
And so, we're starting to see a lot of these, you know, projects provide LLM tags so you can retrieve those through an MCP server and again, the MCP server is a tool.
01:15:36.832 --> 01:15:48.000
So it's letting the LLM decide, when do I actually need this information for my task versus have proactively throwing a bunch of information at it.
01:15:48.000 --> 01:16:01.832
So that that's been a big change in sort of those data access patterns that we've seen from moving from that prompt pipeline where you would try to get it every time, and then letting the model decide when it needs that information.
01:16:01.832 --> 01:16:04.233
Absolutely. And timely.
01:16:04.233 --> 01:16:13.332
Because just, at least from the time we're recording this episode, last week, in the Strands GitHub repo.
01:16:13.332 --> 01:16:14.932
And that's one of the other great things about Strands.
01:16:14.932 --> 01:16:19.233
There's so many great actionable, easy to run, examples on the GitHub.
01:16:19.233 --> 01:16:22.265
I'll have that down in the, the link to that in the show notes.
01:16:22.265 --> 01:16:34.632
But there was just recently one that came out with, Amazon Aurora, de SQL, running through Strands agents and just like you mentioned, it's, it's retrieving data with these tool calls.
01:16:34.966 --> 01:16:35.332
Right?
01:16:35.332 --> 01:16:41.533
So you look at the architecture diagram and it's and it's reading read only query, it's doing transactions.
01:16:41.533 --> 01:16:48.600
There's get schema and different just high level tool call APIs that interact with the database.
01:16:49.033 --> 01:16:55.733
And when, when Mark Broker he was he was on What's New in Data last season.
01:16:55.765 --> 01:17:05.233
And, you know, we were chatting about how agents and, serverless databases actually are very complementary technologies.
01:17:05.365 --> 01:17:08.166
So it's very cool to see sort of the materialization here.
01:17:08.166 --> 01:17:20.832
And, and like you said, it's it's it's super powerful too, rather than having a rag and trying to like populate your, your, your prompts with, with contacts kind of manually or, or overloading it.
01:17:21.533 --> 01:17:32.666
The, the agent can kind of decide when it wants to make a tool call, to, to retrieve very specific data with specific parameters and it makes the application more deterministic.
01:17:33.065 --> 01:17:37.699
And we've also seen that, it also makes it much more generic.
01:17:38.033 --> 01:17:38.600
Right.
01:17:38.600 --> 01:17:52.332
You don't have to have a very specific prompt pipeline where you determine the, the query to the database ahead of time and then add the data to the, to the prompt.
01:17:52.733 --> 01:18:08.565
You're able to apply these agents to much more generic problems than you could before by effectively teaching the model how to query, and then giving it some, some boundaries, right, giving it some guidelines.
01:18:09.265 --> 01:18:16.199
But generally it's, it's much more, flexible for, you know, a lot of different tasks that you could throw at it.
01:18:16.565 --> 01:18:17.765
Yeah, absolutely.
01:18:17.765 --> 01:18:23.865
And, and it gives a good guardrails for, for things like data access.
01:18:24.265 --> 01:18:39.666
You know, even in that example, there's, you know, it specifies, you know, read-only queries and, retrieving schemas, but then it bubbles up into this powerful experience where, you know, the agent can, you know, get all the tables and do some operations.
01:18:39.865 --> 01:18:59.300
But but a smart, engineer who, who understands, you know, how to set up database applications can still predefine those tool calls in a way that's that's safe and scalable, and you can even do your, your explain and analyze to to to make sure those are efficient queries.
01:18:59.600 --> 01:19:06.365
Or you have you might have agents interacting with systems that that with databases that are trafficked on other applications as well.
01:19:06.632 --> 01:19:09.033
So it's a very powerful concept.
01:19:09.033 --> 01:19:22.832
And it's so nice to see in very, succinct and well-organized lines of code, how it can be implemented, in that, in that GitHub repo, along with, with a lot of other fun examples.
01:19:22.832 --> 01:19:32.500
And since, you know, we're on that topic, like, what's one of your favorite tutorials, in that, in that GitHub repo that, that you like to call out?
01:19:32.632 --> 01:19:34.666
It's actually not in the GitHub repo.
01:19:34.666 --> 01:19:40.365
I put it in the, in our blog post, about the Strands launch.
01:19:40.800 --> 01:19:47.632
When we were, we decided to open source Strands.
01:19:47.632 --> 01:19:56.233
It had a different code name internally, right, that we, and so we were trying to come up with, okay, what are we going to call this, this open source project?
01:19:56.233 --> 01:20:03.132
We need to come up with a name and, and, you know, the lawyers have to sign off on it, and it, you know, it can't be too similar to any other companies or whatever.
01:20:03.699 --> 01:20:09.899
And, as it turns out, in the AI space, all words are taken.
01:20:10.233 --> 01:20:13.166
Everybody's got, you know, blank AI.
01:20:13.166 --> 01:20:13.865
Right?
01:20:13.865 --> 01:20:22.800
So, so we were having this trouble where, you know, we do a brainstorming session where we come up with, oh, this would be a really cool name.
01:20:22.800 --> 01:20:24.500
That would be a really cool name.
01:20:24.500 --> 01:20:30.033
And then we'd go and search on it online and it would say, oh, of course, of course it's taken already. Right?
01:20:30.033 --> 01:20:42.265
Or we find one that didn't have a very, very clear company that was already using it, but someone had already registered the domain that we would want to use, or the GitHub org that we would want to use.
01:20:42.733 --> 01:20:51.666
And so I wrote, with Strands a little bot that would, help me to come up with names.
01:20:51.666 --> 01:21:01.332
So in the prompt I would give it some like some guardrails around, you know, how we name things and then, some ideas of themes that I really liked.
01:21:01.500 --> 01:21:04.233
Like we had, you know, a DNA theme going.
01:21:04.233 --> 01:21:08.100
We had a, like a sewing theme going as well for a while.
01:21:08.399 --> 01:21:17.265
We had a, like a car theme going for a while where you would think about, like a motor going around a root or an axle going around and around.
01:21:17.800 --> 01:21:21.932
Anyway, so I gave out all this ideas and then I gave it a few tools.
01:21:21.932 --> 01:21:32.233
I wrote an MCP server, for my favorite thesaurus website, because I always felt that helps when I'm brainstorming.
01:21:32.699 --> 01:21:40.600
I wrote a, a, domain name search, you know, figure out whether it was registered or not.
01:21:41.033 --> 01:21:51.500
I discovered that, through this, I didn't know this, who is is no longer the hot thing for figuring out for domain name registration.
01:21:51.932 --> 01:21:56.533
It's now Rdapper or RDAP.
01:21:56.865 --> 01:22:04.332
And as it turns out, the.dev TLD on my Google doesn't support who is, it only supports RDAP.
01:22:04.332 --> 01:22:08.899
And so I was using this who is MCP server that I found, you know, on GitHub.
01:22:09.265 --> 01:22:15.399
And it nothing.dev was coming back and of course we wanted to use it.
01:22:15.666 --> 01:22:18.632
We wanted to to, you know, explore using that .Dev.
01:22:18.632 --> 01:22:21.300
This was a very developer focused project, right.
01:22:21.300 --> 01:22:28.365
And then I also had like the GitHub MCP server where it could look up, possible org names.
01:22:28.899 --> 01:22:34.033
And that helped us just churn through ideas, which was really cool.
01:22:34.166 --> 01:22:36.733
And we were able to prove like, hey, this one doesn't work.
01:22:36.733 --> 01:22:40.332
This one doesn't work because it's got these, conflicts already.
01:22:40.832 --> 01:22:42.666
And that was just very exciting.
01:22:42.666 --> 01:22:45.832
You know, one, it was super easy to write with Strands.
01:22:45.832 --> 01:22:48.832
It was, I don't know how many lines of code, but really small.
01:22:49.033 --> 01:23:04.800
And then also being able to take advantage of, some of the MCP servers that are already out there, like, I was able to pull the GitHub one off the shelf, and then it was really easy to write up one where I was, web scraping.
01:23:04.800 --> 01:23:07.666
My favorite this the source website to be unnamed.
01:23:07.666 --> 01:23:14.265
And then, you know, adding, this domain name lookup, one that I found.
01:23:14.265 --> 01:23:19.466
So, you know, it was it was just so easy to kind of pull these things together.
01:23:19.899 --> 01:23:27.132
And the the most work that I was really doing was thinking about what are the the real kind of business value.
01:23:27.132 --> 01:23:27.466
Right.
01:23:27.466 --> 01:23:37.632
I was thinking about what are the, themes that we want to explore, what are the names that we really like or don't like, as opposed to doing all this manual lookup.
01:23:37.632 --> 01:23:39.500
So it's really, really fun for me.
01:23:39.500 --> 01:23:48.399
And that that also speaks to, the simplicity and how it was just able to aid your creative process.
01:23:49.166 --> 01:24:02.733
Well, while you were working through a higher level challenge was like naming something and that's, you know, and then, for, for, for AWS, the the scope of naming something obviously has even more, challenges to it.
01:24:02.733 --> 01:24:20.800
Which, which, which, you know, you're able to, to navigate things to, you know, some, some clever, both brand brand thinking and also tooling, but, you know, and it's not going to be we're not far away from a future where, like you said it, you just you you have this tactical thing you have to get done, and.
01:24:20.800 --> 01:24:24.065
Okay, let's let's outsource it to some agents.
01:24:24.500 --> 01:24:28.132
That can do a lot of the, hard work for me.
01:24:28.132 --> 01:24:33.199
And, you know, I didn't know about the RDAPR whois thing.
01:24:33.432 --> 01:24:35.132
And so, yeah, I find it.
01:24:35.132 --> 01:24:39.600
But, you know, it sounds like it's something that, you know, you discovered in the process.
01:24:39.600 --> 01:24:44.132
Maybe your agents discovered in the process and told you, Yeah.
01:24:44.132 --> 01:24:45.932
No, exactly.
01:24:45.932 --> 01:24:50.199
And, and, you know, there's one end where.
01:24:50.300 --> 01:25:03.865
Yeah, you'll you'll use these products which sort of wrap the agent experience and make it easy for you for, for for folks who are, you know, non-technical and, you know, they'll just type in some, some instruction and it's going to deploy a fleet of agents to get it done.
01:25:04.265 --> 01:25:23.565
And then, on the other hand, you're going to have developers who either through hand coding or, AI code gen, can can use like a nice wrapper, framework to, to, to build this stuff for themselves, at a, at a rate which was just not possible before.
01:25:23.932 --> 01:25:37.199
And it's from that perspective, it's, it's super exciting both like as for, for like the non-technical consumers of AI and, and the builders who, who, who want AI at their, at their fingertips.
01:25:38.233 --> 01:25:44.265
So, I, I totally recommend, we'll have a link to that, to your, to your blog in the show notes as well.
01:25:44.966 --> 01:25:46.800
And it's in to the, to the GitHub repo.
01:25:46.800 --> 01:25:55.932
But I always just encourage people like the people who are creating these frameworks, including yourself, are making it easy to run locally and then deploy in the cloud.
01:25:55.932 --> 01:26:02.332
So start with your your laptop, your laptop is, most people's laptops are pretty powerful these days.
01:26:02.332 --> 01:26:20.699
There's a lot of great, tooling, but especially with Python, to build really nice applications that are portable, that, you know, you can you can really get far on your laptop and then still be in a state where you can deploy to the cloud without changing too much, in terms of, you know, just just environment variables.
01:26:20.699 --> 01:26:33.065
So I do totally recommend everyone to just just get, get hacking and, you know, hack around and play with these frameworks and, and try it out because you, because people don't even understand what they can do.
01:26:33.466 --> 01:26:37.132
You know, just because it just seems impossible.
01:26:37.132 --> 01:26:41.932
Like, even if you've been in software engineering for 25 years, I hear that a lot.
01:26:41.932 --> 01:26:44.100
People just didn't understand what was possible.
01:26:44.100 --> 01:26:45.666
And it's it's so powerful.
01:26:45.666 --> 01:26:56.132
Like, you know, you kind of just have to go, go do it and play around and get that, that empirical knowledge, yourself and Strands is a great, great framework for that.
01:26:56.699 --> 01:27:01.533
You know, looking ahead like, you know what foundational.
01:27:01.533 --> 01:27:09.000
But so we're talking about all the amazing stuff that's there, but what are the foundational building blocks that are still missing from the agent ecosystem?
01:27:09.365 --> 01:27:24.265
Well, I think one is that I'm kind of excited about is thinking about how can we reuse these agents across, other developers like, you know, we think about we have libraries, right.
01:27:24.265 --> 01:27:30.533
And, and we've created this amazing ecosystem of libraries across all these package managers.
01:27:30.533 --> 01:27:37.265
But I also think that agents are kind of the new, you know, MCP library and city servers are the new libraries.
01:27:37.265 --> 01:27:39.399
Agents are going to be the new libraries.
01:27:39.399 --> 01:27:56.666
And so how do we share these or, you know, make them available over APIs or something where, you know, as a developer creating an application, maybe what I'm doing is pulling together a set of MCP servers and pulling together a set of agents.
01:27:57.300 --> 01:28:25.533
You know, I've seen that, you know, even just with my little naming bot, I was seeing that the power of being able to just pull in these, these MCP servers, whereas before, you know, I think about API integration work using some like a, like the GitHub library, for example, would still take me a long time to build to write all that client code and, MCP servers, I just pulled it in and, and, you know, wrote a prompt.
01:28:25.932 --> 01:28:30.765
And I think that sharing agents could potentially be even more powerful.
01:28:31.332 --> 01:28:38.800
So I'm, I'm looking forward to that world where, you know, we we start to share them as well, like libraries.
01:28:38.966 --> 01:28:47.733
And then I think also, the it's still very unsolved about building up entire applications around agents.
01:28:48.865 --> 01:28:53.100
You know, you still have to do a lot of work to, to put it behind an API somewhere.
01:28:53.100 --> 01:28:56.800
You have to, you know, figure out authentication.
01:28:57.199 --> 01:29:04.233
You have to figure out how does my my react code in the front end interact with this agent?
01:29:04.233 --> 01:29:06.265
What does it look like in the UX?
01:29:06.265 --> 01:29:10.033
I think we're still very stuck in the, generative.
01:29:10.033 --> 01:29:12.399
The AI means chat world.
01:29:12.399 --> 01:29:17.000
So I'm looking forward to, you know, the UX around agents evolving as well.
01:29:17.365 --> 01:29:25.100
Where we start to think about, how could agents help in my work other than having a conversation with it?
01:29:25.233 --> 01:29:28.565
So I think there's just so much space.
01:29:28.565 --> 01:29:36.699
It's and it's like you said, this space is changing so fast that, you know, you can't even imagine what's going to come in three months, right?
01:29:37.332 --> 01:29:37.932
Yeah.
01:29:37.932 --> 01:30:06.000
And I, I really like your point about just being able to share agents sort of like sharing a, an API, but you have everything, ready for you to use there to execute that agent and, and also, one of the things that's really, it seems critical for AI applications is, you know, the traces, the evals, the, you know, you're kind of your integration tests for, for the AI, like, changes to the, to the prompts.
01:30:06.000 --> 01:30:09.733
How does that actually impact the, the the outputs?
01:30:09.733 --> 01:30:21.432
So when I come back to, you know, your idea of, of sharing agents, doing that in a way that also like, delivers like the trust that it's that it's working the right way.
01:30:21.666 --> 01:30:25.233
And I think we're all as an industry trying to figure out like, what are that?
01:30:25.233 --> 01:30:27.466
What are those sort of units of trust?
01:30:27.466 --> 01:30:36.265
And of course, you know, you have LLMs as a judge and in confidence scores and all these things that are really useful and, you know, just borrowed from, from machine learning.
01:30:36.365 --> 01:30:42.265
I always when I'm working with software engineers who haven't done agentic or AI development before, I'm like that.
01:30:42.399 --> 01:30:43.733
That's where you should really start.
01:30:43.733 --> 01:30:47.166
Like how we come up with your framework for deciding like what's correct.
01:30:47.432 --> 01:31:04.533
One of the things I've noticed working with teams at AWS who are starting to build generative AI capabilities is exactly that hurdle that, us as engineers expect testing to look like a define an input, I define an output.
01:31:04.632 --> 01:31:06.932
I get that output when I provide the input.
01:31:06.932 --> 01:31:08.632
Then things are working.
01:31:08.632 --> 01:31:12.033
And obviously generative AI is not like that.
01:31:12.033 --> 01:31:18.533
You have to you can't rely on it providing you ever the same answer as what you got before.
01:31:18.966 --> 01:31:32.632
And so thinking about, what success criteria looks like, what good looks like is it know 95% accuracy of tool selection.
01:31:32.632 --> 01:31:36.065
Is it, 95%.
01:31:36.300 --> 01:31:41.600
You know, LLM judge saying this is close enough to your ground truth answer.
01:31:42.265 --> 01:31:45.733
How do you even think about creating test sets and ground truth?
01:31:45.932 --> 01:31:47.300
Answers.
01:31:47.300 --> 01:31:51.399
These are problems that, as an industry software engineers have never had to deal with.
01:31:51.399 --> 01:31:54.265
This has been scientist ML scientist stuff.
01:31:54.265 --> 01:31:55.166
Right.
01:31:55.166 --> 01:32:00.666
And so we're all kind of needing to become scientists a little bit.
01:32:00.666 --> 01:32:01.000
Right?
01:32:01.000 --> 01:32:06.699
And we, we're not used to, using the scientific method we used in, learned in grade school at work.
01:32:06.699 --> 01:32:07.432
Right.
01:32:07.432 --> 01:32:12.733
Having a hypothesis and coming up with a test and, and executing that test.
01:32:13.265 --> 01:32:25.632
And so, I think that, evaluation tools today are still really hard for engineers to grok in a lot of ways.
01:32:25.632 --> 01:32:32.832
You know, even that question of, well, how do I come up with the test set that this evaluation tool is going to go through is really hard?
01:32:32.966 --> 01:32:41.265
I think there's so much room for improvement there that speaks to, you know, the fact that we are complete amateurs in space.
01:32:41.300 --> 01:32:43.600
You know, I don't have a I don't have a science PhD.
01:32:43.600 --> 01:32:57.166
So, and then I think also mapping it to, the reality of the, you know, the way that your users are actually interacting with your system.
01:32:57.166 --> 01:32:57.932
Right?
01:32:57.932 --> 01:33:05.332
One thing that we say at AWS, for any service we launch, they're going to be customers that use it in ways you never expected.
01:33:05.733 --> 01:33:08.533
And I think that's probably true of any application.
01:33:08.533 --> 01:33:15.466
And so you have before you launch something, you have this guess as to how people are going to use your system.
01:33:15.466 --> 01:33:34.432
And you create a test set based on that guess, but then you have to follow back and you have to validate, okay, what kinds of questions are people actually asking of my of my agent, or what kind of inputs are they actually providing and try to match that distribution in your test set?
01:33:35.332 --> 01:33:37.332
And that can also be really hard, right?
01:33:37.332 --> 01:33:44.065
How do you how do you think about that kind of matching that data, the live data set with what you're testing against?
01:33:44.632 --> 01:33:45.265
Yeah.
01:33:45.265 --> 01:33:47.832
And we're sort of scraping the surface of this.
01:33:47.832 --> 01:34:00.565
But you know, if I were to ask you generally like what's your advice for software engineers, whether they're new or experienced or just fundamental computer science background, what's your advice to software engineers who are breaking into AI engineering?
01:34:01.265 --> 01:34:04.832
One is, try Jupyter notebooks.
01:34:06.865 --> 01:34:13.399
That's something I had never I had never heard of, honestly, coming out of, you know, a computer science program.
01:34:13.765 --> 01:34:21.533
And then all of a sudden, I'm working with these, you know, ML and AI scientists, and they're all sending me these Jupyter notebook files, you know?
01:34:21.932 --> 01:34:28.865
So, we do actually have a lot of, Jupyter notebook examples in the Strands examples repo.
01:34:28.865 --> 01:34:30.666
So that's a good place to start.
01:34:30.666 --> 01:34:52.332
But I think also thinking about, again, kind of how, how testing is going to look like one thing that I often see from myself as well as other engineers is, what I started call vibe checking, which is, you know, we know about vibe coding, but how do you know when your thing works?
01:34:52.332 --> 01:34:54.065
How do you know when your agent works?
01:34:54.065 --> 01:35:01.100
You just kind of give it a couple of inputs, and then you personally like the outputs, and then you send it on its way.
01:35:01.100 --> 01:35:01.865
Right.
01:35:01.865 --> 01:35:10.632
Starting to think about, okay, how can I actually write these down and think about what do I expect the answer to be?
01:35:10.632 --> 01:35:14.899
And thinking about how you would compare, you know, good versus not good.
01:35:15.132 --> 01:35:22.033
So I think that there is a lot of kind of intuition that you have about what is good and what's hard is writing that down.
01:35:22.500 --> 01:35:28.132
And so trying to kind of look through what, what you would think of as good.
01:35:28.132 --> 01:35:35.632
We actually have a, we have another product called I'm not going to get it right, but it's something like bedrock automated reasoning.
01:35:35.632 --> 01:35:45.966
And one of the challenges that we have with that product is basically looks for hallucinations in, large language model outputs.
01:35:46.432 --> 01:35:50.466
But one of the things you have to do ahead of time is codify.
01:35:50.466 --> 01:35:51.865
What is the truth?
01:35:51.865 --> 01:35:55.899
It can only tell you it's wrong if you've already told it the truth.
01:35:56.166 --> 01:36:00.600
And then it uses mathematical proofs and automated reasoning to determine whether it is true or not.
01:36:00.600 --> 01:36:02.533
When it comes out of the model.
01:36:02.533 --> 01:36:05.500
And one of the things that we found is people don't know it's true.
01:36:05.500 --> 01:36:13.166
And so what's really interesting is, you start with here's what I think is true.
01:36:13.399 --> 01:36:24.265
And then it kind of takes you through some scenarios and says, okay, if this is the case and this is the case based on what you've told me, this would be the result.
01:36:24.265 --> 01:36:26.399
And then you say, no, no, no, no, that's not right.
01:36:26.399 --> 01:36:37.100
So one example is, if you give it a tax code, and it does it best to take a tax code and turn it into mathematical models, then it'll give you.
01:36:37.100 --> 01:36:49.466
Okay, if you are, a married couple with two dependents, then your deduction is ex and someone who's a subject matter expert on on actually doing that math can come in and say, no, no.
01:36:49.466 --> 01:36:50.365
And this is why.
01:36:50.365 --> 01:36:56.166
And it can go back and, and, regenerate those, those models.
01:36:56.166 --> 01:37:01.699
And I think that's a very interesting for how we think about what good looks like in our application.
01:37:02.132 --> 01:37:05.966
You know, play, play kind of a game with yourself.
01:37:05.966 --> 01:37:06.565
Okay.
01:37:06.565 --> 01:37:10.932
If this was the input and that was the output, what would I like about it or not like about it?
01:37:10.932 --> 01:37:13.932
And then that effectively is you're oh I'm judge prompt.
01:37:14.166 --> 01:37:14.699
Right.
01:37:14.699 --> 01:37:44.699
And so if you're able to give those kind of self introspect and come up with some of those, those guidelines and that can really help you to build the application you want to build, that, that follows what, what you, you know, what you want to put out into the world, Yeah, I love the I love the, term vibe checking, you know, equally as important as the, the the vibe coding and, in the AI engineering process and, you know, we.
01:37:44.699 --> 01:37:51.132
Yeah, you know, we we as humans, we have this natural reflex to be really good at kind of critiquing, critiquing what we don't like.
01:37:51.199 --> 01:38:01.699
But, you know, it's funny how just having, like, a positive mindset and defining what's good, in a very structured way, is actually very productive for AI, right?
01:38:01.899 --> 01:38:09.600
So, you know, when you're coming up with your, your ground truth and you're coming up with, just ways to, to to define that clearly.
01:38:09.600 --> 01:38:17.632
And like you said, it takes it takes intuition about the core problems that your AI application, is solving. Right.
01:38:17.632 --> 01:38:23.966
And I think that kind of forces everyone to be, you know, even on the engineering side, better, better product thinkers.
01:38:24.000 --> 01:38:34.332
Because ultimately, yes, having that that domain expert, who can define what's good, and do that up front and think about the thousand scenarios of generating good outputs.
01:38:34.765 --> 01:38:41.733
That's, that's ultimately going to be super helpful, and make the AI application that much better.
01:38:41.899 --> 01:38:56.765
So yes, great, great advice to software engineers who typically think the other way, which is like, hey, I'm going to develop all the code first and then test. When it's yes, I know, it's almost like test driven development could make a comeback.
01:38:56.765 --> 01:39:00.632
It could it could make a big comeback with with AI applications.
01:39:00.699 --> 01:39:05.500
Clare, it was super great having you on this episode of What's New and Data.
01:39:05.666 --> 01:39:08.365
Where can people continue to follow along with your work?
01:39:08.365 --> 01:39:11.365
I'm on LinkedIn and I'm on blue Sky.
01:39:11.832 --> 01:39:12.500
Excellent.
01:39:12.500 --> 01:39:24.332
We'll have links to, Clare Liguori's Blue Sky and her LinkedIn below, along with, her awesome blog post going through and demonstrating Strands agents.
01:39:24.765 --> 01:39:28.500
And, thank you to the listeners for for tuning in today.
01:39:28.500 --> 01:39:29.899
This is super fun episode.
01:39:29.899 --> 01:39:32.865
A lot of great insights. Clare.
01:39:29.899 --> 01:39:32.865
Thank you again for joining.
01:39:32.865 --> 01:39:33.699
Thanks for having me.