Harness Engineering for Reliable, Governed AI Agents

WEBVTT
NOTE
Transcription provided by Podhome.fm
Created: 09/19/2026 20:34:26
Duration: 3811.207
Channels: 1
1
00:00:05.120 --> 00:00:08.000
Hello, and welcome to the AI Engineering Podcast,
2
00:00:08.000 --> 00:00:12.160
your guide to the fast moving world of building scalable and maintainable AI systems.
3
00:00:19.285 --> 00:00:35.630
Your host is Tobias Maci, and today I'm interviewing Nakunj Bajaj about the challenges of keeping your agents reliable when the underlying models aren't and how your harness can help. So, Nikunj, can you start by introducing yourself? Thank you so much for having me here, Tobias. I'm Nikunj, co founder and CEO at TrueFoundry.
4
00:00:35.870 --> 00:00:39.790
My background is in machine learning, used to lead one of the conversational AI teams at Meta.
5
00:00:40.335 --> 00:00:50.975
And prior to that, have worked at the intersection of building ML models and ML infra at a couple of other startups as well. And do you remember how you first got started working in the ML and AI space?
6
00:00:51.375 --> 00:00:52.335
Sure, I do.
7
00:00:53.010 --> 00:00:58.289
My my absolute first exposure to ML and AI happened during undergrad,
8
00:00:58.289 --> 00:01:02.530
and this was not through ML and AI, but the underlying concepts
9
00:01:02.530 --> 00:01:08.765
in ML and AI that was through linear algebra. Okay? At in my freshman year during undergrad,
10
00:01:09.244 --> 00:01:13.885
I got a chance to implement some of these matrix decomposition algorithms
11
00:01:14.125 --> 00:01:15.885
on top of FPGAs,
12
00:01:15.885 --> 00:01:18.365
which are like free field programmable gate arrays.
13
00:01:18.604 --> 00:01:20.045
And when you have to
14
00:01:20.540 --> 00:01:23.340
really break down these complex algorithms
15
00:01:23.420 --> 00:01:28.380
into their unit of operation to be able to run them efficiently on FPGAs,
16
00:01:28.460 --> 00:01:37.105
you understand what's happening under the hood. And I think that learning really exposed me to machine learning, and then it continued through my undergrad and the rest of my career.
17
00:01:37.585 --> 00:01:49.105
In terms of what you're building now, you mentioned that you're at TruFoundry, you've built the TruForge harness. I'm wondering if you can just start by giving a bit of an overview of what is your current area of focus and
18
00:01:49.185 --> 00:01:51.265
how did you end up working in this space?
19
00:01:52.530 --> 00:01:53.330
Absolutely.
20
00:01:53.330 --> 00:01:53.890
Yeah.
21
00:01:54.290 --> 00:01:55.010
So
22
00:01:56.130 --> 00:02:04.130
very quickly, I describe what is TrueFoundry and what is TrueForge and I will talk a little bit about how I personally got interested in this entire area.
23
00:02:05.615 --> 00:02:06.895
So TrueFoundry
24
00:02:06.975 --> 00:02:09.535
is the control plane for enterprise AI.
25
00:02:09.855 --> 00:02:10.655
This is
26
00:02:10.975 --> 00:02:13.935
one pipe through which all the enterprise
27
00:02:14.495 --> 00:02:21.630
token traffic is supposed to flow. And when I say token traffic I mean traffic including calls to models,
28
00:02:22.590 --> 00:02:23.870
to MCPs
29
00:02:23.870 --> 00:02:25.550
and calls to agents.
30
00:02:26.110 --> 00:02:28.510
Once all of the organizations
31
00:02:28.510 --> 00:02:31.630
traffic flows through a common pipe a common gateway
32
00:02:32.485 --> 00:02:34.165
then you get
33
00:02:34.645 --> 00:02:36.165
observability unlocked,
34
00:02:36.245 --> 00:02:40.085
you get the reliability unlocked, you get cost optimization unlocked,
35
00:02:40.485 --> 00:02:41.685
you get security
36
00:02:41.845 --> 00:02:49.200
completely taken care of, and lastly you get your identity management taken care of. Right? So it basically solves for these five pillars,
37
00:02:49.440 --> 00:02:50.480
observability,
38
00:02:50.480 --> 00:02:51.440
reliability,
39
00:02:52.080 --> 00:02:53.280
cost management,
40
00:02:53.280 --> 00:02:54.240
security,
41
00:02:54.480 --> 00:02:55.520
and identity.
42
00:02:55.680 --> 00:03:01.805
Okay? So that's what true Foundry is about. It's an it's a control plane to enterprise AI implemented
43
00:03:01.805 --> 00:03:03.485
through an AI gateway.
44
00:03:04.365 --> 00:03:09.965
And TrueForge on the other hand is actually an open source agent harness.
45
00:03:10.605 --> 00:03:17.100
And the way we think about this is if you think about building an agent as a car,
46
00:03:17.340 --> 00:03:31.545
we think of the model as the engine, but everything else around the around the model, the engine is essentially the agent harness. Right? So it means how do you steer it? How do you put brakes on it? Right? How do you have a dashboard where you can see everything
47
00:03:31.705 --> 00:03:36.265
that's happening? Right? Those are all components of an agent harness.
48
00:03:37.385 --> 00:03:37.625
And
49
00:03:38.730 --> 00:03:50.410
technically speaking, that you include in the agent harness are how does the model get a sandbox to execute some code that it might be writing? How does the model does context and memory management
50
00:03:50.650 --> 00:03:53.425
and the the token optimization?
51
00:03:53.425 --> 00:04:03.665
How which tools, which MCP servers, which sub agents that you call? All of these concepts are handled by the agent harness and that's what TrueForge is. It's a neutral
52
00:04:03.665 --> 00:04:20.030
open source harness that we actually launched recently. Just a little bit about my personal background that led to us working on this. So when I was at Meta, I got a chance to lead some of the conversational AI teams. And what stood out to me was the internally developed ML platform at Meta.
53
00:04:20.815 --> 00:04:22.575
And the core design principle
54
00:04:22.575 --> 00:04:25.695
that really hit home with me was this
55
00:04:26.175 --> 00:04:28.175
notion of federated execution
56
00:04:28.175 --> 00:04:28.735
that
57
00:04:29.295 --> 00:04:41.360
I as a developer can pick the tools, the models, the frameworks that I really care about and be able to move as fast as I as I would like. So I was enabled with the right tooling to just execute,
58
00:04:42.000 --> 00:04:46.400
but at the same time, it never left the sight of centralized governance.
59
00:04:46.800 --> 00:04:54.185
So I didn't have to worry about instrumenting my code. It would always log things that I care about. And similarly a central platform team,
60
00:04:54.425 --> 00:04:56.425
if they had to figure out any
61
00:04:56.985 --> 00:04:58.425
erroneous or
62
00:04:58.745 --> 00:04:59.625
malicious
63
00:04:59.625 --> 00:05:00.425
activity,
64
00:05:00.585 --> 00:05:06.200
they always had the control to check what's happening and be able to take care of that. So this entire dichotomy
65
00:05:06.200 --> 00:05:11.240
of whether there's a trade off between federated execution and centralized governance
66
00:05:11.240 --> 00:05:14.680
turned out to be false by looking at Meta's internal ML platform.
67
00:05:14.840 --> 00:05:19.480
And we want to take that learning and bring it to the rest of the organizations out there.
68
00:05:20.175 --> 00:05:25.535
So the whole space of agent gateways and agent harnesses kind of spans
69
00:05:26.414 --> 00:05:44.410
at least two of the major hype trends over the past year or two, where maybe a year or so ago, AI gateways were gaining a lot of attention. There were a new gateway probably every other day. Some of those have been consolidated into other systems. Some of them have ceased operation.
70
00:05:44.650 --> 00:05:46.889
Some of them are continuing to grow.
71
00:05:47.289 --> 00:05:52.945
And then the past six to eight months has seen a lot of focus on the idea of harness engineering
72
00:05:52.945 --> 00:05:55.345
and that being one of the key differentiators
73
00:05:55.345 --> 00:05:57.425
beyond the models themselves.
74
00:05:57.665 --> 00:06:03.425
And I'm wondering if you can just talk through some of the ways that you're navigating some of the
75
00:06:04.060 --> 00:06:19.020
challenges and complexities of being in such a crowded space and what you see as some of the benefits of working across both of those and some of the ways that having a harness and a gateway that work well together can be a force multiplier.
76
00:06:20.044 --> 00:06:21.085
Absolutely.
77
00:06:21.405 --> 00:06:33.405
Happy to zoom in. And by the way, I cannot agree more, like, you know, with the pace that everything else is changing. In in the startup ecosystem now, like, you there's this new saying. Right? Like, you know, that earlier people had to find PMF
78
00:06:33.720 --> 00:06:48.440
or the product market fit once. Now you are fundamentally finding product market fit every six months. Right? The space is changing that fast. But also the interesting thing around this is that this is what is creating the opportunity as well. So this is certainly a challenge for building,
79
00:06:48.520 --> 00:06:51.905
but this is an opportunity for operating,
80
00:06:51.905 --> 00:06:54.225
right, in the startup space. And the reason being,
81
00:06:54.465 --> 00:06:56.065
that think about what's happening
82
00:06:56.544 --> 00:07:03.185
with every consumer of call it like an AI gateway or a control plane layer, right. An organization
83
00:07:03.345 --> 00:07:05.170
builds out a
84
00:07:05.330 --> 00:07:06.290
centralized
85
00:07:06.610 --> 00:07:13.970
platform engineering team that's going to develop this AI gateway because they care about solving those five problems that I talked about earlier, right.
86
00:07:14.530 --> 00:07:16.850
They start with doing this, they
87
00:07:17.170 --> 00:07:19.890
build a thin proxy layer and they're like okay we are sorted.
88
00:07:20.495 --> 00:07:22.175
And then soon enough,
89
00:07:22.815 --> 00:07:27.535
the model proxy layer that they have developed, all the API signatures start changing out there in the market.
90
00:07:27.935 --> 00:07:30.495
A couple of months later, MCP is launched.
91
00:07:30.655 --> 00:07:46.720
And now they're like, oh, I need to bring my MCP's in the same proxy layer. And they have to solve for authentication and authorization and all that, right? And then A2A is launched. So they have to figure out this agent to agent communication protocol through the platform, right? Then multiple agents that their teams
92
00:07:47.040 --> 00:07:51.265
have developed hit production, so they care about ensuring uptime, reliability,
93
00:07:51.585 --> 00:07:53.265
latency optimization.
94
00:07:53.825 --> 00:07:59.105
And then agent identity becomes a real thing, right? So it's an ever changing space from their standpoint.
95
00:07:59.264 --> 00:08:02.225
New models keep getting launched, new protocols keep getting
96
00:08:02.870 --> 00:08:22.025
launched and the complexity and the requirement from this platform keeps on changing every single day, right. So for thousands of team out there whose main job is not building and maintaining a platform like this, it becomes a completely intractable problem to solve. Right? So we have seen that organizations
97
00:08:22.025 --> 00:08:26.185
went from what we call as this build, break, and then buy cycle.
98
00:08:26.425 --> 00:08:41.890
Right? And that has created an opportunity for a startup like us to build a platform like this. Now, of course, at our end, the complexity does not reduce, it only increases because now we have to deal with the stack of all of those thousands of enterprises. Right? While the space is ever changing.
99
00:08:42.545 --> 00:08:59.960
So so we have done a lot of optimizations at our end where we have built out internal agentic solution that keeps pulling what are the new things that are getting launched in the market, what's breaking in our platform, and how do we continuously keep optimizing this besides of course like the fact that there's a team that really cares about solving this problem statement.
100
00:09:00.200 --> 00:09:03.640
Same thing for agent harness, I feel like this is relatively newer
101
00:09:03.880 --> 00:09:09.960
but we have seen a major demand from organizations that they care about being able to take control
102
00:09:10.605 --> 00:09:12.525
of the agents that are being developed.
103
00:09:13.084 --> 00:09:20.365
And you're just not able to take the control of the agents that that you're developing if you're using a closed proprietary
104
00:09:20.365 --> 00:09:31.810
agent development platform that you just don't have any control over. Right? So so that's where people want more of a vendor neutral harness and it's been fun to fun to work on this.
105
00:09:32.610 --> 00:09:40.834
The reason this becomes extremely interesting is vendor neutral agent harness by design is open to models from multiple providers.
106
00:09:41.154 --> 00:10:00.360
Right? So you are able to access frontier models from the labs, or you're able to work with frontier models that are open source. Right? And all of those models for all of the organizations that you work with are available through the gateway layer. So people can bring their own model extremely straightforward. Right? And this is extremely straightforward to bring their own models, basically.
107
00:10:01.080 --> 00:10:32.810
Similarly, people care about being able to bring their own MCP servers. Now the good news is that a lot of MCP servers that people want to access are these public MCP servers remotely hosted, centrally managed. So they don't have to do a lot of put in a lot of effort to try out different MCP servers. It's all connected through the platform of the gateway. Right? But then these MCP servers can be easily plugged into the the agent harness there and help them invoke these tools. And same thing, you can extend this to agents as well. Agents are all accessible through the through the gateway there. Right?
108
00:10:33.595 --> 00:10:39.115
Now across any of these models, MCP's and agents, if you wanted to apply any kind of guardrails,
109
00:10:39.115 --> 00:10:47.995
your guardrails are also available in your central control plane layer that gets implemented through the gateway, right. So fundamentally what's happening is when you're building your agents,
110
00:10:48.399 --> 00:10:52.320
all the raw building blocks that you need and the corresponding registries
111
00:10:52.399 --> 00:10:54.720
are available as part of your control plane.
112
00:10:55.200 --> 00:10:57.520
The logic to build these agents,
113
00:10:57.520 --> 00:10:58.240
right, the
114
00:10:58.720 --> 00:11:00.320
engine that operates
115
00:11:00.565 --> 00:11:14.725
the model and the tool and the agent loop that sits within the harness and it's able to help you orchestrate your calls better and the two together actually becomes a powerful force because now you're able to build agents that are accurate
116
00:11:14.965 --> 00:11:16.165
but cost efficient
117
00:11:16.460 --> 00:11:28.780
and centrally governed. So that's roughly the unlock that we're getting by bringing the pieces together, if that makes sense. Yeah. It definitely does. And one of the other interesting pieces of the gateway is that
118
00:11:29.135 --> 00:11:35.455
one of the, I think, motivating factors, at least in the initial push, was to give a unified
119
00:11:35.775 --> 00:11:41.135
interface to all of the different models. But as the different providers have
120
00:11:41.440 --> 00:11:53.840
changed their APIs and as OpenAI itself has evolved their API, that that became a much more difficult moving target, as well as the fact that by forcing everything into that one shape, you potentially lose out on capabilities
121
00:11:53.840 --> 00:11:55.120
of Anthropic
122
00:11:55.120 --> 00:12:06.535
or DeepSeek or whomever else. So I'm wondering if you can just talk to some of the tensions there of having an easier onboarding where you just have one API contract to code against versus
123
00:12:06.695 --> 00:12:10.775
being able to actually take full advantage of the underlying providers.
124
00:12:12.070 --> 00:12:16.630
That's that's a very interesting question. There are two aspects to it that I would like to call out. Number one,
125
00:12:17.030 --> 00:12:22.950
that, of course, like, you know, with the API signatures of the model providers continuously changing, this becomes,
126
00:12:23.110 --> 00:12:29.285
like, you know, relatively difficult to to implement this. But to that end, what we have done is,
127
00:12:29.765 --> 00:12:32.565
for the most part, we do the job of
128
00:12:33.125 --> 00:12:35.125
unifying the API signatures
129
00:12:35.125 --> 00:12:44.990
across, like, you know, this like, the the diverging API signatures on the on the on the right side and bring them all with the unified access for our end developers. So we don't lose the capability
130
00:12:45.150 --> 00:12:55.070
of different model providers, but at the same time, we try to solve for, like, you know, bringing them all into a more unified API signature for our end users. So they don't have to worry about these waiting
131
00:12:55.390 --> 00:12:56.030
signatures.
132
00:12:56.585 --> 00:13:01.065
But at the same time, you're right that this is not possible to execute every single time.
133
00:13:01.465 --> 00:13:08.105
And for that, what we have done is we have also implemented what we call as a custom endpoint that you can invoke through the gateway.
134
00:13:08.505 --> 00:13:14.070
And in that case, like you do not have to be compliant with the gateway specific signature,
135
00:13:14.150 --> 00:13:20.630
and you're able to run them through through the gateway, you're able to get a lot of benefits of the gateway to the end customers.
136
00:13:20.950 --> 00:13:26.855
You obviously get full benefits of the model in that case, but you do lose some out of the box capability
137
00:13:26.855 --> 00:13:35.255
that the gateway is able to do so long as it understands the underlying model signatures. So I guess we have divided the platform into those two parts.
138
00:13:35.575 --> 00:13:36.775
One where you get complete
139
00:13:37.149 --> 00:13:40.190
benefit of the gateway and unified API signature,
140
00:13:40.270 --> 00:13:45.230
and the other case, you bring a custom signature and you don't get complete benefit of the gateway.
141
00:13:45.550 --> 00:13:48.510
Digging more into the harness itself,
142
00:13:49.470 --> 00:14:03.605
as we've noted, that's I think the most recent hype cycle that we're in where everybody's building a harness. It's become a very complex market. There are a few that have gained a decent amount of traction. I think pie is probably the most notable one
143
00:14:03.925 --> 00:14:06.644
largely due to the success of OpenClaw.
144
00:14:06.930 --> 00:14:14.130
I'm wondering if you can just talk through some of the key challenges and components of building a harness that actually works well.
145
00:14:14.850 --> 00:14:15.730
Absolutely.
146
00:14:15.970 --> 00:14:16.530
Absolutely.
147
00:14:16.850 --> 00:14:23.885
Our team has been having a fun time building out this harness. And the first thing is getting
148
00:14:25.404 --> 00:14:31.725
the accuracy right. Right? Getting the agent's accuracy right, getting ensuring that you are able to
149
00:14:32.765 --> 00:14:38.710
get the quality of responses that you get from some of these like you know out of the box harnesses,
150
00:14:38.790 --> 00:14:40.070
managed harnesses
151
00:14:40.150 --> 00:15:00.415
is actually a tough problem to solve. Right? You don't quickly implement a harness and get there. Right? But after that, once once you have gotten there, the the challenging aspect of harness is a lot of the companies that additional companies that you have to deal with to ensure that all of this is happening. Right? So for example, how do you how do you make sure that you give access
152
00:15:01.170 --> 00:15:06.370
to a sandbox to an environment and by the way like of course along with all the recent incidents,
153
00:15:06.529 --> 00:15:09.970
how do you check that the agent is truly sandboxed,
154
00:15:09.970 --> 00:15:11.010
right? It's not
155
00:15:11.570 --> 00:15:15.965
it's it's not able to get out of it. How do you check that the agent
156
00:15:16.045 --> 00:15:21.005
when it's when it's executing the code, it's actually executing it efficiently
157
00:15:21.085 --> 00:15:25.725
and in the right environment. Right? How do you make sure that which of the tasks should actually
158
00:15:26.230 --> 00:15:41.655
be done through coding and like when you do the memory management around that, done through coding, but it's it's done through the tokens directly and you have to build the memory management around that. So there's some of those complexities are actually extremely difficult. Then anything around durable execution,
159
00:15:41.655 --> 00:15:50.695
human in the loop that you have to ensure that certain tools you get feedback from the human beings and then only you let the agent proceed forward
160
00:15:50.775 --> 00:15:51.815
is complex.
161
00:15:52.135 --> 00:15:52.775
And lastly,
162
00:15:53.550 --> 00:16:11.795
the the one thing that is extremely challenging with the harness layer is how do you manage permissions and access control through agent chaining. Right? Because what what's happening now is agents are often acting on behalf of users. Right? And what I mean by that is as a user, I can send a Slack message to Tobias
163
00:16:11.875 --> 00:16:21.074
or an agent acting on my behalf can send a Slack message to Tobias. Right? How does Tobias know? Whether this is directly coming from Nikunj or an agent acting on Nikunj's
164
00:16:21.074 --> 00:16:21.634
behalf. Right?
165
00:16:22.310 --> 00:16:23.990
And what happens
166
00:16:24.390 --> 00:16:34.070
if this agent invokes a sub agent invokes another sub agent in the chain basically. Right? And each of these different agents had their own identities and had their own access control.
167
00:16:34.565 --> 00:16:42.085
How do you ensure that the least privilege is managed throughout that entire chain? Right? So all of these, like, know,
168
00:16:42.885 --> 00:16:44.885
are actually fairly difficult to handle.
169
00:16:45.125 --> 00:16:45.685
And
170
00:16:46.404 --> 00:16:53.690
what lives in the harness layer of what has to live in a more centralized governance layer, the right thought process around that is also
171
00:16:54.009 --> 00:16:57.769
also something that that made things very complex to implement.
172
00:16:58.250 --> 00:17:01.449
The other interesting aspect of harnesses is that
173
00:17:01.845 --> 00:17:19.360
that term has become a little bit conflated because most people are thinking about it as far as their Claude code or codex or PI agent harness or any of the numerous other harnesses that are out there specifically for coding agents. But then there are also harnesses that are focused more on deployed
174
00:17:19.440 --> 00:17:32.895
production agents that are interacting with various other systems, and you want to make sure that those are more constrained than your local terminal agent might be and have different capabilities. And so I'm wondering if you can talk to some of the
175
00:17:33.535 --> 00:17:34.975
aspects of
176
00:17:35.295 --> 00:17:42.175
how to build that harness in a way that is layered and composable so that you can have that core functionality
177
00:17:42.175 --> 00:17:45.089
that's shared across all of the different use cases,
178
00:17:45.250 --> 00:17:46.690
but be able to actually
179
00:17:46.929 --> 00:17:53.809
exclude functionality or exclude permissions from the harness for cases where you want it to be operating in a more constrained environment.
180
00:17:54.929 --> 00:17:55.729
That makes
181
00:17:56.850 --> 00:17:57.490
a lot of sense.
182
00:17:58.985 --> 00:18:17.460
And and you're exactly right. Like, you know, first of all, you know, there's actually a lot of noise in the market. Like, people actually are not able to understand the the difference between all of these different products and the corresponding intended use cases with which they were they were developed. Right? From from our standpoint of the the stars that we have taken here is
183
00:18:17.940 --> 00:18:19.940
that we are focused on building
184
00:18:20.260 --> 00:18:40.394
more general purpose agent harness. Right? So the general productivity agent harness is what we are like, know, what we have taken a stance on building. But also like you know as you mentioned that we are developing this with an intent that we start with like this entire personal productivity type of an agent harness. Right? And eventually
185
00:18:40.555 --> 00:18:52.000
if we want to focus a little bit more on the on becoming a coding harness, we are able to steer the harness in that direction, essentially. The fundamental building blocks don't necessarily change between these different harnesses,
186
00:18:52.160 --> 00:19:11.125
but the way these components come together and, like, you know, how in one case you're always dealing with a constant context layer, let's say, from a coding side versus in the other case, like, you know, you get a lot of the context from your MCPs and stuff that you're connecting on the harness. Right? So how do you how how does the harness handle those nuances?
187
00:19:11.530 --> 00:19:24.490
Is something that, like, you know, we have basically kept it configurable from our standpoint. One thing, though, that I would mention that that has been that has been a big win, by the way, in exactly differentiating these between different types of harnesses
188
00:19:24.570 --> 00:19:29.274
is by not trying to be all in one, you actually gain a lot of efficiency.
189
00:19:30.075 --> 00:19:39.595
And what I mean by that is if you try to solve all the problems in one harness from the get go, then essentially, like, you think about all of your system prompt has
190
00:19:39.595 --> 00:19:41.274
to become massive. Right?
191
00:19:41.920 --> 00:20:09.749
And because a harness at any point in time might have to be dealing with any of the underlying, call it, tasks that you care about it's solving. And because of that and that system prompt, by the way, goes in every turn of the agent conversation. Like, you cannot avoid that. And when that starts to happen, like, start paying taxes for that. Right? So you are basically consuming more tokens in every task that you are that you are doing, and you are spending more dollars on every task that's that you're doing.
192
00:20:09.990 --> 00:20:20.630
And that has been actually one of the design decisions that we have made that the way we have designed the harness that we we are able to specialize this for different tasks and, like, you you don't have to do all in one at the same time.
193
00:20:21.855 --> 00:20:37.350
That said, I also wanted to share that because of this being one of the reasons, and actually this is one of the reasons, not everything, we have seen that when we ran the benchmark of our harness against other tools out there like cloud managed agents and all, we were able to
194
00:20:37.750 --> 00:20:38.470
get
195
00:20:38.630 --> 00:20:42.470
over 30% cost reduction for a maintained accuracy
196
00:20:42.470 --> 00:20:44.630
while using this exact same model
197
00:20:44.870 --> 00:20:47.830
and as much as 70%
198
00:20:47.830 --> 00:21:03.075
cost reduction when we let the harness choose the models, like, you know, choose its own model essentially. Right? So it it also had that flexibility of being able to choose the right model from different providers and stuff. So that's how much range we have been able to achieve by, like, you know,
199
00:21:03.715 --> 00:21:05.875
on the task that the harness is accomplishing as well.
200
00:21:06.580 --> 00:21:09.300
On that concept of model routing,
201
00:21:09.300 --> 00:21:15.940
there's also been a lot of focus there. The project that I'm most familiar with in that space is the VLLM
202
00:21:15.940 --> 00:21:28.164
router that lets you dynamically choose which model to route to for a particular request or set of requests. And I'm curious whether that's something that you see is belonging in the harness itself,
203
00:21:28.245 --> 00:21:29.205
in the gateway,
204
00:21:29.525 --> 00:21:43.909
something in between, and just some of the aspects of how to make sure that you are effectively choosing which model is going to have the right result without burning a bunch of tokens on a failed result and then having to retry it with a different model.
205
00:21:45.110 --> 00:21:49.429
The way I see it, you have to have this kind of routing capability
206
00:21:50.134 --> 00:21:58.454
at this point in both places at least the way the industry is going right now. You have to have this in both places and to serve a different purpose in the two places.
207
00:21:59.095 --> 00:22:00.614
Right? A harness
208
00:22:01.095 --> 00:22:04.534
has to be able to figure out smartness
209
00:22:03.950 --> 00:22:10.350
when it's running the loop, right? That which agent I'm calling, which model am I calling, which tool am I calling, right?
210
00:22:10.910 --> 00:22:23.114
Around that time it also has to have certain level of smartness that I know that certain models are really good at coding tasks. When I'm getting a coding task, I route my query to those models. In this task, I have such
211
00:22:23.755 --> 00:22:25.595
a large token context
212
00:22:25.595 --> 00:22:28.634
window that I need to use a model of
213
00:22:28.875 --> 00:22:38.419
model accordingly. Right? So it has to have a certain level of that intelligence built into it. And that's the routing logic that I believe the harness will continue to implement and own.
214
00:22:39.140 --> 00:22:40.100
That said,
215
00:22:40.740 --> 00:22:43.220
you can also imagine that when you're using
216
00:22:43.860 --> 00:22:53.085
when you're making calls to the models, many of these calls are coming from the harness, but many of these calls are directly coming from end users where there isn't an intelligence layer
217
00:22:53.645 --> 00:23:15.905
built into the caller to begin with. Right? So so from a and all of those calls are fielded by a gateway layer. Right? So a gateway has to be also smart by looking at the prompt, right, and figure out which model should it route it to. And of course, gateway solves a problem from a different lens as well, right? Because the gateway has the awareness, the real time awareness
218
00:23:16.065 --> 00:23:36.110
of what's the status of different model providers. Like is one of them serving high latency and I may want to route to a similar model but to a different region or the same model from a different provider, right? And it also of course has a lot of context around how much overall budget and rate limits have been consumed
219
00:23:36.110 --> 00:23:45.985
so far. So it has a lot of real time awareness and it will do the routing based on complexity but also all of this awareness as well and that layer sits rightly within the gateway.
220
00:23:46.305 --> 00:23:53.424
And the task contextual awareness routing sits more within the harness, and both of them need to coexist.
221
00:23:53.825 --> 00:23:57.985
The other interesting aspect of harnesses is because you're working
222
00:23:58.679 --> 00:24:05.799
with these models so closely and because testing anything that touches a model is so challenging due to their probabilistic
223
00:24:05.799 --> 00:24:06.519
nature,
224
00:24:07.000 --> 00:24:08.599
what are some of the
225
00:24:08.840 --> 00:24:14.655
validation methods that you use when iterating on a harness to make sure that you're not regressing
226
00:24:14.655 --> 00:24:20.894
and that it is able to actually work effectively across such a wide swath of different models with different capabilities?
227
00:24:21.455 --> 00:24:31.920
I think that is certainly a challenge for for for the exact reasons that you mentioned. Right? And I I wouldn't say that, like, we have we have fully solved the problem, like, where we have absolutely contained
228
00:24:32.480 --> 00:24:49.725
the the the test cases and regressions and ensure that, like, this is always fully solved. But, of course, we try as much as possible and the community understands this problem statement. Right? So now there are a bunch of benchmark datasets against which, like, you know, you can always keep running
229
00:24:50.285 --> 00:24:54.750
your harness to ensure that you are not regressing on your quality,
230
00:24:55.070 --> 00:24:56.510
and cost benchmarks.
231
00:24:56.510 --> 00:25:02.190
Alright? So we keep running that. And then internally for ourselves, you know, think
232
00:25:02.430 --> 00:25:08.510
I I think of these benchmark datasets as more integration tests from the from the old software world. Right?
233
00:25:09.175 --> 00:25:20.935
And then you still need your functional test and unit test as well. So what we have done is for different modules that we are building within the harness, we have actually designed some of these test cases as well. Right? So the independent modules don't progress,
234
00:25:21.480 --> 00:25:27.559
but meaningfully with like, you know, so many variables and the stochastic nature of the models and the harness,
235
00:25:27.800 --> 00:25:33.480
the only true way of testing this out is against the benchmark data sets from our standpoint.
236
00:25:33.720 --> 00:25:36.865
But even then so, it so happens that
237
00:25:37.265 --> 00:25:39.985
the end users of Harness should also
238
00:25:40.065 --> 00:25:47.025
actually, like depending on the use cases, they should have at least some coverage of these test cases at their end as well. That's
239
00:25:47.025 --> 00:25:51.799
a final litmus test to ensure that this doesn't regress.
240
00:25:51.960 --> 00:25:53.559
As people are
241
00:25:54.440 --> 00:25:56.200
designing and implementing
242
00:25:56.279 --> 00:25:58.279
agents of various shapes
243
00:25:58.600 --> 00:26:00.039
and given the
244
00:26:00.360 --> 00:26:14.195
wide variety of harnesses as we've discussed, what are some of the ways that people should be figuring out what are the differentiating factors across these different harnesses and how to identify which one is going to suit their particular use case?
245
00:26:14.435 --> 00:26:16.595
It's very interesting question,
246
00:26:16.835 --> 00:26:32.260
Tobias, because the way I'm thinking about this, and I would love to brainstorm with you on this one, like hear your thoughts as well. Right? But of course, while we see that there's a bunch of harnesses out there in the market today that you're able to leverage to build some of your agents, the divide
247
00:26:32.404 --> 00:26:33.524
in the industry
248
00:26:33.924 --> 00:26:45.205
is at a very macro level. Right? And actually, we are seeing our customers choose their harnesses based on those macro divides at the moment rather than trying to get super
249
00:26:45.205 --> 00:26:47.764
micro with it with a bunch of
250
00:26:48.380 --> 00:26:52.700
low level capabilities of different harnesses or at least that's what we're seeing at the moment.
251
00:26:53.100 --> 00:26:54.619
What what do I mean by that?
252
00:26:54.940 --> 00:26:58.859
At the first level you can imagine that you have harnesses that are
253
00:26:59.340 --> 00:27:04.404
SaaS like managed harnesses that you get to use out of the box.
254
00:27:04.645 --> 00:27:15.125
There are some of the harnesses that are more vendor neutral, more open harnesses. Right? And people are able to take a call based on their own needs that am I happy to be locked into this ecosystem
255
00:27:15.929 --> 00:27:20.330
and get the advantage of an integrated ecosystem, right, which could come at a price?
256
00:27:20.570 --> 00:27:27.769
Or do I want to do a little bit more work and get into this open source neutral type of a harness, right? So that's one type of a thing.
257
00:27:28.735 --> 00:27:30.975
The the second here is
258
00:27:31.294 --> 00:27:33.774
that the way some of these harnesses are implemented,
259
00:27:33.855 --> 00:27:40.815
they're intended for certain audience. Like, some some harnesses assume that you're a developer, you understand, like, you know, you're able to set up a bunch of configurations.
260
00:27:41.720 --> 00:28:00.375
You're even, if need be, you're able to write some code, right, or micromanage how the harness is working. Right? So it's intended for a more technical audience, where some harnesses actually are just, like, you know, built for a knowledge worker who does not necessarily come from a coding background and does not want to manage complex configurations
261
00:28:00.375 --> 00:28:01.095
overall.
262
00:28:01.495 --> 00:28:02.135
So
263
00:28:02.375 --> 00:28:12.429
today, I'm seeing that, like, you know, there's a few handful of tools that are out there that, you know, people for the most part are leveraging can be divided based on based on this itself.
264
00:28:12.590 --> 00:28:22.910
Right? We people don't have to go too low level that oh like in this case they harnessed us very well on memory management and in this case they harnessed us very well on a sandbox execution
265
00:28:22.910 --> 00:28:30.304
so and so forth. And we are not seeing people take calls based on that. Are you seeing a different trend by any chance? So, yeah, I think that
266
00:28:30.705 --> 00:28:32.225
in the harness space,
267
00:28:32.945 --> 00:28:40.200
one of the interesting pieces, just as an anecdote, is that a lot of them seem to be written in TypeScript and running on the node ecosystem,
268
00:28:40.280 --> 00:28:40.760
which
269
00:28:41.160 --> 00:28:43.080
as a Python developer,
270
00:28:43.160 --> 00:28:47.799
it is curious to me, but I understand that JavaScript is taking over the world.
271
00:28:48.520 --> 00:28:49.000
But,
272
00:28:49.320 --> 00:28:53.320
yeah, the the key factors that I see when I'm looking at different harnesses are,
273
00:28:54.145 --> 00:28:57.825
yeah, what is the model support, which is generally fairly broad,
274
00:28:58.465 --> 00:28:59.264
but also
275
00:28:59.425 --> 00:29:18.109
how does it integrate with some of the different providers. So I think Claude and Codex are probably the more interesting ones, Claude specifically because they've been having a challenging time of figuring out how their pricing that when you're going through their Claude desktop and Claude code work versus if you're using a third party harness where
276
00:29:18.270 --> 00:29:25.644
you don't get the benefits of their flat rate pricing. So that's one of the considerations when I'm looking at some of the harnesses as a software engineer.
277
00:29:26.684 --> 00:29:38.609
But the other pieces are the MCP and a two a support. So how quickly they're adopting some of these different protocols, particularly as the protocols themselves change. So MCP just released their latest revision
278
00:29:38.850 --> 00:29:40.049
maybe a month ago.
279
00:29:40.610 --> 00:29:42.289
And then the
280
00:29:42.529 --> 00:29:49.085
what is the footprint of the harness? How much resource does it take? How does that factor into deployability,
281
00:29:49.164 --> 00:29:55.885
particularly for if I'm using this as a deployed agent that I want to run? How does it fit into things like instrumentation
282
00:29:55.885 --> 00:29:57.005
and observability?
283
00:29:57.005 --> 00:30:08.379
Because I wanna make sure that I have a full trace of what the agent is doing so that I can figure out when things go wrong. Why did they go wrong? How did they go wrong? Because they're moving at machine speed,
284
00:30:08.620 --> 00:30:23.914
you have to have other agents that are monitoring those outputs to make sure that you can catch those failure modes versus just if you just have a harness that says, yes. I can run the model. I can plug into all these protocols, but it's a black box and opaque, you have no introspection into the system.
285
00:30:24.235 --> 00:30:31.989
That's definitely a nonstarter for me as an operations engineer. So those are the some of the things that I look at when I'm thinking about different harnesses.
286
00:30:32.790 --> 00:30:38.230
That that that makes a lot of sense. That makes a lot of sense, Tobias. For sure. Like, you know, we we definitely
287
00:30:38.230 --> 00:30:42.230
see our users considering these, but, like, you know, a lot of the times that,
288
00:30:42.710 --> 00:30:47.434
at least from our lens, what we are seeing is we work with organizations.
289
00:30:47.595 --> 00:31:04.370
And on an organization level, I think the the decision matrix is broader than this because people really care about being able to suit some harness to their own broad scope needs, and they choose based on that. But we'll get exposed to more and more of these nuances,
290
00:31:04.370 --> 00:31:07.250
like as our open source agent harness
291
00:31:07.250 --> 00:31:09.809
gets a broader adoption as well. So I think
292
00:31:10.435 --> 00:31:12.674
that would definitely be an interesting learning for us.
293
00:31:13.235 --> 00:31:14.195
So
294
00:31:14.355 --> 00:31:27.929
digging now further into the TrueForge harness itself, could you talk through some of the key design and architectural elements of it and some of the ways that the shape and scope have evolved from when you first started working on it?
295
00:31:28.810 --> 00:31:32.490
So so in terms of, like, I just generally, the way we have designed
296
00:31:32.810 --> 00:31:50.004
first of all, like, I just let me give you little bit of a background on how this entire concept of Harness started from from our standpoint, right, and then I will describe how some of the architectural choices have evolved for us as a team as well. So there were two main pulling factors around this.
297
00:31:50.325 --> 00:31:58.909
We wanted to build some capabilities on our own product like like TrueFoundry AI gateway product where people are able to consume
298
00:31:59.309 --> 00:32:01.710
a lot of the information through the product
299
00:32:02.110 --> 00:32:08.244
but in a more agentic way. I'll give an example. TrueFoundry AI gateway sees a bunch of traces,
300
00:32:08.325 --> 00:32:09.684
agentic traces,
301
00:32:09.924 --> 00:32:18.645
and we want to be able to reason on the traces, alright, that which of my agents are doing well, which ones are prone to any guardrail problem injection attacks,
302
00:32:19.050 --> 00:32:19.930
or
303
00:32:19.930 --> 00:32:25.450
where is PII leaking that I don't want want it to, how can optimize for cost and stuff like that. Right?
304
00:32:26.170 --> 00:32:38.144
So we wanted our users to have that capability where they can more naturally improve the agents that they are, like, you know, invoking through our gateway. Right? So fundamentally, this means that we wanted to
305
00:32:38.705 --> 00:32:45.504
bake in a first class agent as part of our own product so that people are able to leverage this.
306
00:32:46.145 --> 00:32:47.505
And to that end,
307
00:32:48.980 --> 00:32:49.940
fundamentally,
308
00:32:49.940 --> 00:32:52.740
everything that we needed this agent to access,
309
00:32:53.140 --> 00:32:56.980
all the context, all the traces, all the models and
310
00:32:57.460 --> 00:33:05.365
MCPs that the user had permission to was available within the product. And we had all of these things available as context and as MCPs.
311
00:33:05.525 --> 00:33:06.965
So from our standpoint,
312
00:33:07.925 --> 00:33:14.085
this felt like it should have been as simple as that we're able to connect a bunch of these, like, in a context and
313
00:33:14.885 --> 00:33:23.500
use a harness to quickly spin up this agent that we are able to ship with the product. Right? But when we try to use the tools out there today,
314
00:33:23.660 --> 00:33:29.019
like back then, right, we realized that you are either locked into a bunch of these closed harnesses,
315
00:33:29.100 --> 00:33:35.205
which does not allow you this capability of shipping agents as part of the product itself. Right? Or you would have to fundamentally
316
00:33:35.205 --> 00:33:36.005
leverage
317
00:33:36.725 --> 00:33:38.485
more pro code frameworks,
318
00:33:38.485 --> 00:33:44.005
which, like, require you to micromanage the performance of your agents and then quickly things diverge and you keep,
319
00:33:44.485 --> 00:33:50.860
like, upgrading that essentially. Right? So so we really wanted to strike that balance where we are able to optimize for speed,
320
00:33:51.420 --> 00:33:55.419
ship things fast, and they cannot get locked into any ecosystem.
321
00:33:55.740 --> 00:34:11.795
So that's that was one of the motivation that, like, you know, led us to kick things off. And at the same time, we started seeing that a lot of our customers started caring about, like, you know, not being locked into a certain vendor that way. Right? So when we started building this out,
322
00:34:12.115 --> 00:34:13.235
the neutrality,
323
00:34:13.315 --> 00:34:14.355
the openness,
324
00:34:14.355 --> 00:34:16.115
the speed at which you are able to ship,
325
00:34:16.619 --> 00:34:21.099
these were some of the design principles with which we had started building out the harness.
326
00:34:21.740 --> 00:34:23.020
Over a period of time,
327
00:34:23.420 --> 00:34:29.420
when we started actually implementing this, and this actually changed a lot in how we thought about the overall architecture,
328
00:34:29.660 --> 00:34:30.779
we realized that
329
00:34:33.155 --> 00:34:38.595
while we are able to make building of the agent relatively straightforward,
330
00:34:38.835 --> 00:34:43.235
being able to run this, right, and being able to run this
331
00:34:43.950 --> 00:34:51.950
at times and in environments that we care about is another challenge. Like, there's a lot of things that you want this agent to do that you want to be able to schedule,
332
00:34:52.270 --> 00:34:58.335
that you want to run every day or every few hours. Right? Or you want to run this on a trigger.
333
00:34:58.495 --> 00:35:17.450
How do you expose or sometimes you want the agents that you're developing, you want this agent to run as part of another ecosystem. Right? So how do you build out that entire runtime of the agent is another major problem that we realized that we needed to solve. Right? So, like, we continue to evolve the harness from, like, you know, primarily being neutral,
334
00:35:17.530 --> 00:35:22.490
like, you know, to this entire runtime. Now the two different types of runtimes, basically. Right? Similarly,
335
00:35:23.055 --> 00:35:25.535
some of the design choices that we needed to change was
336
00:35:25.855 --> 00:35:28.415
how do you think about sandboxing
337
00:35:28.415 --> 00:35:30.895
overall? And what we thought was sandbox
338
00:35:31.214 --> 00:35:36.415
should be exposed to the agents of the tool. That's the other design choice that we had to make. Most of the harnesses that we know
339
00:35:37.290 --> 00:35:43.210
run the entire agent itself in a sandbox. We actually ended up flipping that design principle over a period of time.
340
00:35:43.610 --> 00:35:47.530
So now what we do is we run the agent loop in the server itself,
341
00:35:47.690 --> 00:35:55.985
and we provision a sandbox only when the agent actually needs to execute that code. Right? So the advantage that we got by doing this was
342
00:35:56.145 --> 00:35:59.905
one server is now able to run many of our agents concurrently.
343
00:36:00.224 --> 00:36:00.865
And
344
00:36:01.345 --> 00:36:15.000
it turns out that that when you are not using writing code, basically, it makes those turns go much cheaper and much faster. And you're ensuring that things like your entire secrets and all are never leaving
345
00:36:16.840 --> 00:36:19.155
the layer that the hardest can touch at all. Right?
346
00:36:19.635 --> 00:36:22.515
So a lot of these things that we started
347
00:36:22.755 --> 00:36:25.075
editing in our architecture
348
00:36:25.555 --> 00:36:34.980
came as evolution of what we saw as this accuracy cost and efficiency trade off that we really, really wanted to nail down. And
349
00:36:35.060 --> 00:36:36.980
over a period of time, these sort of things evolve.
350
00:36:37.460 --> 00:36:39.460
And as you have been
351
00:36:39.940 --> 00:36:44.820
working on the harness and using that to validate the gateway,
352
00:36:45.380 --> 00:36:47.060
what are some of the
353
00:36:47.940 --> 00:36:48.580
most
354
00:36:48.755 --> 00:36:51.475
complex or challenging aspects
355
00:36:51.475 --> 00:36:52.115
of
356
00:36:52.275 --> 00:36:52.995
building
357
00:36:53.235 --> 00:37:02.890
these two different systems and using them to play off against each other and just some of the discoveries that you've run into that were unanticipated
358
00:37:02.890 --> 00:37:06.330
roadblocks on your path to actually building these products?
359
00:37:07.210 --> 00:37:07.770
Yeah.
360
00:37:08.090 --> 00:37:08.650
So
361
00:37:08.890 --> 00:37:14.490
in all honesty, I think one of the problems that you always have to solve for in such settings is
362
00:37:14.650 --> 00:37:15.530
organizational,
363
00:37:15.530 --> 00:37:15.770
right?
364
00:37:16.855 --> 00:37:18.055
Because the way
365
00:37:18.295 --> 00:37:32.280
you organize your teams building out the by the way, these are problems that everyone needs to solve for. Right? Like, like, people have to figure out the solution. So I would like to share this with with folks as well listening listening to the podcast. When you think about
366
00:37:32.520 --> 00:37:36.360
building out the the gateway itself, you are building for
367
00:37:36.680 --> 00:37:42.520
one intended audience, which is getting used in a certain way. Right? Organizations buy a gateway
368
00:37:42.840 --> 00:37:44.280
for observability,
369
00:37:44.280 --> 00:37:46.680
for control, for being able to
370
00:37:47.295 --> 00:37:48.095
access
371
00:37:48.095 --> 00:37:48.735
the
372
00:37:49.055 --> 00:37:57.535
rich ecosystem relatively fast. Right? When you're building out an open source harness instead, right, you are building for the capabilities
373
00:37:57.694 --> 00:38:07.140
to to empower the developers with a bunch of these capabilities. Right? So that they are able to developers or even knowledge workers to take up a harness and they're gonna build out these agents.
374
00:38:07.460 --> 00:38:08.339
Now because
375
00:38:09.539 --> 00:38:11.140
the the user
376
00:38:11.460 --> 00:38:18.180
is slightly different in the two cases. Right? The team that's building it has to always continuously
377
00:38:17.595 --> 00:38:18.555
empathize
378
00:38:18.635 --> 00:38:20.795
with the end user that they're building towards.
379
00:38:21.115 --> 00:38:24.155
And the pace at which the team is developing
380
00:38:24.155 --> 00:38:29.435
different components also needs to change because one is serving more production systems.
381
00:38:29.755 --> 00:38:32.075
The other one is serving more
382
00:38:32.075 --> 00:38:41.970
building phase of the ecosystem as well. Right? So so from our standpoint, how do we allocate the bandwidth? How do we divide the team? How do we ensure
383
00:38:42.050 --> 00:38:46.930
that we are able to meaningfully make progress on the on both the two fronts was actually
384
00:38:47.090 --> 00:38:58.005
a relatively big problem for us to solve for. Right? The way we we came up with like, the the approach that we came up is to solve this problem is we actually combined
385
00:38:58.085 --> 00:38:58.725
the
386
00:38:59.605 --> 00:39:01.925
the part of our gateway
387
00:39:01.660 --> 00:39:03.420
that was leveraged
388
00:39:03.500 --> 00:39:10.540
to build a bunch of these agentic solutions directly. Like, we had a component in our gateway where people are able to register registries,
389
00:39:10.540 --> 00:39:11.580
like agent
390
00:39:11.580 --> 00:39:12.620
to a registry.
391
00:39:12.940 --> 00:39:13.660
And from there,
392
00:39:14.285 --> 00:39:16.285
they would they would always
393
00:39:16.365 --> 00:39:18.765
run into some roadblocks
394
00:39:18.765 --> 00:39:30.320
into how they have built out an agent using another closed source product or some other framework. They want to register to the registry and then access it centrally through the gateway. They would keep running into those roadblocks.
395
00:39:30.560 --> 00:39:34.160
And there were a lot of learnings that were coming out about how
396
00:39:34.400 --> 00:39:38.640
the agent development platform or the agent development harness itself should solve the problem.
397
00:39:39.315 --> 00:39:40.595
So we combined
398
00:39:40.595 --> 00:39:48.275
the the team that was always getting that learning from this part of the product to also work on the agent harness layer.
399
00:39:48.355 --> 00:39:55.714
Right? So the top of the funnel, like, the core pain points that people want to leverage an outsourced
400
00:39:54.720 --> 00:39:56.000
open source harness
401
00:39:56.320 --> 00:39:58.480
was always coming to this team. So
402
00:39:59.360 --> 00:40:00.880
the empathy for
403
00:40:01.040 --> 00:40:02.000
the user,
404
00:40:02.320 --> 00:40:06.320
both from a gateway standpoint and from a harness standpoint, is how
405
00:40:07.040 --> 00:40:19.325
we solve for. And a lot of the capabilities that we ended up building within the harness by design, the roadmap of the harness actually came from our early users of the gateway directly. So that way, a lot of the downstream
406
00:40:19.725 --> 00:40:25.710
challenges that we would have needed to solve for actually got front loaded by
407
00:40:26.030 --> 00:40:28.430
working with the right set of users
408
00:40:28.510 --> 00:40:34.430
using the product as a design partner, I guess. You touched a little bit on some of the
409
00:40:34.590 --> 00:40:40.665
cost aspect of the harness being a contributing factor where that's largely the system prompt and
410
00:40:40.985 --> 00:40:54.220
trimming that down can have a substantial savings. But as you have been building the system, what are some of the other key elements of the harness itself that contribute the most to the cost efficiency,
411
00:40:54.380 --> 00:40:57.500
the accuracy of the outputs, and the reliability
412
00:40:57.500 --> 00:40:58.220
of
413
00:40:58.300 --> 00:40:59.340
the agent
414
00:40:59.500 --> 00:41:00.860
operating within that harness?
415
00:41:01.755 --> 00:41:05.994
So there three major factors to this entire cost optimization
416
00:41:05.994 --> 00:41:13.355
and the accuracy and reliability trade off. Like one we covered, which is your system prompt, right? The second one is
417
00:41:13.910 --> 00:41:16.630
generally your harness efficiency.
418
00:41:16.630 --> 00:41:28.790
And what I mean by that is the number of tool calls that you are making, the order in which you are making the tool calls, the order in which you are applying any kind of policies that you may want to do,
419
00:41:29.565 --> 00:41:31.885
which actually has a significant impact
420
00:41:32.285 --> 00:41:33.085
on how
421
00:41:33.325 --> 00:41:37.405
overall harness is working. Right? And to this end, there are certain
422
00:41:37.805 --> 00:41:41.165
more technical concepts that you also bake into the harness.
423
00:41:41.484 --> 00:41:42.285
For example,
424
00:41:43.005 --> 00:41:44.204
do deferred
425
00:41:44.610 --> 00:41:54.050
loading of tools, for example. When you do not need access to certain tools at a certain point, you defer that, right? So how do you bake in that intelligence
426
00:41:54.050 --> 00:41:57.490
that for any given task that the harness is accomplishing,
427
00:41:58.205 --> 00:42:00.525
you always pick the right combination
428
00:42:00.685 --> 00:42:01.325
of
429
00:42:01.645 --> 00:42:05.965
whether you are loading a certain tool, whether you are using the
430
00:42:06.205 --> 00:42:07.725
actual persistence
431
00:42:07.965 --> 00:42:10.445
to be able to store some context and only
432
00:42:11.245 --> 00:42:13.245
get a part of it when you need
433
00:42:13.900 --> 00:42:16.060
to answer a certain question.
434
00:42:16.460 --> 00:42:36.005
What do you solve for using tokens of the model directly versus where you think that being able to write code and execute this would be the right way to solve this problem? So this is what I call as the hardest efficiency. And the right choices that you make in this one is extremely critical to solve for the cost and accuracy trade off. And the last one,
435
00:42:36.405 --> 00:42:38.005
which is the overall
436
00:42:38.085 --> 00:42:42.850
model benchmarks overall. Right? Like, that's the that's the other one that's that's extremely critical.
437
00:42:43.250 --> 00:42:47.570
Many times, like, for the same task, like, models can have
438
00:42:47.810 --> 00:42:51.330
an order or even two orders of magnitude cost difference.
439
00:42:52.224 --> 00:43:01.985
But their performance could be fairly comparable at that point. Right? So how do you choose the appropriate model for different types of tasks that you're doing? And how do you train the harness
440
00:43:02.224 --> 00:43:03.025
so that
441
00:43:03.905 --> 00:43:15.380
it's able to make those design choices? I think these are the three key factors that we have worked to optimize on. Obviously, a lot of effort that has gone into is problem number two, where different
442
00:43:16.180 --> 00:43:18.660
components of the harness, how do you optimize against
443
00:43:18.980 --> 00:43:19.299
those?
444
00:43:20.115 --> 00:43:25.635
Like, that's where a lot of efficiencies and and citing some numbers here, there have been tasks where we have seen
445
00:43:25.875 --> 00:43:27.555
some other harnesses
446
00:43:27.875 --> 00:43:29.955
which are taking, like, more than 75,000
447
00:43:29.955 --> 00:43:35.590
tokens to implement, like, know, to accomplish the task. Our harness was able to do within 23,000
448
00:43:35.590 --> 00:43:39.510
tokens basically. So we have seen anywhere from 50
449
00:43:40.390 --> 00:43:42.470
even 7075%
450
00:43:42.470 --> 00:43:46.310
of token reduction in similar kind of tasks by getting this part right.
451
00:43:46.974 --> 00:43:53.055
And then going back to what we touched on briefly of different models behaving differently
452
00:43:53.295 --> 00:43:55.535
given even the same prompts,
453
00:43:55.695 --> 00:44:02.840
the general wisdom that has built up over the past couple of years, and, of course, that general wisdom is frequently invalidated
454
00:44:02.840 --> 00:44:04.680
because everything moves so quickly,
455
00:44:05.160 --> 00:44:12.200
is that just changing a model given all of the same inputs can have wildly different outcomes.
456
00:44:12.680 --> 00:44:13.320
And
457
00:44:13.560 --> 00:44:14.840
given the current
458
00:44:15.345 --> 00:44:18.305
generation of models that we are operating,
459
00:44:18.865 --> 00:44:19.665
is that
460
00:44:20.065 --> 00:44:24.145
assumption still accurate? And what are some of the ways that people should be
461
00:44:24.545 --> 00:44:26.785
testing and thinking about their
462
00:44:26.865 --> 00:44:27.665
preconceived
463
00:44:27.665 --> 00:44:45.905
notions given what they've learned from the past two years of evolution where I think the hallucination rate was one of the first ones to gain a lot of ground, and that has is it's still a factor, but the the rate has dropped a lot. I'm just wondering as the models evolve and as they all become more capable,
464
00:44:46.465 --> 00:44:48.065
has that assumption of
465
00:44:48.785 --> 00:44:51.825
switching models will dramatically change the outcome?
466
00:44:52.145 --> 00:44:53.345
Does that still hold?
467
00:44:54.385 --> 00:44:55.265
Actually,
468
00:44:55.910 --> 00:44:58.550
we have seen that assumption break.
469
00:44:59.030 --> 00:45:03.270
It's been now a few months that we have seen that assumption break, in all honesty.
470
00:45:03.750 --> 00:45:06.950
And what that means is by the way, hallucination
471
00:45:06.950 --> 00:45:22.835
is a term. Like, I I think if there was a way to build a heat map of how frequently that term is being used, I think now over the last few months, it has become quite cold, that heat map. Right? We don't we don't hear people talk about hallucinations as much when they are talking about
472
00:45:23.600 --> 00:45:28.400
the model performance. Same thing. Earlier, we really needed to craft a
473
00:45:28.560 --> 00:45:46.515
prompt for each of the models, and that was a big thing that people needed to solve for. Now we are seeing that models are getting better, that that you don't need to micromanage your prompts as much to work with these different models. Right? Now, of course, like when you go from a frontier model to a very specialized model, the specialized model expects
474
00:45:46.595 --> 00:45:51.795
sometimes even if you have fine tuned some of these models and we we work with still with some of the fine tuned models,
475
00:45:52.650 --> 00:45:56.810
That expects prompt to come in a certain way, and that really helps to
476
00:45:57.609 --> 00:46:09.415
to be able to change the prompts. But, like, so long as you're working in a, like, a beyond a certain level of, call it, the number of parameters in the model. Right? Like, once you're working with a threshold,
477
00:46:09.895 --> 00:46:13.975
model is larger than a certain threshold, you actually don't need to worry about switching prompts
478
00:46:14.055 --> 00:46:16.615
anymore for to to get similar kind of outputs now.
479
00:46:17.190 --> 00:46:19.830
And that, of course, has downstream
480
00:46:19.830 --> 00:46:23.430
implications now in terms of implementation of the harness that
481
00:46:24.230 --> 00:46:25.590
you you just
482
00:46:25.990 --> 00:46:29.109
if you you're able to make choices more
483
00:46:30.445 --> 00:46:31.965
independently now. Right?
484
00:46:32.525 --> 00:46:34.445
But at the same time, we
485
00:46:34.685 --> 00:46:49.440
we in our gateway product as well, we keep this flexibility that if people are switching model, they have a scope to be able to change the the prompt as well. And it's especially useful when you're using fine tuning and stuff. Right? So we actually leverage that capability also
486
00:46:50.079 --> 00:46:51.680
in our harness as well.
487
00:46:52.400 --> 00:46:53.920
One of the other
488
00:46:54.160 --> 00:46:56.480
factors of model selection
489
00:46:57.035 --> 00:47:05.515
that I've experienced is that you might need at least a certain threshold of model to be able to effectively drive the harness where,
490
00:47:05.755 --> 00:47:37.395
a while ago, they I've had a conversation with somebody who had the analogy of the LLM is the engine and everything else that you put around it is the car. So you can have all the horsepower you want, but if there's no car, you're not going anywhere. And so bringing that a little further, it sort of feels like you you have sort of as you're going on the amusement park rides, you must have this many parameters to drive this harness. And I'm wondering what you're seeing as some of the cutoff threshold where if you get below a certain size model, it's not going to be able to effectively use the harness because there is too much in the context window.
491
00:47:37.875 --> 00:47:38.515
So
492
00:47:38.750 --> 00:47:43.230
and and, like, the most the most interesting thing that we are seeing with this is
493
00:47:44.670 --> 00:47:45.310
people,
494
00:47:45.549 --> 00:47:52.510
the the users of the harness, including including ourselves. Right? We have built out a lot of systems that, like, leverage our own harness. And
495
00:47:52.865 --> 00:48:00.385
people actually don't necessarily need to go below a certain threshold anymore to make things work for them. Effectively,
496
00:48:00.464 --> 00:48:05.505
it's not like people are thinking about, should I use a 3,000,000,000 parameter open source
497
00:48:06.224 --> 00:48:08.065
model to run my entire harness
498
00:48:08.550 --> 00:48:15.510
versus should I use Fable five, for example, right? Like, it's not like people are typically operating in those extremes.
499
00:48:15.510 --> 00:48:16.870
People are thinking like,
500
00:48:17.190 --> 00:48:19.830
like, am I doing a Fable or a GLM?
501
00:48:19.830 --> 00:48:21.110
Am I thinking
502
00:48:21.350 --> 00:48:22.150
Haiku,
503
00:48:22.150 --> 00:48:30.705
right? So people are still operating in that domain. And what what's what's nice about this is the overall token cost has reduced,
504
00:48:31.105 --> 00:48:33.105
like, you know, by such a large
505
00:48:33.744 --> 00:48:46.840
margin over the over the period that intelligence per token, as I think about this. Right? Intelligence per dollar that you're spending has has gone up so much that that's the ballpark in which we are we are talking about. So now
506
00:48:47.320 --> 00:48:49.080
you are still able to
507
00:48:49.835 --> 00:48:52.234
like, even if you choose a base model,
508
00:48:52.555 --> 00:49:07.860
something like a HYCU and above, basically, right, the the hardest works. And then for extremely complex task, you go to, like, your GLMs and the Opus and the Fable equivalent, basically. And otherwise, you're able to a bunch of tasks using high equivalent type of products.
509
00:49:08.580 --> 00:49:11.060
And as you have been
510
00:49:11.380 --> 00:49:20.635
working on TrueForge and TrueFoundry and working with your customers, what are some of the most interesting or innovative or unexpected ways that you've seen your harness used?
511
00:49:21.195 --> 00:49:21.915
So
512
00:49:22.155 --> 00:49:25.195
first of all, we noticed that
513
00:49:25.595 --> 00:49:26.555
people are
514
00:49:27.355 --> 00:49:28.555
We saw
515
00:49:29.115 --> 00:49:30.714
through our product itself directly
516
00:49:30.990 --> 00:49:33.390
that we see queries in languages
517
00:49:33.470 --> 00:49:36.110
that we absolutely don't understand. We
518
00:49:36.270 --> 00:49:38.030
were under the assumption that
519
00:49:38.430 --> 00:49:45.815
most people are using the harness in English with the most common user group that we work with. They're saying queries coming in completely
520
00:49:45.815 --> 00:49:47.974
different languages that don't
521
00:49:47.974 --> 00:49:48.615
understand.
522
00:49:48.775 --> 00:49:51.175
That was just a fun thing that we noticed
523
00:49:51.494 --> 00:49:53.415
in terms of the usage of the platform.
524
00:49:53.655 --> 00:49:57.015
Outside of this, what we have seen with our harness usage,
525
00:49:58.119 --> 00:49:58.920
people
526
00:49:59.240 --> 00:49:59.880
are
527
00:50:00.440 --> 00:50:03.720
doing a couple of things that we just did not imagine
528
00:50:05.240 --> 00:50:11.560
will be a use case because it's supposed to be such a deterministic use case. People have actually built out their own pricing systems,
529
00:50:12.225 --> 00:50:15.665
like how they price their product overall
530
00:50:15.825 --> 00:50:19.745
based on some data that they capture using our gateway
531
00:50:19.985 --> 00:50:25.665
and how they build out the end pricing dashboard for their end users using our harness.
532
00:50:26.070 --> 00:50:37.590
And this entire loop came together for our end users and they were able to build out this entire thing within a few hours that their understanding was it would have taken them weeks or months to build this entire system.
533
00:50:37.910 --> 00:50:41.575
That kind of a usage, we had not expected
534
00:50:41.575 --> 00:50:44.455
people to leverage the harness for. So basically,
535
00:50:44.775 --> 00:50:46.695
the input comes from the gateway,
536
00:50:46.855 --> 00:50:51.335
the implementation happens through the harness, and that combination became very powerful.
537
00:50:51.815 --> 00:50:52.375
So
538
00:50:52.960 --> 00:50:57.760
what this has opened up for our users and also just in our
539
00:50:58.080 --> 00:50:59.600
mind in terms of what
540
00:51:00.000 --> 00:51:07.155
we were expecting versus what we are seeing is people leverage gateway to build out pretty much all of their agentic capabilities.
541
00:51:07.315 --> 00:51:10.994
All of the agent capabilities by default get routed through the gateway.
542
00:51:11.315 --> 00:51:15.234
Now, any derivative component that they need to build from here,
543
00:51:15.555 --> 00:51:23.730
that's where now the harness is starting to get used very strongly. So I think that's that's been an insight that we have learned through the usage of the product.
544
00:51:24.210 --> 00:51:32.690
And in your experience of working in this space and building these systems, what are the most interesting or unexpected or challenging lessons that you've learned personally?
545
00:51:33.744 --> 00:51:37.745
So one of the very interesting things that I've learned building
546
00:51:37.984 --> 00:51:41.825
these products over the last two, three, four years is
547
00:51:42.145 --> 00:51:43.665
when you work
548
00:51:43.744 --> 00:51:45.185
with different
549
00:51:45.345 --> 00:51:47.665
personas with their own priorities,
550
00:51:47.665 --> 00:51:47.905
right,
551
00:51:48.520 --> 00:51:53.160
they are looking at the same solution from completely
552
00:51:53.160 --> 00:51:54.280
different lens.
553
00:51:55.480 --> 00:51:57.800
And it so turns out that if you're building
554
00:51:58.040 --> 00:52:02.120
a system that caters to people with different
555
00:52:02.120 --> 00:52:03.880
goals that they're optimizing for,
556
00:52:04.575 --> 00:52:11.215
and this is the only system through which all of those goals get a common ground, right,
557
00:52:11.615 --> 00:52:14.255
to be solved for, right, then
558
00:52:14.335 --> 00:52:23.240
the only way that you can think about building a product like that is by having a good mix of opinion from all of these different varying
559
00:52:23.320 --> 00:52:26.760
voices in the room. And I'll give an example of why
560
00:52:26.920 --> 00:52:28.120
that has been such
561
00:52:28.360 --> 00:52:30.200
an important learning experience for us.
562
00:52:30.840 --> 00:52:32.680
In the very beginning, I talked about
563
00:52:33.320 --> 00:52:37.000
different pillars that this
564
00:52:35.675 --> 00:52:40.795
gateway and harness product is solving for. It's solving for reliability
565
00:52:41.035 --> 00:52:48.155
that your platform teams, your development teams care about. The fact that you build an application, it should be always up, always running, operate at low latency.
566
00:52:49.130 --> 00:52:53.450
Then there's a security aspect to it that your CISO org cares a lot about.
567
00:52:55.690 --> 00:52:58.810
The four walls within my organization should always be secure.
568
00:52:59.530 --> 00:53:04.410
Then there's a CFO aspect to it that they really care about being able to do this cost optimization.
569
00:53:05.424 --> 00:53:06.145
Overall,
570
00:53:06.305 --> 00:53:19.470
they should have an ROI, and that entire story should be clear. Which agents are we building? How do we do the reporting and stuff like that? And then there's this compliance aspect to it that if tomorrow I have to be able to answer why my agent built a certain decision,
571
00:53:20.110 --> 00:53:21.070
how do I do that?
572
00:53:21.550 --> 00:53:23.070
And turns out that the
573
00:53:23.390 --> 00:53:31.150
interest of the CISO, the CFO, the compliance officer, the CTO, all of these things are now being answered by this common layer.
574
00:53:32.265 --> 00:53:33.545
And the
575
00:53:33.944 --> 00:53:35.465
most interesting way
576
00:53:35.625 --> 00:53:38.184
that you can solve for this
577
00:53:38.184 --> 00:53:42.265
entire problem statement is by hearing the voice of all of these personas.
578
00:53:42.904 --> 00:53:51.680
And where we have seen some of the solutions within the industry where they have tried to create a completely specialized view for the CFO
579
00:53:51.680 --> 00:53:53.360
in terms of the FinOps layer.
580
00:53:54.560 --> 00:53:56.400
And then they have basically
581
00:53:57.040 --> 00:53:58.895
done so much instrumentation
582
00:53:58.975 --> 00:54:03.295
that it doesn't get adopted by the platform engineering or the actual development layer. So
583
00:54:03.855 --> 00:54:09.615
yes, you have built this amazing, beautiful FinOps layer. But because there's no usage coming from
584
00:54:10.335 --> 00:54:13.100
the left side, there's nothing to see on the right side,
585
00:54:13.340 --> 00:54:13.900
and
586
00:54:14.380 --> 00:54:33.065
so on and so forth. We have seen multiple combinations of these break in the industry. We have seen multiple personas pull our product in different directions. And the way we are able to maintain this balance is by listening to these point of views. And I guess that's been a personal lesson that you really want to understand that, yes, who is your main buyer persona? But
587
00:54:33.065 --> 00:54:42.220
who are the other extremely important stakeholders in the room? And you want to make sure that you hear that holistic voice. That's been a major learning for me.
588
00:54:42.859 --> 00:54:45.420
And as people are evaluating
589
00:54:45.420 --> 00:54:47.180
different agent harnesses,
590
00:54:47.180 --> 00:54:50.619
what are the cases where you would say that TrueForge is the wrong choice?
591
00:54:52.105 --> 00:54:53.945
Overall, number one,
592
00:54:54.665 --> 00:55:02.265
at the moment, like, no, this is all a point in time answer. Right? So as of whatever, 09/04/2026,
593
00:55:02.265 --> 00:55:02.825
right,
594
00:55:04.105 --> 00:55:19.890
this answer likely is valid today. But but today, we are not specializing ourselves for a coding harness. So if somebody wanted to build out the users of the coding harness, probably I find out other open source or closed source harnesses that serve that use case better.
595
00:55:20.845 --> 00:55:24.365
In some other cases, if you really care about being able to
596
00:55:25.565 --> 00:55:27.725
basically, if you have a lot of visibility
597
00:55:27.965 --> 00:55:36.770
that roughly this is how my system should behave and these are the three lanes in which my agent should go on and there should not be any other fourth lane that they need to explore because
598
00:55:37.330 --> 00:55:46.530
I care about whatever ten millisecond latency implications, etcetera, etcetera, right? If such is your use case, then you are actually better off being able to
599
00:55:47.650 --> 00:55:50.475
more be in control of the code that you're writing and
600
00:55:50.955 --> 00:55:53.995
not let it run through a harness layer like TrueForge,
601
00:55:54.075 --> 00:55:54.795
I choose
602
00:55:55.355 --> 00:56:01.515
a more deterministic framework for building out your agents. I think those are a couple of situations where I would not pick TrueForge
603
00:56:01.515 --> 00:56:05.200
at the moment. If you're building anything that is more general purpose,
604
00:56:05.599 --> 00:56:06.080
that
605
00:56:06.720 --> 00:56:07.920
you care about
606
00:56:08.079 --> 00:56:09.280
the evolution
607
00:56:09.520 --> 00:56:11.520
of the agents over a period of time,
608
00:56:11.839 --> 00:56:15.279
you care about being able to build it quickly, you care about being able to build it efficiently,
609
00:56:15.725 --> 00:56:23.645
and you care about being in control of the type of models that you're able to use, you care about being able to control the cost, we feel like TrueForge is a great choice.
610
00:56:24.045 --> 00:56:32.779
And as you continue to build and iterate on these systems and continue to explore the current state of the ecosystem,
611
00:56:32.779 --> 00:56:38.859
what are some of the things you have planned for the near to medium term or any particular projects or problem areas that you're excited to explore?
612
00:56:41.035 --> 00:56:54.955
Absolutely. Absolutely. This is the thing that we have been talking a lot about. And a lot of our own roadmap is shaped directly by our customers as well, the current users of the platform as well. So
613
00:56:54.350 --> 00:57:01.470
one of the things that we have been very excited about is going much deeper into this entire agent identity area.
614
00:57:02.270 --> 00:57:06.270
And how do you manage cross app access? How do you manage
615
00:57:06.670 --> 00:57:09.965
this agent chaining, permission controls? And
616
00:57:10.605 --> 00:57:17.565
I've given an interesting problem statement here. I had mentioned briefly about how do you do least privilege across agent chaining. But
617
00:57:18.765 --> 00:57:24.490
when you start doing least privilege across agent chaining, one of the problems that happen is now your configuration
618
00:57:24.490 --> 00:57:31.290
management of which agents have access to which MCP's and which tools and which users have access to which MCP's and which agents,
619
00:57:31.849 --> 00:57:33.369
that becomes a
620
00:57:33.690 --> 00:57:35.130
very complex problem to solve.
621
00:57:35.744 --> 00:57:41.665
So then how do you take this into a direction where you're not compromising for the UX, where
622
00:57:42.385 --> 00:57:46.145
you understand the intent as well? It's not just about theoretically
623
00:57:46.305 --> 00:57:55.410
what is possible, you understand the intent when you implement that without compromising the user experience while ensuring the safety and security of the system. That's
624
00:57:55.410 --> 00:57:56.450
the direction that
625
00:57:57.010 --> 00:58:00.930
we are taking the product in. And then, of course, there's
626
00:58:01.250 --> 00:58:04.210
constant work happening around cost optimization,
627
00:58:04.610 --> 00:58:05.570
sandbox security.
628
00:58:06.275 --> 00:58:09.795
So some of those areas we are continuing to work on and that's
629
00:58:09.795 --> 00:58:11.555
part of our immediate roadmap as well.
630
00:58:11.955 --> 00:58:12.515
Are
631
00:58:12.755 --> 00:58:16.675
there any other aspects of the work that you're doing at TruFoundry
632
00:58:16.675 --> 00:58:23.250
or on TruForge or the overall space of Vagintic harnesses that we didn't discuss yet that you would like to cover before we close out the show?
633
00:58:23.970 --> 00:58:27.170
There's one thing that maybe I'll I'll call out here is,
634
00:58:27.570 --> 00:58:28.210
overall,
635
00:58:28.610 --> 00:58:29.170
the
636
00:58:29.570 --> 00:58:33.010
like, one of one of the learnings that we have had in this space is
637
00:58:33.485 --> 00:58:35.645
people have been trying to address
638
00:58:36.125 --> 00:58:36.605
problems
639
00:58:36.925 --> 00:58:41.165
around the control plane for enterprise AI or how
640
00:58:41.165 --> 00:58:43.165
you bring together this harness
641
00:58:43.325 --> 00:58:44.525
using smaller
642
00:58:45.725 --> 00:58:47.405
targeted point solutions.
643
00:58:47.810 --> 00:58:48.610
The
644
00:58:49.650 --> 00:58:51.170
goal has been for
645
00:58:51.330 --> 00:59:00.130
a lot of folks has been that, can I do my model routing there? Can I do my MCP registry there? Can I build out my A2A
646
00:59:00.525 --> 00:59:10.685
optimal agent to agent communication layer? People have been thinking a lot about that. One of the learnings that we have had working with the organizations that we work with is
647
00:59:12.444 --> 00:59:15.805
the only meaningful way
648
00:59:16.760 --> 00:59:20.280
that you build this overall control plane is if
649
00:59:22.359 --> 00:59:29.640
you bring these systems together. Like if you bring all of your tokens together in one place is how you only meaningfully solve this problem.
650
00:59:30.435 --> 00:59:36.915
So it turns out that a model proxy or a separate MCB proxy or an agent registry in isolation
651
00:59:36.915 --> 00:59:37.955
or a
652
00:59:38.195 --> 00:59:39.555
guardrail solution
653
00:59:40.355 --> 00:59:41.235
overall
654
00:59:41.235 --> 00:59:44.915
is just not able to solve the problem that organizations are facing because
655
00:59:45.910 --> 00:59:49.750
the true problems don't happen within these point
656
00:59:49.750 --> 00:59:50.550
solutions.
657
00:59:50.870 --> 00:59:54.070
The true problems happen at the interface of these point solutions.
658
00:59:54.550 --> 00:59:59.510
And to meaningfully solve for this entire thing, you need to bring these components together
659
00:59:59.605 --> 01:00:02.325
and do it in a very AI native way.
660
01:00:02.565 --> 01:00:04.085
That's been one of the learnings
661
01:00:04.485 --> 01:00:09.285
that I wanted to share with the folks as well. So whoever is thinking about the architecture
662
01:00:09.525 --> 01:00:10.085
of
663
01:00:10.885 --> 01:00:13.445
how to build and scale and manage all of their agents,
664
01:00:14.000 --> 01:00:16.480
maybe there's a considerable consideration to be made here.
665
01:00:17.040 --> 01:00:35.765
Alright. Well, for anybody who wants to get in touch with you and follow along with the work that you and your team are doing, I'll have you add your preferred contact information to the show notes. And as the final question, I'd like to get your perspective on what you see as being the biggest gaps in the tooling technology or human training that's available for AI systems today. This is such a deep topic.
666
01:00:35.925 --> 01:00:42.565
Right? It's it's it's very difficult to give a quick answer to this. But the way I'm seeing this is
667
01:00:43.340 --> 01:00:44.140
we need
668
01:00:45.020 --> 01:00:47.020
a holistic shift to
669
01:00:47.660 --> 01:00:50.780
solve for this entire problem statement. I literally talk to
670
01:00:52.060 --> 01:01:01.355
teenagers, college kids, and folks are asking what should we be thinking about? What's the next thing that we should be doing? What's the discipline that I should go study in?
671
01:01:01.915 --> 01:01:07.275
Should I still learn coding? Should I not learn coding? How do I upskill myself in this entire
672
01:01:07.435 --> 01:01:08.235
world of AI?
673
01:01:08.960 --> 01:01:17.200
And it's such a deep question that's there in everyone's mind that I feel like a tactical solution around how you learn AI
674
01:01:17.359 --> 01:01:24.495
or how you learn this latest new tool or any framework out there is just not an answer to the question. I think
675
01:01:25.375 --> 01:01:29.935
the only way to meaningfully solve this is what are some of the more fundamental
676
01:01:29.935 --> 01:01:33.215
layers where we expect human beings to
677
01:01:33.455 --> 01:01:39.710
contribute and add value. And what are the fundamental concepts, aspects in which we expect AI
678
01:01:39.870 --> 01:01:40.430
to
679
01:01:40.670 --> 01:01:46.990
assist us? We need to think about that. And that needs to get inculcated into our education system more fundamentally
680
01:01:47.315 --> 01:01:49.875
as opposed to solving this problem superficially.
681
01:01:49.875 --> 01:01:53.155
So I don't have a great answer to this question, but it's a problem that
682
01:01:53.795 --> 01:01:55.555
I resonate with very,
683
01:01:55.715 --> 01:01:58.515
very deeply through many of the latest conversations.
684
01:01:59.235 --> 01:02:00.195
Absolutely.
685
01:02:00.195 --> 01:02:01.395
Yeah. It's definitely
686
01:02:01.670 --> 01:02:02.550
a big
687
01:02:02.790 --> 01:02:12.870
a big thorny issue that the entire world is having to tackle right now. Even just in my own small ecosystem of working in an engineering team,
688
01:02:13.190 --> 01:02:23.715
it requires a lot of reimagining how you work every day because you can't human review everything the agent ships because otherwise, it'll actually never get to production. So it requires
689
01:02:23.795 --> 01:02:25.875
rethinking and reimagining
690
01:02:25.875 --> 01:02:28.435
entire processes and entire workflows,
691
01:02:28.435 --> 01:02:31.315
and it'll be an interesting few years to come.
692
01:02:32.170 --> 01:02:33.290
Absolutely.
693
01:02:33.290 --> 01:02:34.010
Absolutely.
694
01:02:34.250 --> 01:02:39.050
Well, Tobias, I really enjoyed the conversation. Thank you so much for having me on the show here.
695
01:02:39.370 --> 01:02:59.870
Yeah. Thank you very much for taking the time today to join me and share the work that you and your team are doing on TruFoundry and TruForge and sharing some of your lessons learned on harness engineering and AI gateways. It's definitely a very interesting space and very relevant topic, so I appreciate all of the time and effort you're putting into that. I hope you enjoy the rest of your day. Thank you so much. Take care.
696
01:03:01.470 --> 01:03:13.865
Thank you for listening. Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management, and podcast. In it covers the Python language, its community, and the innovative ways it is being used.
697
01:03:14.105 --> 01:03:24.731
Visit the site to subscribe to the show, sign up for the mailing list, read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email hosts@aiengineeringpodcast.com
698
01:03:24.731 --> 01:03:25.451
with your story.