WEBVTT
00:00:08.400 --> 00:00:10.160
Hello and welcome to CX Today.
00:00:10.400 --> 00:00:11.599
I'm Nicole Willing.
00:00:11.759 --> 00:00:22.640
Joining me is Michelle Sharmer, CTO at Search Unify, an enterprise agentic AI platform focused on unifying knowledge to power intelligent search, AI agents, and analytics for customers support.
00:00:22.960 --> 00:00:24.640
Michelle, it's great to have you here.
00:00:25.920 --> 00:00:26.559
Thank you, Nicole.
00:00:26.800 --> 00:00:27.679
Happy to be here.
00:00:28.559 --> 00:00:28.879
Great.
00:00:29.039 --> 00:00:42.960
So as I mentioned, you know, we're hearing about this shift from, you know, the first gen of kind of generative AI co-pilots, and we're now moving into this brave new world of autonomous AI agents acting on behalf of enterprises and customers.
00:00:43.039 --> 00:00:48.079
So what's the biggest um operational change that that really brings for customer support teams?
00:00:50.159 --> 00:00:50.560
Great.
00:00:51.039 --> 00:01:02.399
I'm glad you asked that question because at Search Unify uh we've also had a similar journey from find, sort of like assist and then act.
00:01:02.640 --> 00:01:16.400
We started as an enterprise search engine for customer support teams, and the idea was to give them a single box that they can use to find out any information which is sitting inside their organization while they're working on a support ticket.
00:01:16.640 --> 00:01:22.159
The second stage of evolution was more of how do we make this search box more powerful?
00:01:22.319 --> 00:01:29.680
And can we end up actually assisting these agents in the final uh actions that they want to take with this information?
00:01:29.760 --> 00:01:32.640
Which was more of like, can we help them draft a response?
00:01:32.879 --> 00:01:35.200
Can we help them quickly summarize a case?
00:01:35.439 --> 00:01:39.120
Can we help them do a little bit of a swamming also while working on a ticket?
00:01:39.200 --> 00:01:42.319
Because we can help them find the right experts they should talk to.
00:01:42.480 --> 00:01:54.480
And then finally, obviously, the the current stage of evolution, I would say, is more of an act phase, uh, which is more of getting into the autonomous mode.
00:01:54.640 --> 00:01:58.480
So our journey has been pretty much uh close to that.
00:01:58.640 --> 00:02:10.000
Um, but honestly, if you ask me, um I don't think I don't think most of the organizations are ready at this point of time to do a completely autonomous um shift.
00:02:10.159 --> 00:02:20.000
Um so they've been doing co-pilots, but if you ask me why they're not on completely autonomous support at this point of time, um I see there are primarily two reasons.
00:02:20.319 --> 00:02:34.960
One, I feel uh the underlying workflows should be redefined for completely autonomous systems or even assisted AI also needs to if it has to work, those underlying workflows should be revisited.
00:02:35.120 --> 00:02:40.240
Um for AI, at the end of the day, it is going to amplify your current system.
00:02:40.400 --> 00:02:44.080
So if you've got a crap in place, it is going to make it worse.
00:02:44.319 --> 00:02:50.240
If you've got a well-designed system for AI to take advantage of, it's going to amplify and make it great.
00:02:50.400 --> 00:02:54.159
So that's one part that the existing workflows should be revisited.
00:02:54.400 --> 00:03:04.319
But I think the second most important thing that we've been noticing as part of a lot of implementations is the underlying stack also needs to be changed for agents to consume.
00:03:04.560 --> 00:03:14.479
At the end of the day, um the support stack is going to be used by agents now in the autonomous world, not by just the human human beings.
00:03:16.960 --> 00:03:23.520
To give you an example, I would say the knowledge base, for example, historically was consumed by human agents.
00:03:23.680 --> 00:03:26.479
Now it has to be consumed by AI agents.
00:03:26.719 --> 00:03:31.439
So some sort of restructuring in that knowledge base is required.
00:03:31.680 --> 00:03:44.319
Similarly, for I think the support consoles also and in general the ticket workflows also, the the traditional thinking was to reduce the UI UX cognitive load for support engineers.
00:03:44.479 --> 00:03:56.639
But now with AI agents and system, more thinking has to go towards how do we make it API driven, um, more MCPs driven, uh, very efficient, multiple calls in a few seconds.
00:03:56.800 --> 00:03:58.879
So, how do you do that sort of a setup?
00:03:58.960 --> 00:04:15.280
So essentially, if you if you work on those two underlying blocks first and set those pillars in place, then I think um you as an organization is probably ready to make that shift from um assisted, co-pilot uh to autonomous.
00:04:15.680 --> 00:04:16.879
Yeah, yeah, exactly.
00:04:16.959 --> 00:04:22.160
Because you mentioned, you know, um enterprises aren't ready, but they're going ahead doing it.
00:04:22.480 --> 00:04:31.360
So why do you think that is that they're kind of leaping in and missing that they need to do more preparation before they get to that stage?
00:04:32.879 --> 00:04:46.160
Yes, I think a lot of preparation is required because I think um most so one of the mistakes that I see uh a lot of enterprises are doing is that assuming that it is going to automate everything.
00:04:46.399 --> 00:04:52.720
Um and that I feel is not a great way to look at uh adoption of AI into customer support.
00:04:52.879 --> 00:04:54.319
There are like simple use cases.
00:04:54.480 --> 00:05:04.000
You would always want to start with them first, for example, looking at the readiness of the ticket which is coming into the system, um, maybe classification of the ticket, maybe triaging of the ticket.
00:05:04.160 --> 00:05:06.399
So there are like some simple use cases.
00:05:06.560 --> 00:05:18.800
I would always recommend to start with them first uh before assuming that everything can be automated uh on Demon itself, because then I think we're going to create more chaos in the enterprise.
00:05:18.879 --> 00:05:19.279
Yeah.
00:05:19.839 --> 00:05:21.360
Yeah, because that's the thing, isn't it?
00:05:21.439 --> 00:05:24.480
And there's so much enthusiasm and excitement about AI.
00:05:24.639 --> 00:05:30.800
It seems that sometimes teams want to be able to use AI for their task they might not necessarily need to.
00:05:30.959 --> 00:05:41.199
Um, you know, they want to just throw AI and say, well, you know, we're implementing this agent or this system, but actually the task they're looking to accomplish with it might not necessarily need that.
00:05:42.959 --> 00:05:44.079
Yes, yes, absolutely.
00:05:44.160 --> 00:05:47.839
There are a lot of stuff that can be done with simple automation as well.
00:05:48.079 --> 00:05:54.319
Um and you don't need uh that fancy AI uh to be implemented and make it more complex.
00:05:54.480 --> 00:06:06.800
And what what you also have to understand at the end of the day is that within enterprises, you always have to be you you you would value a deterministic setup versus more probabilistic setup, right?
00:06:06.959 --> 00:06:10.480
You would need an accurate, reliable system to work.
00:06:10.639 --> 00:06:11.759
Um yeah.
00:06:12.160 --> 00:06:12.639
Yeah.
00:06:12.959 --> 00:06:23.040
So then um before if a company decides that an AI agent is appropriate, what guardrails should they put in place before they allow it to interact with customers?
00:06:24.319 --> 00:06:31.199
So guardrails are extremely important, and especially in enterprises, I would say um they become very, very important.
00:06:31.519 --> 00:06:41.920
Um, somebody was saying to me the other day um that AI and LLMs are more of like those eager employees for you who really want to show you the impact.
00:06:42.160 --> 00:06:52.319
And guardrails are more of like those managers for you who would want to make sure that your uh most employees are not making the costly mistakes for you.
00:06:52.480 --> 00:06:59.680
Um so guardrails are extremely important, and there are multiple levels at which you can you can go ahead and implement those guardrails.
00:07:00.000 --> 00:07:23.360
Um, for example, just to start with ensuring that your system is fairly accurate and giving that credibility to the end user that that that that end user or the human or the support engineer or your customer can actually trust on that system is you would implement a simple um uh simple retrieval engine to ground the responses.
00:07:23.519 --> 00:07:29.839
Uh, you would use a good rag engine, underlying rag engine to to ground your responses so that it's not just going anywhere.
00:07:30.000 --> 00:07:32.800
Um, you would want to build citations.
00:07:32.879 --> 00:07:41.519
Um, so anything that you're showing, uh, you have to give the credibility to the end user that, okay, here is the context that I use, the reference that I used in order to produce my answer.
00:07:41.759 --> 00:07:47.680
Um, you also have to define um what is that fine line where you have to say, I don't know.
00:07:47.759 --> 00:07:49.519
Uh, I really don't know the answer to that.
00:07:49.680 --> 00:07:53.839
And then is there a seamless transition to assisted support?
00:07:54.079 --> 00:07:56.560
Um, also, that's very well defined there.
00:07:56.720 --> 00:08:01.680
Um, so that's those are the the kind of things that you can do on the on more on the accuracy and experience side.
00:08:01.759 --> 00:08:05.279
But then there are other guardrails also, for example, on security and privacy.
00:08:05.680 --> 00:08:09.040
Have you implemented the PII uh checks in place?
00:08:09.279 --> 00:08:13.519
Have you uh on the semantic side, have you ensured there's no toxicity in the system?
00:08:13.680 --> 00:08:18.639
It's not really talking about your competition, it's not going um outside the boundaries.
00:08:18.800 --> 00:08:24.160
Um, and then also on the UX, also, I think there are certain levels of um guardrails that you can implement.
00:08:24.240 --> 00:08:26.720
That's more of like, are we stuck in a loop?
00:08:26.800 --> 00:08:31.680
Um, because uh if that is the case, then you really don't want to come back and use that system again.
00:08:31.839 --> 00:08:46.480
So, yeah, I mean, there are like multiple levels, um, and everything that you do in terms of the guardrails, just like security, like every layer that you add, it's going to make the system more robust, but then obviously it's going to make the system a little bit of slower as well, slow as well.
00:08:46.639 --> 00:08:52.879
Um, so you have to take a call at the end of the day that, okay, where should I then draw that line?
00:08:52.960 --> 00:09:02.240
Okay, this is fairly balanced for me in terms of cost, accuracy, and then um the the delay or the the other thing as well.
00:09:02.399 --> 00:09:04.080
Yeah, speed as well.
00:09:04.159 --> 00:09:04.480
Yeah.
00:09:04.879 --> 00:09:06.000
Yeah, definitely.
00:09:06.240 --> 00:09:22.559
And then, you know, we're starting to see now this talk of orchestration between you know multiple agents, um, you know, companies introducing now layers and of agents kind of with different tasks all working together and that showing up in support environments.
00:09:22.720 --> 00:09:27.039
So what has to be in place before companies then move to that step of deployment?
00:09:28.720 --> 00:09:31.759
Well, multi-hop system is a is a big leap, I would say.
00:09:31.919 --> 00:09:47.360
And honestly, if you ask me, um if you think your definition of agent is becoming too complex, too long, uh, there's always a possibility that this might just start um going south and hallucinate and then won't do the job.
00:09:47.519 --> 00:09:52.720
So it's always a good idea to break that into like small agents and have them work together.
00:09:52.879 --> 00:09:58.720
But then there are a lot of things that you have to keep in mind uh while designing a multi-agent system.
00:09:59.200 --> 00:10:04.559
Um and because at the end of the day, like one agent, very well defined outcome, very well defined job is fine.
00:10:04.639 --> 00:10:07.919
But then if there are multiple agents, coordination becomes a challenge.
00:10:08.159 --> 00:10:20.720
So um, as I said, like when you're defining a multi-agent system, the first thing you probably would want to do is that every agent has a very, very clear definition, what it is supposed to do, and then what are the outcomes uh of that agent.
00:10:20.879 --> 00:10:22.080
So that's number one.
00:10:22.320 --> 00:10:28.000
Second, I would say uh I think the the memory part is very important.
00:10:28.320 --> 00:10:43.519
Um, so because you don't want to be in a situation like um your front end, like the L1 support agent having a conversation with the customer, and then your LQ AI agent really has no idea what conversation was there uh with a with a frontline agent.
00:10:43.759 --> 00:10:46.000
So there has to be a concept of shared memory.
00:10:46.159 --> 00:10:49.600
You pass that signals and information from one agent to the other agent.
00:10:49.679 --> 00:10:51.360
That is extremely important as well.
00:10:51.519 --> 00:10:55.519
Otherwise, you really lose the whole context and there's no part of building a multi-agent system then.
00:10:55.679 --> 00:11:06.320
Um and I think the third point I would say, which is which kind of like crucial here, is that um a lot of times what you would see is that agents would start competing or blocking each other.
00:11:06.480 --> 00:11:18.639
Um, so general recommendation is that you really have a master agent doing the orchestration and then some sort of like you call them like uh the agents that are like uh following that master agent for the instruction.
00:11:18.799 --> 00:11:30.080
So this is why at least there's somebody sitting in the room and trying to unblock when you're like stuck on the same thing and trying to force each other that you should do this versus I should do that, that because that's like classic, we've seen that happening.
00:11:30.240 --> 00:11:36.399
Um so general recommendation is that like some orchestration needs to happen through a master agent as well.
00:11:36.559 --> 00:11:51.279
Um, but yeah, I mean, um we've we can see the the world is moving toward single agent system towards um multi-agent system, and then eventually, if you see like uh we've seen um different LLMs coming with different capabilities also.
00:11:51.440 --> 00:12:02.480
Once you design a multi-agent system, you get that flexibility to uh cut down the cost on on one use case because that specific agent is supposed to do just the summarization, right?
00:12:02.639 --> 00:12:05.759
For that, you don't need a very expensive LLM setup.
00:12:05.840 --> 00:12:17.200
Um so you can always plug to a low-cost LLM for that agent, and for somebody who's doing a deeper troubleshooting, you probably would want to run that uh with OPES or Sonnet or some of the entropic models as well.
00:12:17.279 --> 00:12:20.320
Um, so you have that flexibility in that multi-agent system, yeah.
00:12:20.639 --> 00:12:22.159
Yeah, yeah, exactly.
00:12:22.399 --> 00:12:25.840
So then uh I always like to end with the a forward-looking question.
00:12:26.000 --> 00:12:32.000
Um we the things move so quickly in AI that a few years anything can happen.
00:12:32.159 --> 00:12:41.600
But um, from your perspective um over the the near term, what do you think will separate the companies that get AI agents right from those that struggle?
00:12:42.559 --> 00:12:50.960
So, as I said, I think the definition of um documentation, I would say, of your current workflow is extremely important before you deploy any AI.
00:12:51.120 --> 00:12:58.559
You gotta make sure that you understand what is your current workflow, and you should be ready to take out certain items from there.
00:12:58.720 --> 00:13:09.759
Because I've seen a lot of companies coming back and saying that, okay, we need AI, we need agent TKI to work on uh on our existing setup, but uh but uh but they're not ready to change.
00:13:09.919 --> 00:13:17.200
And as I said, like for Agent TKI to work, you need to revisit your workflows and your processes and a lot of times at Tech Stack as well.
00:13:17.440 --> 00:13:32.639
Um, I think the organizations who are early adopters in terms of uh the ones who understand that some of these changes are required for these new EI agents to come and work, either as a coworker or in a completely autonomous fashion, uh, I think they're doing a great job.
00:13:32.799 --> 00:13:43.120
Uh, for example, like um deploying a frontline L1 support agent, uh, especially in high-tech, like where the cost of ticket can be$200,$200 to$250 per ticket.
00:13:43.200 --> 00:13:47.440
Uh, I see some customers taking huge success there for repetitive issues.
00:13:47.519 --> 00:13:49.919
Uh, they can deflect them through using AI agents.
00:13:50.159 --> 00:13:59.679
Similarly, a case quality, like you can like cover 100% of your tickets and do a case quality analysis now, versus historical days just taking one or two tickets.
00:13:59.840 --> 00:14:05.279
Similarly, for creating new knowledge base, um, now you're not dependent on one or two knowledge workers.
00:14:05.440 --> 00:14:10.480
Um, moment your ticket is closed, the resolution is captured somewhere in the closing comments.
00:14:10.559 --> 00:14:17.519
Uh, AI agent can capture that into a formal piece of knowledge space that can be used in future to avoid a similar ticket.
00:14:17.679 --> 00:14:27.679
So I think uh the organizations who are who are who understand that we have to make some changes in our existing environment for AI to come uh and help us, uh, they're seeing those early success.
00:14:28.240 --> 00:14:28.559
Yeah.
00:14:29.600 --> 00:14:37.919
It'll be exciting to see how uh these uh systems develop and the different um strategies that companies convey to do that.
00:14:38.480 --> 00:14:40.639
Thank you, Michelle, for taking the time.
00:14:40.799 --> 00:14:42.320
I really appreciate it.
00:14:42.480 --> 00:14:52.080
Um to for our audience, thank you for joining us and to learn more about um the issues that enterprises are facing in adopting a Gen CI.
00:14:52.320 --> 00:14:55.519
You can find more coverage on our website, cxday.com.
00:14:55.759 --> 00:15:00.320
Also, uh be sure to join our LinkedIn community and sign up for our newsletter.
00:15:00.559 --> 00:15:02.879
Thanks for watching, and we'll see you next time.
00:15:03.200 --> 00:15:04.000
Thank you, Nicole.
00:15:04.159 --> 00:15:05.120
Thank you, everyone.
00:00:08.400 --> 00:00:10.160
Hello and welcome to CX Today.
00:00:10.400 --> 00:00:11.599
I'm Nicole Willing.
00:00:11.759 --> 00:00:22.640
Joining me is Michelle Sharmer, CTO at Search Unify, an enterprise agentic AI platform focused on unifying knowledge to power intelligent search, AI agents, and analytics for customers support.
00:00:22.960 --> 00:00:24.640
Michelle, it's great to have you here.
00:00:25.920 --> 00:00:26.559
Thank you, Nicole.
00:00:26.800 --> 00:00:27.679
Happy to be here.
00:00:28.559 --> 00:00:28.879
Great.
00:00:29.039 --> 00:00:42.960
So as I mentioned, you know, we're hearing about this shift from, you know, the first gen of kind of generative AI co-pilots, and we're now moving into this brave new world of autonomous AI agents acting on behalf of enterprises and customers.
00:00:43.039 --> 00:00:48.079
So what's the biggest um operational change that that really brings for customer support teams?
00:00:50.159 --> 00:00:50.560
Great.
00:00:51.039 --> 00:01:02.399
I'm glad you asked that question because at Search Unify uh we've also had a similar journey from find, sort of like assist and then act.
00:01:02.640 --> 00:01:16.400
We started as an enterprise search engine for customer support teams, and the idea was to give them a single box that they can use to find out any information which is sitting inside their organization while they're working on a support ticket.
00:01:16.640 --> 00:01:22.159
The second stage of evolution was more of how do we make this search box more powerful?
00:01:22.319 --> 00:01:29.680
And can we end up actually assisting these agents in the final uh actions that they want to take with this information?
00:01:29.760 --> 00:01:32.640
Which was more of like, can we help them draft a response?
00:01:32.879 --> 00:01:35.200
Can we help them quickly summarize a case?
00:01:35.439 --> 00:01:39.120
Can we help them do a little bit of a swamming also while working on a ticket?
00:01:39.200 --> 00:01:42.319
Because we can help them find the right experts they should talk to.
00:01:42.480 --> 00:01:54.480
And then finally, obviously, the the current stage of evolution, I would say, is more of an act phase, uh, which is more of getting into the autonomous mode.
00:01:54.640 --> 00:01:58.480
So our journey has been pretty much uh close to that.
00:01:58.640 --> 00:02:10.000
Um, but honestly, if you ask me, um I don't think I don't think most of the organizations are ready at this point of time to do a completely autonomous um shift.
00:02:10.159 --> 00:02:20.000
Um so they've been doing co-pilots, but if you ask me why they're not on completely autonomous support at this point of time, um I see there are primarily two reasons.
00:02:20.319 --> 00:02:34.960
One, I feel uh the underlying workflows should be redefined for completely autonomous systems or even assisted AI also needs to if it has to work, those underlying workflows should be revisited.
00:02:35.120 --> 00:02:40.240
Um for AI, at the end of the day, it is going to amplify your current system.
00:02:40.400 --> 00:02:44.080
So if you've got a crap in place, it is going to make it worse.
00:02:44.319 --> 00:02:50.240
If you've got a well-designed system for AI to take advantage of, it's going to amplify and make it great.
00:02:50.400 --> 00:02:54.159
So that's one part that the existing workflows should be revisited.
00:02:54.400 --> 00:03:04.319
But I think the second most important thing that we've been noticing as part of a lot of implementations is the underlying stack also needs to be changed for agents to consume.
00:03:04.560 --> 00:03:14.479
At the end of the day, um the support stack is going to be used by agents now in the autonomous world, not by just the human human beings.
00:03:16.960 --> 00:03:23.520
To give you an example, I would say the knowledge base, for example, historically was consumed by human agents.
00:03:23.680 --> 00:03:26.479
Now it has to be consumed by AI agents.
00:03:26.719 --> 00:03:31.439
So some sort of restructuring in that knowledge base is required.
00:03:31.680 --> 00:03:44.319
Similarly, for I think the support consoles also and in general the ticket workflows also, the the traditional thinking was to reduce the UI UX cognitive load for support engineers.
00:03:44.479 --> 00:03:56.639
But now with AI agents and system, more thinking has to go towards how do we make it API driven, um, more MCPs driven, uh, very efficient, multiple calls in a few seconds.
00:03:56.800 --> 00:03:58.879
So, how do you do that sort of a setup?
00:03:58.960 --> 00:04:15.280
So essentially, if you if you work on those two underlying blocks first and set those pillars in place, then I think um you as an organization is probably ready to make that shift from um assisted, co-pilot uh to autonomous.
00:04:15.680 --> 00:04:16.879
Yeah, yeah, exactly.
00:04:16.959 --> 00:04:22.160
Because you mentioned, you know, um enterprises aren't ready, but they're going ahead doing it.
00:04:22.480 --> 00:04:31.360
So why do you think that is that they're kind of leaping in and missing that they need to do more preparation before they get to that stage?
00:04:32.879 --> 00:04:46.160
Yes, I think a lot of preparation is required because I think um most so one of the mistakes that I see uh a lot of enterprises are doing is that assuming that it is going to automate everything.
00:04:46.399 --> 00:04:52.720
Um and that I feel is not a great way to look at uh adoption of AI into customer support.
00:04:52.879 --> 00:04:54.319
There are like simple use cases.
00:04:54.480 --> 00:05:04.000
You would always want to start with them first, for example, looking at the readiness of the ticket which is coming into the system, um, maybe classification of the ticket, maybe triaging of the ticket.
00:05:04.160 --> 00:05:06.399
So there are like some simple use cases.
00:05:06.560 --> 00:05:18.800
I would always recommend to start with them first uh before assuming that everything can be automated uh on Demon itself, because then I think we're going to create more chaos in the enterprise.
00:05:18.879 --> 00:05:19.279
Yeah.
00:05:19.839 --> 00:05:21.360
Yeah, because that's the thing, isn't it?
00:05:21.439 --> 00:05:24.480
And there's so much enthusiasm and excitement about AI.
00:05:24.639 --> 00:05:30.800
It seems that sometimes teams want to be able to use AI for their task they might not necessarily need to.
00:05:30.959 --> 00:05:41.199
Um, you know, they want to just throw AI and say, well, you know, we're implementing this agent or this system, but actually the task they're looking to accomplish with it might not necessarily need that.
00:05:42.959 --> 00:05:44.079
Yes, yes, absolutely.
00:05:44.160 --> 00:05:47.839
There are a lot of stuff that can be done with simple automation as well.
00:05:48.079 --> 00:05:54.319
Um and you don't need uh that fancy AI uh to be implemented and make it more complex.
00:05:54.480 --> 00:06:06.800
And what what you also have to understand at the end of the day is that within enterprises, you always have to be you you you would value a deterministic setup versus more probabilistic setup, right?
00:06:06.959 --> 00:06:10.480
You would need an accurate, reliable system to work.
00:06:10.639 --> 00:06:11.759
Um yeah.
00:06:12.160 --> 00:06:12.639
Yeah.
00:06:12.959 --> 00:06:23.040
So then um before if a company decides that an AI agent is appropriate, what guardrails should they put in place before they allow it to interact with customers?
00:06:24.319 --> 00:06:31.199
So guardrails are extremely important, and especially in enterprises, I would say um they become very, very important.
00:06:31.519 --> 00:06:41.920
Um, somebody was saying to me the other day um that AI and LLMs are more of like those eager employees for you who really want to show you the impact.
00:06:42.160 --> 00:06:52.319
And guardrails are more of like those managers for you who would want to make sure that your uh most employees are not making the costly mistakes for you.
00:06:52.480 --> 00:06:59.680
Um so guardrails are extremely important, and there are multiple levels at which you can you can go ahead and implement those guardrails.
00:07:00.000 --> 00:07:23.360
Um, for example, just to start with ensuring that your system is fairly accurate and giving that credibility to the end user that that that that end user or the human or the support engineer or your customer can actually trust on that system is you would implement a simple um uh simple retrieval engine to ground the responses.
00:07:23.519 --> 00:07:29.839
Uh, you would use a good rag engine, underlying rag engine to to ground your responses so that it's not just going anywhere.
00:07:30.000 --> 00:07:32.800
Um, you would want to build citations.
00:07:32.879 --> 00:07:41.519
Um, so anything that you're showing, uh, you have to give the credibility to the end user that, okay, here is the context that I use, the reference that I used in order to produce my answer.
00:07:41.759 --> 00:07:47.680
Um, you also have to define um what is that fine line where you have to say, I don't know.
00:07:47.759 --> 00:07:49.519
Uh, I really don't know the answer to that.
00:07:49.680 --> 00:07:53.839
And then is there a seamless transition to assisted support?
00:07:54.079 --> 00:07:56.560
Um, also, that's very well defined there.
00:07:56.720 --> 00:08:01.680
Um, so that's those are the the kind of things that you can do on the on more on the accuracy and experience side.
00:08:01.759 --> 00:08:05.279
But then there are other guardrails also, for example, on security and privacy.
00:08:05.680 --> 00:08:09.040
Have you implemented the PII uh checks in place?
00:08:09.279 --> 00:08:13.519
Have you uh on the semantic side, have you ensured there's no toxicity in the system?
00:08:13.680 --> 00:08:18.639
It's not really talking about your competition, it's not going um outside the boundaries.
00:08:18.800 --> 00:08:24.160
Um, and then also on the UX, also, I think there are certain levels of um guardrails that you can implement.
00:08:24.240 --> 00:08:26.720
That's more of like, are we stuck in a loop?
00:08:26.800 --> 00:08:31.680
Um, because uh if that is the case, then you really don't want to come back and use that system again.
00:08:31.839 --> 00:08:46.480
So, yeah, I mean, there are like multiple levels, um, and everything that you do in terms of the guardrails, just like security, like every layer that you add, it's going to make the system more robust, but then obviously it's going to make the system a little bit of slower as well, slow as well.
00:08:46.639 --> 00:08:52.879
Um, so you have to take a call at the end of the day that, okay, where should I then draw that line?
00:08:52.960 --> 00:09:02.240
Okay, this is fairly balanced for me in terms of cost, accuracy, and then um the the delay or the the other thing as well.
00:09:02.399 --> 00:09:04.080
Yeah, speed as well.
00:09:04.159 --> 00:09:04.480
Yeah.
00:09:04.879 --> 00:09:06.000
Yeah, definitely.
00:09:06.240 --> 00:09:22.559
And then, you know, we're starting to see now this talk of orchestration between you know multiple agents, um, you know, companies introducing now layers and of agents kind of with different tasks all working together and that showing up in support environments.
00:09:22.720 --> 00:09:27.039
So what has to be in place before companies then move to that step of deployment?
00:09:28.720 --> 00:09:31.759
Well, multi-hop system is a is a big leap, I would say.
00:09:31.919 --> 00:09:47.360
And honestly, if you ask me, um if you think your definition of agent is becoming too complex, too long, uh, there's always a possibility that this might just start um going south and hallucinate and then won't do the job.
00:09:47.519 --> 00:09:52.720
So it's always a good idea to break that into like small agents and have them work together.
00:09:52.879 --> 00:09:58.720
But then there are a lot of things that you have to keep in mind uh while designing a multi-agent system.
00:09:59.200 --> 00:10:04.559
Um and because at the end of the day, like one agent, very well defined outcome, very well defined job is fine.
00:10:04.639 --> 00:10:07.919
But then if there are multiple agents, coordination becomes a challenge.
00:10:08.159 --> 00:10:20.720
So um, as I said, like when you're defining a multi-agent system, the first thing you probably would want to do is that every agent has a very, very clear definition, what it is supposed to do, and then what are the outcomes uh of that agent.
00:10:20.879 --> 00:10:22.080
So that's number one.
00:10:22.320 --> 00:10:28.000
Second, I would say uh I think the the memory part is very important.
00:10:28.320 --> 00:10:43.519
Um, so because you don't want to be in a situation like um your front end, like the L1 support agent having a conversation with the customer, and then your LQ AI agent really has no idea what conversation was there uh with a with a frontline agent.
00:10:43.759 --> 00:10:46.000
So there has to be a concept of shared memory.
00:10:46.159 --> 00:10:49.600
You pass that signals and information from one agent to the other agent.
00:10:49.679 --> 00:10:51.360
That is extremely important as well.
00:10:51.519 --> 00:10:55.519
Otherwise, you really lose the whole context and there's no part of building a multi-agent system then.
00:10:55.679 --> 00:11:06.320
Um and I think the third point I would say, which is which kind of like crucial here, is that um a lot of times what you would see is that agents would start competing or blocking each other.
00:11:06.480 --> 00:11:18.639
Um, so general recommendation is that you really have a master agent doing the orchestration and then some sort of like you call them like uh the agents that are like uh following that master agent for the instruction.
00:11:18.799 --> 00:11:30.080
So this is why at least there's somebody sitting in the room and trying to unblock when you're like stuck on the same thing and trying to force each other that you should do this versus I should do that, that because that's like classic, we've seen that happening.
00:11:30.240 --> 00:11:36.399
Um so general recommendation is that like some orchestration needs to happen through a master agent as well.
00:11:36.559 --> 00:11:51.279
Um, but yeah, I mean, um we've we can see the the world is moving toward single agent system towards um multi-agent system, and then eventually, if you see like uh we've seen um different LLMs coming with different capabilities also.
00:11:51.440 --> 00:12:02.480
Once you design a multi-agent system, you get that flexibility to uh cut down the cost on on one use case because that specific agent is supposed to do just the summarization, right?
00:12:02.639 --> 00:12:05.759
For that, you don't need a very expensive LLM setup.
00:12:05.840 --> 00:12:17.200
Um so you can always plug to a low-cost LLM for that agent, and for somebody who's doing a deeper troubleshooting, you probably would want to run that uh with OPES or Sonnet or some of the entropic models as well.
00:12:17.279 --> 00:12:20.320
Um, so you have that flexibility in that multi-agent system, yeah.
00:12:20.639 --> 00:12:22.159
Yeah, yeah, exactly.
00:12:22.399 --> 00:12:25.840
So then uh I always like to end with the a forward-looking question.
00:12:26.000 --> 00:12:32.000
Um we the things move so quickly in AI that a few years anything can happen.
00:12:32.159 --> 00:12:41.600
But um, from your perspective um over the the near term, what do you think will separate the companies that get AI agents right from those that struggle?
00:12:42.559 --> 00:12:50.960
So, as I said, I think the definition of um documentation, I would say, of your current workflow is extremely important before you deploy any AI.
00:12:51.120 --> 00:12:58.559
You gotta make sure that you understand what is your current workflow, and you should be ready to take out certain items from there.
00:12:58.720 --> 00:13:09.759
Because I've seen a lot of companies coming back and saying that, okay, we need AI, we need agent TKI to work on uh on our existing setup, but uh but uh but they're not ready to change.
00:13:09.919 --> 00:13:17.200
And as I said, like for Agent TKI to work, you need to revisit your workflows and your processes and a lot of times at Tech Stack as well.
00:13:17.440 --> 00:13:32.639
Um, I think the organizations who are early adopters in terms of uh the ones who understand that some of these changes are required for these new EI agents to come and work, either as a coworker or in a completely autonomous fashion, uh, I think they're doing a great job.
00:13:32.799 --> 00:13:43.120
Uh, for example, like um deploying a frontline L1 support agent, uh, especially in high-tech, like where the cost of ticket can be$200,$200 to$250 per ticket.
00:13:43.200 --> 00:13:47.440
Uh, I see some customers taking huge success there for repetitive issues.
00:13:47.519 --> 00:13:49.919
Uh, they can deflect them through using AI agents.
00:13:50.159 --> 00:13:59.679
Similarly, a case quality, like you can like cover 100% of your tickets and do a case quality analysis now, versus historical days just taking one or two tickets.
00:13:59.840 --> 00:14:05.279
Similarly, for creating new knowledge base, um, now you're not dependent on one or two knowledge workers.
00:14:05.440 --> 00:14:10.480
Um, moment your ticket is closed, the resolution is captured somewhere in the closing comments.
00:14:10.559 --> 00:14:17.519
Uh, AI agent can capture that into a formal piece of knowledge space that can be used in future to avoid a similar ticket.
00:14:17.679 --> 00:14:27.679
So I think uh the organizations who are who are who understand that we have to make some changes in our existing environment for AI to come uh and help us, uh, they're seeing those early success.
00:14:28.240 --> 00:14:28.559
Yeah.
00:14:29.600 --> 00:14:37.919
It'll be exciting to see how uh these uh systems develop and the different um strategies that companies convey to do that.
00:14:38.480 --> 00:14:40.639
Thank you, Michelle, for taking the time.
00:14:40.799 --> 00:14:42.320
I really appreciate it.
00:14:42.480 --> 00:14:52.080
Um to for our audience, thank you for joining us and to learn more about um the issues that enterprises are facing in adopting a Gen CI.
00:14:52.320 --> 00:14:55.519
You can find more coverage on our website, cxday.com.
00:14:55.759 --> 00:15:00.320
Also, uh be sure to join our LinkedIn community and sign up for our newsletter.
00:15:00.559 --> 00:15:02.879
Thanks for watching, and we'll see you next time.
00:15:03.200 --> 00:15:04.000
Thank you, Nicole.
00:15:04.159 --> 00:15:05.120
Thank you, everyone.
