Conquer QA Overwhelm with Stacy’s Fearless Ten-Ticket Method | Episode 20

WEBVTT
00:00:00.240 --> 00:00:04.011
Consistent quality review, I think, is better than no quality review.
00:00:04.011 --> 00:00:17.272
Since we are a small group of people who are carving out time to do this consistently, we decided to go with a cadence of 10 support interactions a month.
00:00:18.060 --> 00:00:21.411
Hey friends, welcome back to Live Chat with Jen Weaver.
00:00:21.411 --> 00:00:39.301
It's not often that I get to do a podcast episode with somebody who I admire as a professional in our field, but who also has been a great friend and a mentor to me, so I'm super excited to talk today with Stacey Justino, who's the product support manager at PetDesk.
00:00:39.301 --> 00:00:47.665
Stacey arrived last fall and spun up a brand new quality program for their veterinary support in just a few weeks.
00:00:47.665 --> 00:00:51.750
Now you know we're all about QA here at Support Band.
00:00:51.750 --> 00:01:07.861
That's what we do, so I'm also super excited to unpack her topic her 10-ticket cadence, the four-point rubric her team actually uses and the simple Google Sheets trick that keeps her QA reviews fair and random.
00:01:07.861 --> 00:01:16.111
Stacey spun up this program to be lightweight so that she could actually keep it going over time with minimal overhead.
00:01:16.531 --> 00:01:22.909
I think it will be really helpful for support leaders who don't maybe have a ton of resources for building a QA program.
00:01:22.909 --> 00:01:25.165
Let's be real, that's probably most of us.
00:01:25.165 --> 00:01:37.600
Before we get started, though, our QA tool, supportman, is what makes this podcast possible, so if you're listening to this podcast, head over to the YouTube link in the show notes to get a glimpse.
00:01:37.600 --> 00:01:51.787
Supportman sends real-time QA from intercom to Slack with daily threads, time QA from intercom to Slack with daily threads, weekly charts and done-for-you AI-powered conversation evaluations.
00:01:51.787 --> 00:02:00.087
It makes it so much easier to QA intercom conversations right where your team is already spending their day in Slack.
00:02:00.087 --> 00:02:03.233
All right on to today's episode.
00:02:26.580 --> 00:02:28.444
All right on to today's episode, like for for stacy in a week at work.
00:02:28.444 --> 00:02:32.455
First, there's something that's like maybe starting with things that like are multiple times a day, sort of things.
00:02:32.455 --> 00:02:33.217
Yeah, checking on the chat queue.
00:02:33.217 --> 00:02:37.937
Uh, checking out our ticket queues, is there any ticket that's been sitting there at the top for a while?
00:02:37.937 --> 00:02:39.722
Let's make sure that somebody gets on that.
00:02:39.722 --> 00:02:42.247
Oh, we have six people waiting in chat.
00:02:42.247 --> 00:02:43.909
We have wait times about three minutes.
00:02:43.909 --> 00:02:52.173
Let's post in Slack to say, hey, we need anybody who's not on break at lunch or in a meeting to grab a chat.
00:02:52.173 --> 00:02:55.549
So that's something that's happening, you know, multiple times.
00:02:55.568 --> 00:03:01.671
Throughout the day Beginning of the week, there's, you know, some weekly reporting that needs to get done.
00:03:01.671 --> 00:03:09.585
So doing that on Monday, weekly reporting that needs to get done.
00:03:09.585 --> 00:03:10.287
So doing that on Monday.
00:03:10.287 --> 00:03:12.574
In terms of other stuff, you know there's projects to be done.
00:03:12.574 --> 00:03:16.002
So we have, you know, we call our quarterly projects, or OKRs, like rocks.
00:03:16.002 --> 00:03:26.281
So making sure that I have time and I try to book at least a few hours in my calendar every week to make sure I can focus on those things.
00:03:26.981 --> 00:03:28.264
And where do you keep that?
00:03:28.264 --> 00:03:29.388
What is that?
00:03:29.388 --> 00:03:31.352
Do you do that like on paper?
00:03:31.352 --> 00:03:32.662
Or In a notebook.
00:03:32.662 --> 00:03:34.108
Nice, same girl.
00:03:37.147 --> 00:03:40.120
And then I have another colored pen where I check it so I can check out the boxes.
00:03:40.300 --> 00:03:42.801
Of course, yeah, colored pens are where it's at.
00:03:42.801 --> 00:03:45.302
So you're talking about like planning out your week.
00:03:45.302 --> 00:04:07.141
I'm going to try that out, that simple little just like writing down what needs to be done today and then this week and with due sort of like for weekly reporting, um that needs to get done, um.
00:04:07.362 --> 00:04:13.626
And then I have one-on-ones with each of my direct reports 30 minute one-on-ones with each of my direct reports, um, each week.
00:04:13.626 --> 00:04:16.190
So have those sort of sprinkled throughout.
00:04:16.190 --> 00:04:26.894
I try to put them in like blocks of two or three, so not too long of blocks, but that I, I like, I'm in that mindset Like, okay, this is my time one-on-one with the people who report to me.
00:04:26.894 --> 00:04:32.423
So I really aim to be present and not be distracted during those times.
00:04:32.423 --> 00:04:36.451
And then, of course, we have our team meetings on Tuesdays and Thursdays.
00:04:36.451 --> 00:04:46.392
I also have a weekly meeting with our senior product support specialist and our product solutions advisor on Tuesdays, as well as a Monday and Wednesday meeting with the support leadership team.
00:04:46.392 --> 00:04:54.028
So it's with our boss, the senior director of global support, and the support managers of all of the pet advisors.
00:04:54.329 --> 00:04:59.228
Pet desk support teams and the other support teams serve other parts of the business.
00:04:59.288 --> 00:05:04.843
Other tools yes, we collaborate a bunch because of the way the products interact with each other.
00:05:04.843 --> 00:05:06.985
Yes, we collaborate a bunch because of the way the products interact with each other.
00:05:06.985 --> 00:05:13.711
We now have a weekly pet desk communication, support and customer success leadership meeting which has been really great.
00:05:14.211 --> 00:05:15.971
Today, we're doing something a little bit different.
00:05:15.971 --> 00:05:21.237
We're going to take a deep dive into your QA process for your support team.
00:05:21.237 --> 00:05:31.952
Will you just kind of get me started on where you work and what the context for the conversation is yeah, so I am a product support manager at PetDesk.
00:05:33.120 --> 00:05:45.634
Petdesk is a company that has multiple products, but I specifically support our PetDesk communications product, which is basically kind of like a CRM for veterinary providers that integrates with their practice management system.
00:05:45.634 --> 00:05:50.547
And we also have an accompanying app for pet parents.
00:05:50.547 --> 00:06:12.434
So veterinary provider has our clients that have booked appointments and then, if they have a pet test communication subscription, we can handle appointment reminders, health service reminders oh Pixie is overdue for a rabies vaccine and we can send those kind of reminders.
00:06:12.434 --> 00:06:18.841
And then the pet parent can see all their appointments in the app, see a lot of other information.
00:06:18.841 --> 00:06:26.512
If the clinic has it set up, they can earn loyalty points or reorder prescriptions for their pet.
00:06:26.512 --> 00:06:41.245
So that's kind of the scope of what my team does, and we hadn't had a quality program before I got here and I started a pet desk in end of October 2024.
00:06:41.245 --> 00:06:45.071
And so we spun one up.
00:06:45.791 --> 00:06:46.533
So that's exciting.
00:06:46.533 --> 00:06:53.310
But putting that aside, the quality program that you created, can you tell me a little bit more about, like, how you got started?
00:06:53.310 --> 00:06:54.012
It must have been.
00:06:54.012 --> 00:06:59.512
You saw the absence of a quality program and you thought this is an initiative I can run.
00:06:59.512 --> 00:07:00.942
Did I get that right?
00:07:01.242 --> 00:07:01.423
Yes.
00:07:01.762 --> 00:07:01.923
Yes.
00:07:02.245 --> 00:07:06.413
So we already had some loose sort of quality standards.
00:07:06.413 --> 00:07:19.089
One of our senior product support specialists those are the folks who do training and onboarding of new people David, he would always go over sort of quality what does quality mean when it comes to pet desk support and how do you do that?
00:07:19.089 --> 00:07:27.882
So we already had a sort of a basis for that and I was lucky to come into an organization where quality was already sort of emphasized.
00:07:27.882 --> 00:07:35.668
You know it wasn't straight up like answer as many tickets as you can, so that foundation was already there.
00:07:35.668 --> 00:07:42.843
So I wasn't sort of starting from a point of sort of oh, our quality needs tons and tons of work.
00:07:42.843 --> 00:07:49.471
We need to put this in place, because where we're at versus where we need to be is a pretty big gap.
00:07:50.153 --> 00:07:59.072
So that was a good starting place and I think the team, like all of the specialists on the team, were eager for more regular feedback.
00:07:59.072 --> 00:08:10.355
Right, because basically the feedback mechanisms up until we launched the quality program was oh, ticket got escalated by a CSM or negative CSAT, right.
00:08:10.355 --> 00:08:18.206
So or in the cases where you know specialists on the team in their one-on-ones with their manager would be like, hey, I think I could have done better on this.
00:08:18.206 --> 00:08:19.310
Can we talk about this ticket?
00:08:19.310 --> 00:08:24.206
So a little haphazard and not consistent and very reactive.
00:08:24.949 --> 00:08:28.468
Okay, cool, yeah, and so you saw a big opportunity to systematize that.
00:08:28.468 --> 00:08:29.750
Where did you start?
00:08:30.822 --> 00:08:33.320
Well, I drew on a lot of my previous experiences.
00:08:33.320 --> 00:08:42.447
So when I was at Big Fish Games, I ran the quality team within the customer support team, so you have some experience with that.
00:08:42.587 --> 00:08:44.544
Yes, yes, and also at Wistia.
00:08:44.544 --> 00:08:47.503
I spun up a quality program there.
00:08:47.503 --> 00:09:01.422
So I was able to take a lot of what I'd found success with in the past and we decided to sort of start with what the quality guidelines were already in place, right for the team.
00:09:01.422 --> 00:09:03.005
So what did we?
00:09:03.005 --> 00:09:11.431
We cover in training, what did we have in our one guru card around quality and um, rather than sort of totally rework it.
00:09:11.431 --> 00:09:31.220
But then I incorporated some of the things that I found pretty important and foundational in a quality rubric, um, and then, since the folks doing the reviews are the two managers and three senior product support specialists who, spoiler alert, we all have a lot of responsibilities already on our plate.
00:09:31.360 --> 00:09:38.875
So I had to really wanted to focus on something that was something we could do and deliver on consistently.
00:09:38.875 --> 00:09:57.874
I know a lot of organizations, especially ones with a dedicated quality team or that are using software that enables you to do things a little more easily.
00:09:57.874 --> 00:10:09.442
I know that those orgs tend to do like a percentage of total interactions small group of people who are carving out time to do this consistently.
00:10:09.442 --> 00:10:31.249
We decided to go with a cadence of 10 support interactions a month, so two QA reviews each month, five tickets in the first review, five tickets in the second review, so that people get regular feedback review so that people get regular feedback but it's not too cumbersome for the folks doing the reviews.
00:10:31.269 --> 00:10:42.100
Yeah, that makes total sense, because you don't want you're trying to initiate this program and get buy-in and you don't want people to, you know, feel overwhelmed and feel like it's an addition to their workflow.
00:10:42.100 --> 00:10:49.785
But that brings me to a burning question a lot of people have about QA how do you choose which conversations?
00:10:49.785 --> 00:10:54.208
Is it the ones with negative CSAT or is it random?
00:11:00.052 --> 00:11:11.966
be random but with some parameters, to make sure that you are selecting tickets that are sort of representative of the work that they do, right?
00:11:11.966 --> 00:11:28.812
So in our case we have live channels, we have folks who do phone support and tickets and folks that do live chat support and tickets and the phone volume and the chat volume is greater percentage than the ticket volume, the handle.
00:11:28.812 --> 00:11:49.981
So for folks who are working live channels, we pick over the course of the month for the first QA, we will pick three chats or phone calls, depending on which channel they are on, and then two email tickets are on and then to email tickets.
00:11:49.981 --> 00:11:55.013
So that's kind of how we make sure that it's representative of sort of their work over the course of the month.
00:11:55.013 --> 00:11:56.559
And then, in terms of other sort of things we look at.
00:11:56.721 --> 00:11:59.349
You know, recency is pretty important, right.
00:11:59.349 --> 00:12:05.861
I always want the person whose ticket is being reviewed to be able to remember that ticket, right, of course?
00:12:05.861 --> 00:12:11.892
Yeah, that makes sense, and so if you pick a ticket, they handle it in the beginning of the month, you know, and this is their second QA review.
00:12:11.892 --> 00:12:14.799
Then it's not as impactful, right?
00:12:14.799 --> 00:12:21.614
So I have set so we're using Zendesk and so I've created a report in Zendesk.
00:12:21.614 --> 00:12:27.263
Unfortunately, I have to create two reports, one for chats and one for tickets, because of how the data sets work.
00:12:27.263 --> 00:12:38.408
But we look at tickets that were solved within the past seven days and exclude some ticket types.
00:12:38.408 --> 00:12:44.366
Like we have a not for support ticket type where it's meant for another team, or we're just escalating to the CSM.
00:12:44.366 --> 00:13:11.389
I haven't gone too far into like adding more filters, but that is sort of how we select them and then I make sure that the ticket created date is the day after they were emailed their feedback so that we're reviewing tickets that they interacted with after their last round of feedback.
00:13:12.115 --> 00:13:14.181
Yeah, so it's not reaching back into the past.
00:13:14.181 --> 00:13:17.376
So you have these sets of time that you're QAing.
00:13:17.376 --> 00:13:27.284
That's really interesting and the filters you're using are essentially it needs to be a recent conversation and it needs to be representative of wherever they're working.
00:13:27.284 --> 00:13:31.859
So email and chat or email and phone, that makes total sense.
00:13:31.859 --> 00:13:46.499
And so just to get a handle on what you're working with, what's the typical volume of tickets or total conversations that your team is handling On an average month.
00:13:46.678 --> 00:13:52.929
A single support specialist on our team probably is doing between 350 to 400.
00:13:52.929 --> 00:13:56.524
So it's like a pretty small percentage that we're looking at over the course of the month.
00:13:56.524 --> 00:14:01.645
But consistent quality review I think is better than no quality review.
00:14:02.246 --> 00:14:03.916
Yeah, absolutely, and especially.
00:14:03.916 --> 00:14:18.469
It makes sense to me that you're starting this program with you don't have a QA team, right, and so you're being very mindful of the workload that the managers you said it's team leads and managers who are doing QA.
00:14:18.556 --> 00:14:22.623
Senior product support specialists they're kind of like a tier two type role.
00:14:22.802 --> 00:14:23.325
That makes sense.
00:14:23.325 --> 00:14:24.226
They've been there a while.
00:14:24.226 --> 00:14:25.660
They know what a good ticket looks like.
00:14:25.660 --> 00:14:30.701
Exactly so, as far as the people who are doing QA, it's the senior specialists.
00:14:30.701 --> 00:14:32.341
What about who receives QA?
00:14:32.341 --> 00:14:34.101
Is it everyone, or is it new people?
00:14:35.336 --> 00:14:39.642
All specialists, like we have level one, level two and then we have our senior product support specialists.
00:14:39.642 --> 00:14:43.046
So all level one and level two folks get QA.
00:14:43.046 --> 00:14:50.263
The way we've sort of mapped it out is the you don't get QA during your training and onboarding.
00:14:50.263 --> 00:15:00.830
And then we recently just decided that for that first month post-onboarding, the QA will be done by the senior product support specialist, because those are the folks that they've been interacting with the most.
00:15:00.830 --> 00:15:07.903
And then after that first month that they're like fully trained then and they'll also deliver that feedback right and they'll also deliver that feedback right.
00:15:07.903 --> 00:15:14.850
And then the next month it depends on how we've sort of assigned who's going to QA who for the month.
00:15:14.850 --> 00:15:27.034
But then the manager will be the one who has that feedback session going forward, because then they'll already have a couple one-on-ones under their belt with their manager and have built some rapport before the manager takes over that QA feedback.
00:15:27.695 --> 00:15:30.644
Yeah, so I'm getting a picture of a really gentle process for everyone.
00:15:30.644 --> 00:15:33.844
Right, it's gentle for the folks who have onboarded.
00:15:33.844 --> 00:15:44.407
It eases them into it, and then it allows them some time with their manager to kind of just gain rapport and get to know them before they start doing QA with their manager.
00:15:44.407 --> 00:15:47.061
Yeah that makes total sense.
00:15:47.061 --> 00:15:51.942
And is it the senior specialist who does training?
00:15:51.942 --> 00:15:53.245
Like are they doing onboarding at?
00:15:53.265 --> 00:16:10.895
the new timber Okay so it's an extension of training, exactly, exactly, exactly that first month, getting sort of folded into the QA process is an extension of training this program.
00:16:10.916 --> 00:16:17.179
If I'm a customer support leader who's looking ahead to a team that doesn't have a quality program, what would you say is the biggest mistake I'm likely to make that you could maybe help me avoid?
00:16:17.179 --> 00:16:17.962
Well?
00:16:18.124 --> 00:16:51.885
I think the first thing would be like focusing on like negative CSATs, because I think that's what people think they should do, but that's not really representative of probably the majority of their work, and so I think it's still important to review those, and I think that I was actually just having a conversation earlier today with someone about this and say, hey, I think that if you want to review those against your quality rubric, that's perfectly good and okay, but, like if their quality score is going to be part of their performance guidelines and performance expectations, that is not fair and equitable.
00:16:51.885 --> 00:17:04.414
To have your like ticket selection criteria be all negative c-sats plus a handful of random ones, because that's going to tank their score and is, like, like I said, not representative.
00:17:04.414 --> 00:17:07.457
So like to me is making sure that that's one of the things.
00:17:07.457 --> 00:17:10.598
Keys to success is the quality the tickets.
00:17:10.618 --> 00:17:14.101
Google Sheets has a really cool feature where you can randomize a range.
00:17:14.101 --> 00:17:40.067
So you just copy all of the rows that you want to do this to and you right-click.
00:17:40.067 --> 00:17:56.364
In the very bottom there's like additional options, I think it's something like that and it says randomize range, and so I just filter by the one specialist randomized range and then click through the tickets like top down, because they're in a random order, I do sort of look into them.
00:17:56.364 --> 00:17:59.503
Yeah, I figured that out like two months ago when I was first doing this.
00:17:59.503 --> 00:18:00.980
I was like there's got to be a way to do this.
00:18:00.980 --> 00:18:13.784
So I was trying to use the random function, but that's annoying because it refreshes and so you have to, like, use the random function, copy and paste this value and then sort um, and so this was much easier to do, yeah that sounds great.
00:18:14.184 --> 00:18:19.381
You copy it into a new sheet so that you're yeah, you're working with separate data.
00:18:19.842 --> 00:18:40.928
Yes, yeah um, and so what I also do is I'll click through those tickets because because of certain situations with our software, we get a lot of tickets from providers who are reaching out to support to say, hey, can you update this client's phone number, can you update this client's name, and so that's a big percentage, but that's so.
00:18:40.928 --> 00:18:47.619
That is the one case where it's like the tickets I'm selecting aren't maybe quite representative of like 30 percent of our tickets.
00:18:47.619 --> 00:18:48.961
Are these account modification tickets?
00:18:48.961 --> 00:18:57.921
But I don't want to QA 30% of their tickets being those, because those are sort of very much standard use, a macro updated thing in our tools.
00:18:57.921 --> 00:19:00.747
So I try to make sure that there's a balance.
00:19:00.767 --> 00:19:06.003
So it's like not all of one ticket type because that's not helpful to them.
00:19:06.003 --> 00:19:09.969
So still random in terms of still going top down.
00:19:09.969 --> 00:19:27.041
But what I'm kind of working through now is, now that I've gone through this for a couple months, let me add to our guru card about support interaction selection criteria and I want to put that in there because I think one of the things to your question is transparency, like being really transparent about the process.
00:19:27.041 --> 00:19:36.817
So the cadence, the support interaction selection criteria, all that stuff is in our guru card about our quality program that everybody has access to.
00:19:37.298 --> 00:19:46.881
So, speaking of transparency, do you feel like it's always a good idea to do QA more privately?
00:19:46.881 --> 00:20:01.644
So if I'm giving a specialist feedback, what are the pros and cons to doing that in a DM or in a meeting, versus maybe sometimes shouting it out that the specialist has done a great job in putting that in a team channel?
00:20:01.644 --> 00:20:03.007
Do you have thoughts about that?
00:20:03.576 --> 00:20:07.820
I think there's situations where both of those are probably good mechanisms.
00:20:08.301 --> 00:20:33.836
I think the one-on-one session just gives them this face-to-face time to get to talk through it, and I think one of the things that also I think is really important is you need to give mechanisms and your team really needs to feel that if they don't agree with something, that they can bring it up right with something that they can bring it up right.
00:20:33.836 --> 00:20:36.301
So that to me is better and you can establish that sort of rapport in like a one-on-one session.
00:20:36.301 --> 00:20:42.702
But I think, as like a team culture, shouting out when people do a really great job is something that should be happening right.
00:20:42.702 --> 00:20:50.067
So like we have two team huddles every week and in the first one of the week I always do a CSAT shout out.
00:20:50.067 --> 00:21:01.310
So, and one of the great things about having a quality program when you do that is you can call out things that are in your quality rubric about what that person did really well.
00:21:01.310 --> 00:21:09.689
So that's the other thing I think to make it successful is everybody needs to really understand and buy into your rubric.
00:21:10.295 --> 00:21:11.701
I love that you mentioned this rubric.
00:21:11.701 --> 00:21:17.228
I would like to dive into what your rubric is, how you determined it.
00:21:17.228 --> 00:21:23.103
Is that something that came from your previous work, or did you kind of tailor that to your new company?
00:21:23.103 --> 00:21:24.125
Both.
00:21:25.955 --> 00:21:38.330
So David, the senior support specialist I mentioned, he had been working on a rubric with the previous manager and so took some of that because that was based on the quality training that new hires on our team go through.
00:21:38.330 --> 00:21:55.244
And then I incorporated, like I said, some of the elements of the criteria that I've used in the past that I found that I think is pretty foundational, incorporated those together to come up with what our pet desk communication support team quality criteria are.
00:21:55.244 --> 00:22:11.757
So, for example, our four criteria at pet desk for pet desk communications team are accuracy, completeness are accuracy, completeness, customer excellence and empathy tone.
00:22:11.757 --> 00:22:14.099
We're dealing with veterinary providers.
00:22:14.099 --> 00:22:18.023
It's a very high-stress job, it's very busy, right.
00:22:18.023 --> 00:22:25.090
So empathy tone might not be a full criteria in some places, but for us it was really important.
00:22:27.454 --> 00:22:30.285
So there's some things that are sort of universal in terms of, like, a quality support experience.
00:22:30.285 --> 00:22:41.718
But then I would always recommend to someone that one of your criteria should be really rooted in, like your company values or what is sort of specific to your company.
00:22:41.718 --> 00:23:15.691
So one of the elements of customer excellence is guidance right Controlling the communication, guiding to the best options, effective solution, providing additional information to the customer to address their next question or issue based upon the original reason for writing in that last one is actually one that I carried over from Big Fish Games and Wistia, but we put it under this guidance Like we need to serve as guides sometimes for folks, whereas in other support orgs that I've led that hasn't been as important.
00:23:18.055 --> 00:23:18.556
That makes perfect sense.
00:23:18.556 --> 00:23:24.407
So it's almost the customer profile that guides that rubric quality.
00:23:24.428 --> 00:23:25.537
Correct Interesting.
00:23:26.078 --> 00:23:31.723
I just wonder if you have any other advice for customer leaders who are working on quality programs.
00:23:32.325 --> 00:24:05.788
I think one piece of advice is, if you're incorporating the quality score into your monthly performance or quarterly performance for individuals in your team, is making sure that whatever benchmark you've set for your internal quality score and what you're looking for in your rubric is complementary to sort of your productivity expectations or your workflows or your business needs, sometimes right.
00:24:05.788 --> 00:24:11.481
So I can give a real-life example of where this happened when I was at Big Fish Games.
00:24:11.481 --> 00:24:20.887
So when I was at Big Fish Games we had monthly performance and we had sort of meets expectations and exceeds expectations and like needs improvement.
00:24:20.887 --> 00:24:26.106
But there were and we had monthly bonuses for our support team.
00:24:26.106 --> 00:24:33.249
So, um, turn a level one bonus, you had to meet expectations and productivity quality c-set.
00:24:33.249 --> 00:24:48.057
And then, uh, have we had a maximum threshold for average number of replies per ticket to make sure that like they're really productive, but on average it's taking three replies to solve a ticket.
00:24:48.057 --> 00:24:52.445
We should not be giving a bonus for that behavior.
00:24:52.445 --> 00:24:55.477
So that was sort of like the like a threshold.
00:24:55.477 --> 00:25:00.326
So but we have that level one and level two criteria for quality, right.
00:25:00.326 --> 00:25:06.846
So it's like 85 if you got an 85 to 90 qa score, you were meeting expectations.
00:25:06.846 --> 00:25:09.461
If you get 90 or above, you're exceeding expectations.
00:25:09.461 --> 00:25:18.375
Um, and then we also had minimum standards which, if you missed one of those minimum standards you were automatically at means improvement for quality.
00:25:19.276 --> 00:25:28.690
And so we basically created this system with the best of intentions that somebody could miss a minimum standard on their first QA.
00:25:28.690 --> 00:25:31.082
They know they're not going to get a bonus.
00:25:31.082 --> 00:25:36.606
Why would they be incentivized to exceed expectations and productivity?
00:25:36.606 --> 00:25:51.368
They'll still want to meet expectations because they don't want to get a raise in the annual performance review cycle, they don't want to go on a performance improvement plan, but they have no incentive to try to exceed expectations of productivity.
00:25:51.368 --> 00:26:02.262
And I actually got this feedback service to me by a specialist on the team in a skip level meeting I had and I was like, oh wow, this is really great.
00:26:02.262 --> 00:26:08.923
Because one of the minimum standards was around selecting the correct ticket fields, like ticket type, because reporting is very important, right.
00:26:08.923 --> 00:26:13.161
But like, yeah, and we we still believe that it was really important.
00:26:13.161 --> 00:26:21.815
So we actually adjusted it to put that as like a tense standard or we incorporated that into like complete back-end processes or whatever.
00:26:21.815 --> 00:26:29.423
So we took that out of being a minimum standard and then we made that one of like our regular, folded it into our regular email standards.
00:26:29.895 --> 00:26:43.231
And then also the fact that if you missed a minimum standard because not all the minimum standards were like the same weight, right, but we weren't going to weight them differently so I was like, okay, that's also a good point, because that also is something like okay, my first two way I missed a minimum standard.
00:26:43.231 --> 00:26:44.221
I was like, okay, that's also a good point, because that also is something like okay, my first QA I missed a minimum standard.
00:26:44.221 --> 00:26:48.565
I'm not going to bust my butt to exceed expectations of productivity.
00:26:48.565 --> 00:26:52.164
And I was like, totally understand that that's valid.
00:26:53.055 --> 00:27:01.364
What we did when we were using my QA there so I had a little bit more flexibility in terms of getting creative with the scorecard.
00:27:01.364 --> 00:27:09.689
So what we did is I figured out how we should weigh a minimum standard.
00:27:09.689 --> 00:27:21.741
So basically I figured out the math that a minimum standard would deduct 3% from your overall score, because that's what it would come out to, to be kind of double what a regular standard missed would.
00:27:21.741 --> 00:27:49.561
And so somebody could still miss one minimum standard on a qa, but they'd have to like really knock down to the park on everything else, but it wouldn't disqualify them and that change was like really well received, made sure we were incentivizing the right behavior and not sort of shooting ourselves in the foot um yeah so I thought that thought that to me is always like a really powerful example of me having a really good learning, of an aha moment.
00:27:49.582 --> 00:27:57.121
I'm like, oh yeah, this is best of intentions, but in practice it was actually not serving the right purpose.
00:27:57.781 --> 00:28:24.032
Yeah, and what really stands out to me about that is that you found this out in a skip level meeting, and it really highlights to me the importance of leadership, listening to frontline specialists and really taking time with intention, digging into what is your working life like, what are these well-intentioned programs that we're creating doing to your daily life, and how does that motivate or not motivate you Exactly?
00:28:24.516 --> 00:28:43.682
Yeah, I love that, in terms of cadence, if somebody is like fully trained and doing well and they meet the their internal quality score, the next month will only look at five tickets, and so it's sort of like it helps keep the the workload manageable, but it also is something that somebody could be working towards right.
00:28:44.103 --> 00:28:46.165
Yeah, so it's elastic.
00:28:46.165 --> 00:28:53.282
You grow the number of tickets as needed or shrink them as needed as people demonstrate.
00:28:53.282 --> 00:28:53.816
Yeah.
00:28:53.955 --> 00:29:20.561
And I think if we had a big fundamental change in the product or new features, and then we would potentially feel like, okay, this month everybody's going to get 10 tickets because there's been a lot of change, we made some significant shifts in process and we want to make sure that, like everybody's sort of moving forward with the new processes, and so that's a way to kind of do that.
00:29:20.694 --> 00:29:48.040
And then one other thing that I could maybe foresee in the future is, certain months, we might do sort of like a sprint, where maybe we know that our account-related tickets account modification tickets are always really, really good, so we're going to focus on, like technical issues this month, right, but you're doing that for everyone issues this month, right, but you're doing that for everyone.
00:29:48.040 --> 00:30:07.538
And the thing that you might want to do, though, is like, okay, if it's part of their performance metrics, maybe we say like, hey, let's look at how everybody did, and if the team average is way lower than maybe for this month, we adjust what that like internal quality score goal is, because it kind of goes back to being fair and equitable and transparent.
00:30:08.701 --> 00:30:13.640
And if everyone is experiencing the same thing, then something's clearly going on Exactly.
00:30:13.640 --> 00:30:21.483
Yeah, I love the emphasis on fairness and making it work for your team's workflow, as is I think that's probably why it's been successful.
00:30:21.483 --> 00:30:23.557
Thank you so much for being here.
00:30:23.557 --> 00:30:25.944
I just really appreciate you.
00:30:25.944 --> 00:30:27.638
I have adored working with you.
00:30:27.638 --> 00:30:29.964
You're one of my favorite people, so.
00:30:30.005 --> 00:30:31.856
I'm really glad, aw, thanks.
00:30:31.856 --> 00:30:36.345
That's Stacey's setup story in a nutshell.
00:30:36.345 --> 00:30:40.477
Here are the steps that you can take back to your team.
00:30:40.477 --> 00:30:42.861
One 10 interactions.
00:30:42.861 --> 00:30:44.202
Two reviews every month.
00:30:44.202 --> 00:30:45.184
Keep it lightweight.
00:30:45.184 --> 00:30:49.597
Two random recent and representative ticket picks.
00:30:49.597 --> 00:30:52.604
Three a four-pillar rubric.
00:30:52.604 --> 00:30:54.688
So keeping that lightweight.
00:30:54.688 --> 00:30:55.615
Make it simple.
00:30:55.615 --> 00:30:59.948
Hers is accuracy, completeness, customer excellence and empathy.
00:30:59.948 --> 00:31:08.188
Four private coaching for anything that maybe doesn't go perfectly, plus public kudos for what goes well.
00:31:08.188 --> 00:31:09.155
5.
00:31:09.155 --> 00:31:13.461
Shrink to 5 tickets if an agent is crushing it.
00:31:13.461 --> 00:31:16.425
Bump back up if things change.
00:31:16.425 --> 00:31:19.851
So only offering QA where it's really needed.
00:31:19.851 --> 00:31:31.827
If, as you listen to this podcast, an idea landed, or if you're using this process in the future, please share this episode with a fellow support leader and drop us a quick review and a subscribe.
00:31:31.827 --> 00:31:36.845
Until next time, keep leading with clarity and with care.
00:31:36.845 --> 00:31:37.887
See you soon.