Jonathan: [00:00:00] I think this is to the point of that I'm starting companies, just a hugely inefficient endeavor. this is just not an efficient way to, to. Yeah, right on something. It is, it's an extremely efficient way to figure out what it is you should be building.
so I think what's was much more expensive for us was, was building what we thought made sense and then finding out that it actually wasn't the right thing to build.
Erin: [00:00:37] Hi everybody, and welcome back to awkward silences. Today we're here with Jonathan Anderson. He's the CEO and cofounder of Candu that's c, a, N, D. U. They have just launched on Monday, which is very exciting after a year of daily user interviews. So we want to talk about the process that that got Jonathan and team from sort of an idea to a launched product. It's the first editor ever for your SAS app. So Jonathan, thanks so much for joining us,
Jonathan: [00:01:10] Thank you so much for having me here
Erin: [00:01:12] and we've got JH here too.
Jonathan: [00:01:14] and JH as well.
JH: [00:01:15] Yeah, yeah. I'm fascinated by the, the interview a day, so I'm curious to see how, how you pulled that off.
Jonathan: [00:01:20] yeah. So we, it actually, it didn't start that way. We didn't plan on doing an interview a day, but it ended up being by far the best way for us to gather information and then kind of adapt the product. so it just ended up being, we learned so much more on these interviews than we would otherwise.
So, I spend, have spent most of my time sourcing good interviews, running them, and then working with our designer and our engineers to. To put those ideas into practice, into our app.
JH: [00:01:44] Yeah. How literal are we talking? Like literally every day you were, you were talking to somebody or like most days.
Jonathan: [00:01:50] We probably do four or five interviews a week. they started in an hour. We shortened to 30 to 45 minutes, but yeah, no. Yeah. And we're very serious about the amount of user research we do.
JH: [00:01:59] Yeah. It's super impressive.
Erin: [00:02:01] So you said that you didn't start out with this idea of right. Cause research is a means to an end, although we do like sort of doing it as a habit. so how did you get to that place where it became this thing that you were doing that was valuable? This daily habit.
Jonathan: [00:02:17] I actually, I love this question. We, I think when you first of all, starting a company is probably the least efficient thing you can do. literally you're trying to take a, take an idea, and bring it to market. And the problem with that is that the idea is always works is wrong. You just don't know why it's wrong yet.
and so for us, it was really, The reason I started doing more and more user interviews was because, our first MVPs ended up, the ones that we had intuition about the ones that we believe were correct. the ones that we built just ended up not being, wanted by other people. So we had to flip it on its head and said, okay, what do people actually want?
Well, the best way to do that is to actually just ask them so.
Erin: [00:02:51] Great. And so what, what you, you said your initial MVPs was the, sort of. Essence of the idea of the company, the same all throughout, or where did you start and where did you end up and you know, a simple terms
Jonathan: [00:03:06] Yeah. So my background's in, in like the, in the SAS world, and especially like on the B2B side at business business, and I love software adoption. I love the idea of helping people just do better at their work by using the tools that are actually useful to them. and so we really came in with like, what's the problem we want to solve?
the, I think it's a great starting frame for starting a company, but you have to iterate through a lot of ways of solving that problem before you land on one that you actually believe in.
Erin: [00:03:32] Yeah. And so the problem that can do is solving is.
Jonathan: [00:03:36] The problem that we were solving is that, we help you change the user interface to match your user's needs. So we're really working on how do you make, B2B software usable by everybody? Our mission is to leave no customer behind, and it's really this idea that, today applications are kind of one size fits most on.
They do tons of things, lots of features. the problem with that is that there's lots of users who look at that horrible, really different needs sets. you know, user one might have nothing, nothing in common with user two. and so really what we do is we try to actually change the interface itself to match the, to match the, what the user is actually here to do.
Erin: [00:04:12] Cool.
JH: [00:04:13] Cool. and like how just practically like day to day, like would you talk to somebody on Monday, learn some things, iterate on that feedback, and then in the conversation on Tuesday. You know, it's a little different. Or like what'd you do? A couple in a row before you incorporated stuff like how, how real time and continuous was like the adjustments you're making between these conversations.
Jonathan: [00:04:34] Yeah. So we are blessed with having a really fantastic, I'd say, squad of folks who do these interviews so. it's, basically a small team, but really I can credit to, our product designer. and the way we actually work, as we would sort of show mockups, we would talk to users about those mockups.
And then split the, we can talk more about how the structure is interviews, but effectively you've moved through a series of designs, within about a week. So you do about four or five interviews on that set of mockups before sort of earrings. The next set,
JH: [00:05:02] Gotcha. Gotcha. And was it just, it was just easier for you all scheduling wise to, you know, do the hour a day basically instead of doing like, you know, one day with four hours or five hours? Like I guess I'm just geeking out on like the process here because obviously we're big fans of like continuous and frequent interviewing, but I feel like the way we often hear it from folks or teams is.
Kind of like the weekly batch of like, Tuesdays are our user, you know, day. And we'd go out and talk to people and then we process and iterate, and next Tuesday we'd do it again. so the daily thing is just for me, it's like a real trip and I'm just super curious about it.
Jonathan: [00:05:36] I think that actually has more to do with stamina than anything else. I don't know about you. I find if I do more than three interviews in a day, I'm just exhausted. and you just get less curious about what the users, the users are saying. So it's actually, it's been really nice about having. A set period.
You know, we always do ours between, like nine and 11 o'clock in the morning, where you can actually talk to the user about what they, what they're interested in. And of course it's going to end up looking along. Theater life is not a perfectly scheduled thing, especially when you're dependent on other people who are busy and are doing great work.
But, but yeah, we found that we had to cap it at three, otherwise we asked. We just, yeah, we didn't care about the answers enough. I think.
Erin: [00:06:11] I love that. I think it's such an undervalued aspect of being effective at work is, you know, playing to your strengths in the reality of, you know, does this plan actually hold up? If I actually do it, do I actually want to do a full day of interviews every week, or, you know, once a month or once a sprint or whatever it is, which might be great for some people in some teams, but there's no, you know, rule that says that's what you have to do right?
Jonathan: [00:06:38] Yeah. I'm also just not sure if I'm extroverted enough or, you know, have the, the willpower Degas go through them all. So for us it was just a lot easier to break them up. So.
JH: [00:06:47] Yeah, no, it seems really smart. I think, after, you know, the sprint methodology kind of took off and there's the, it ends with the user testing day. I think people really got into the batch of like, this is our day for testing. And, there's a lot of merits to it. I think it's a really cool idea in a lot of ways, but.
I don't think people explore it from the other side of like, well, what are the downsides of doing all these conversations in one day? so it's really interesting to hear about those kind of pieces you're bringing up of, yeah. You know, he get burned out. Like, I can't, I can't talk to five people back to back to back to back.
And then also it's a lots of process and everything comes out of that. did you find those easier to deal with the outputs of each conversation on like a smaller batch size.
Jonathan: [00:07:22] Yeah. And actually I feel like I am having trouble, a little trouble with the question because for us, so we work in a scrum methodology as well. But the, it's, I actually think of user and user reviews as an input to making progress and iterative progress. Not as a, Hey, we have this great demo. Let me show off what we have.
We do that as well. That's a, that's a great fun Friday activity where we show off what we've made. But the interview itself is actually about these iterative designs for changing day to day. so it's, it's, I think we think of it more as an input than an output.
Erin: [00:07:51] When you say day to day, I'm just back to that. So you're doing, you know, well, first of all, okay, so let's talk about the process. You said you'd tell us a little bit more about how you structure the interviews so you have some sort of mock up or a prototype or something that you're talking to people about.
tell us more about like, what's happening in those interviews.
Jonathan: [00:08:09] Yeah. So, the interviews are, are pretty, we've gotten pretty specific about them. there's a pattern. so the way it works would be someone who's going to lead the interview, and they are going to show the mockup, as well as somebody taking notes and then somebody to help coordinate and schedule.
And fill in, like basically just be like a helper for the interviewee. and we basically have one question which we always ask, which is, what would you expect? and that's really because we're, we're busy trying to be as neutral as possible in the interview process.
and then really what we're looking for is the reaction. So we split up our, the feedback that's given into. You know, happy moments or things, the questions or concerns or, you know, opportunities for new growth or even pain points. but the question itself is supposed to be very neutral. what we're really looking for is the reaction to the, interview with you has to, what we've made.
Erin: [00:08:56] So you show them the thing and then what? What do you expect? What happened? What would this do?
Jonathan: [00:09:02] Yes. Or like, Hey, where would you click first? Great. okay, if I were to click that button, what would be the next screen you would see? And then you'd click it and you'd say, and you'd hear the reaction about saying, Hey, Oh, great. Or, Oh shit, this is
Erin: [00:09:14] yeah,
Jonathan: [00:09:15] And that's how you basically suck out All of those unfiltered reactions, which I have to admit as a research subject, is not super fun because you're constantly feeling like you're being tricked. so I think there's like a, it's not fun to, to have to like uncover our biases, but that, that, that those sort of unfiltered reactions are, I think what's so powerful about running a good user interview.
Erin: [00:09:37] Yeah, absolutely.
JH: [00:09:39] who from the team was in these sessions? Is it the whole squad that you referenced, or a subset.
Jonathan: [00:09:44] Yeah, so the way it works is that we have, our, our designer usually leads them, the person who's creating the designs. And then we have a designated note taker who's usually on the customer team. and then, my cofounder and I are often, In the group as well, to listen to it. And then we actually decided to open it up to the whole team that they allowed to.
Everyone on the engineering and product side, I can listen in, a sort of a, a lurker on the line. so that's where we run them
Erin: [00:10:08] How big is your company? How many folks are there all together?
We're
Jonathan: [00:10:11] a team of eight.
Erin: [00:10:13] Cool. That's about where I got . Something like that. but you have all these departments. I'm like, wow, this can be really a lot of people. Okay. So you've got about eight people and everybody's sort of participating, you know, to a different level from time to time,
Jonathan: [00:10:25] Yeah. So like if they're working on the editor, for example, they might join those interviews. If we're focused on maybe segmentation, maybe the data side of our product engineers might come in for that. Yeah.
Erin: [00:10:36] And these are all remote interviews or in person or both?
Jonathan: [00:10:40] Yeah, we've, we've tried blended as well, so I should say that our company is remote my cofounder is in London, along with most of our engineering team. we also have folks in San Paulo, in Seville, Spain, and in San Francisco. I'm based in LA, so yes, we do a lot of remote stuff. I actually found that, Blended interviews where you have some people in the rooms and people out are really hard to run.
I've also found that, it, we really need one research subject at a time, so we have some rules that it's really helpful to have a video interview. I forgot to be all remote. Everyone do remote and, and yeah, only one person can be sort of a, as the interviewee.
JH: [00:11:15] So say you, you, you pick a part of your flow, right? And, you're. Talking to people every day about it. You're making it better, you're improving it. that's great. Right? I think I always feel in product that's hard is it never really feels like you're done. Like something always could be better. and I imagine at some point you start to hit a little bit of diminishing returns where you've kind of gotten over that 80 20 hump.
You've made a lot of the obvious improvements and the really important stuff that came out in the early conversations. How do you know when like maybe that part of the app or that workflow is good enough and you need to like pivot and start getting feedback on a different part of the experience? Like.
It just feels like once you're pulling on a thread and you keep talking to people and they keep pulling out some other small thing, you could like stay on it forever. Like how did, how did you navigate that aspect.
Jonathan: [00:11:54] Yeah. I think this is the, this is the Promise of scrum, right? You say, particularly for a new app, you have so many different aspects that have to be built. and so for us to be just have these pull up points that are, you know, every week or every two weeks, and I really credit my cofounder, who was a product manager at TransferWise and at Coursera to sort of, do the marching orders on the ship.
Because the truth is that yes, your point you, you definitely hit diminishing returns. it's incredibly obvious when UX does not work correctly. It's a little bit less obvious when UI could be improved. but I think it's, unless there's like a glaring reason why you should move on, you should really think about the milestones.
It's like, okay, we're going to tackle this V, this flow of screens within this week, and then move on to the next set.
Erin: [00:12:32] Yeah. fixed fixed time variable scope. We've been, we've been
Jonathan: [00:12:36] I'm not trying to reinvent the wheel over here. We're just trying to, you know, stay on the track, you know? So,
Erin: [00:12:41] Right. No, totally. Cool. Okay. So, back to these, these, artifact mock up things that you're showing are these all kind of the same level of fidelity? Do you like to do, are they interactive prototypes, envision balsamic mockups, PA pen and paper drawings? Like what are, what are we showing people.
Jonathan: [00:12:57] Yeah. This, madness, we've been through so many different versions of this. So I guess concretely, when we think about artifacts, we really use a lot of Figma. there's a couple of limitations with Figma's prototyping tool in that you can't have an endless amount of things to, to test, but it's very fast to create.
so it makes it very easy to throw something together. and also the rest of it together does not, is not the right language for the amount of work that goes into. Creating low fidelity design, so let's give credit where credit's due. but yeah, we found that that having something to look at is super powerful for user interview.
It can be as simple as, you know, you drawing boxes on top of a Figma file with zoom, like, that's totally fine. but yeah, having some sort of visual repetition is very useful. I will say that there's a completely separate type of user interviews, which is the not how to build this, but should we build this or what
should we build? and those are actually really, those are actually, I think we would argue much harder conversations to have. and those are mostly done verbally, with ideas not so much with, with prototyping. So.
Erin: [00:13:53] Right. And so how do you know when you're planning out your different sprints and cycles, whatever you call them, how far ahead do you know like what you're building versus like, hold the phone, we need to do some generative stuff and figure out what to do next. You have a cadence of that or do you sort of get through.
The known unknowns and then kind of reset, like how does that work in this year of getting to launch?
Jonathan: [00:14:19] I think the, the way I, I feel like this, we think about this is often, you know, you think about via feature, you know, okay, we want to figure out some way to group users. How are we going to do that? What are the right, you know, is it via file? Is it, is it via, you know, user traits? Like what's the right way to you?
The problem, the, the reality is actually much more, simple, when you're early on in a product's life cycle, you're doing a lot more of the generative stuff, and then the closer you get to launch, the more concrete you get it. So it's, it's, I think it's, well, I'd like to think that it would be, I feature the realities that it's been, very much driven by our viral launch schedule.
Yeah.
JH: [00:14:55] Cool. I feel like there's going to be a lot of people like nodding their heads and be like, this sounds amazing. I would love to talk to a user, you know, every day or most days. But you're, you're all in, you know, an early team and like sourcing all these people and scheduling and planning for the calls and processing what you learned for inputs into the next work.
Like there's a lot of costs there, both like in time and, you know, in dollars in, in other, aspects of it. Like, how did you all decide to make this a priority? It was just something from the start that was like, this is important and we're doing it. Or, was it like a thoughtful choice? Or like how did it actually become like, you know, get sign off in that sense.
Jonathan: [00:16:08] Well, so I guess I, I sign off on my own time so that, that makes us lot of processes easier. But, yeah, I mean, this is, I think this is to the point of that I'm starting companies, just a hugely inefficient endeavor. this is just not an efficient way to, to. Yeah, right on something. It is, it's an extremely efficient way to figure out what it is you should be building.
so I think what's was much more expensive for us was, was building what we thought made sense and then finding out that it actually wasn't the right thing to build. Software has gotten so cheap, but it's still expensive for a team, especially with the opportunity cost of what you could be building.
So. it was a super deliberate choice after the third MVP built, turned out not to be that useful. I feel like that, and I think the, the key thing was that we built out a, I think a lot of startups do this, but we built, had a design partner program, which was basically a series of folks who gave really good feedback, who are currently working on the problem that we're trying to solve, who wanted to be part of this product.
Yeah, this new product, process. And so for them, it was really an opportunity to sort of come in and join an early stage team. And they really became full time partners for us. And so the way the actual scheduling would work was that we would talk to them about once a month. and so we had a group of, you know, 10 or so companies who we had an on retainer basically.
And then we complemented those with, with new folks throughout.
JH: [00:17:26] Gotcha. Gotcha. Okay, cool. Yeah, the, the partner piece seems, really helpful in terms of how to do something like this because you at least have a group that you can go back to you consistently. And you mentioned keeping them on retainer. Like was there an actual, like financial relationship with those people or those companies or were they just so invested in this problem space that they were happy to help?
Jonathan: [00:17:42] Yeah, no, I think, no, there wasn't a financial relationship. This was purely a, you know, please come help us. We need your help. but I think to your, to that point, like I think there's something really, really, really fun. I'm seeing a product evolve over time and seeing how your input, even if it's interactive, is causing these sort of changes.
I think people got very invested in sort of the outcome. And as we moved toward launch.
JH: [00:18:05] Cool. It seems like a good market signal too, right? Of like, this problem for me is so cute. I'm willing to spend some my time to help with this potential solution to it. That that feels like a meaningful signal that like you're, you're solving something that really needs to be solved.
Jonathan: [00:18:18] Oh, I feel like you're, I wish, I, you, I wish I have to agree with you, but I have to actually strongly disagree. I, so I think that this is actually the. Biggest problem that we face, which is that people want to be helpful. they love, like, you know, they, they want to be agreeable. They want to, they want to help out folks they think are smart and interesting and fun to talk to.
And so they tend to be like, Oh yeah, I would be, you know, happy to sort of join this endeavor. trying to figure out what would actually be value to that valuable to them in their jobs. That took a lot longer, I think getting someone to agree to an initial interview and then agree to the next interview.
so I think that. That was actually a big insight for us was, was how do you get past the, Hey, let me be as helpful as possible to, Oh, yeah, we, I, this is what I need. That's a, that's a big, big jump.
Erin: [00:19:06] And also too, what are you willing to pay for? Right, the
Jonathan: [00:19:09] Yeah, exactly. Yeah,
JH: [00:19:11] yeah. yeah. The willingness to pay. I get, I get off my track, I guess. Just, you know, this is my own bias, right? But like, I couldn't spend time on something if I didn't care about it. Like I would want to be helpful, but I'd also be like, Hey, like I'm busy. I'm not, you know? So, that's
Jonathan: [00:19:22] Yeah. Very
JH: [00:19:22] That's an interesting part.
Erin: [00:19:25] So, just to back up to repeat what you said, so basically you found these like 10 or so companies who are these ongoing partners in terms of delivering interviewees to you?
Jonathan: [00:19:36] actually, no, sorry. The, just to quickly clarify, it was
the, uh, the way that design partner program worked is that we would find a company who was, who specifically was in the space that we're looking for. but then it really came down to finding the person, who had, specifically, in our case, worked on activation or user onboarding and then actually working with them directly.
so it was really a personal commitment, not a company commitment. And that's, that was super valuable for us. And then also, cause it's not like I, the whole company knows what we're working on. It's, it's much easier if you know, specific, if Sarah knows exactly what we worked on one month ago and then can sort of reflect on the changes to the product and then give more candid feedback.
Erin: [00:20:12] Got it. So you sort of build up your own panel that happened to be comprised of representatives from 10 or so companies and would talk to each of those people like once a month or so. Okay, cool. Yeah, that's interesting. How did you get at did once a month feel like, I dunno. we, you know, we talked to people in terms of how often is too often to hit up the same person in terms of annoying them or in terms of they get too familiar with what you're building and then they're, you know, feedback isn't as valid versus the benefits of what you're talking about there is that longitudinal and being invested.
How did you kind of come up with the right amount of. You know, time to talk to each of these people,
Jonathan: [00:20:55] Yeah. I think there's also just the more fundamental problem, like is it a busy time for them?
Are they, you know, are they on maternity leave or are they is it just like, you know, picks up a new project and are running with it so. Um, you know, over the course of this year, we've had people switch jobs.
We've had people, you know, have kids. We've had people, you know, come in and out for all kinds of reasons. so I think for us it was more around finding out who gives good feedback, because in theory, I think everyone should be able to, but it really takes someone who's like really engaged with a problem, to give the best feedback and then trying to sort of fit them into this, framework and beg and plead for their, for their insights.
So.
JH: [00:21:34] If you know for, I'm an early stage team, there's a couple of us we are know, just trying to figure out this idea. Do you have like what would like your playbook be if, if I wanted to get this thing set up and do it ourselves, like what would be like the best lessons learned and recommendations you'd have from doing it yourself?
Jonathan: [00:21:50] Yeah. I guess it's only through experience of doing it wrong. I really didn't know how to do a user interview. I would get too excited about what I was talking about, which is a super important quality in anyone starting any kind of thing. because you need to inspire people. You need to get people excited about this.
Vision, you're painting for the world, that is completely at odds with building something that's actually valuable. so, you kind of need both that optimism. but also the, the realism of. I'm okay actually. Okay. If you would use it, why would you use it? what, okay. How long would it take you to use it?
Why wouldn't you be able to use it? I think, for me it was, it was really working with my cofounder B Kelly who had done product before. That was very helpful. And then also just, you know, things like reading the mom test a great book about, how to get past the, the bias to agree.
JH: [00:22:38] Hmm. Yeah, I've heard that before. That's really hard for founders to turn off the selling instincts cause they're always just in the mode of, you know, selling the vision and trying to be, to your point, like inspirational. so it's, it's a good reminder that in this context, that's actually not the right mode to be in.
Jonathan: [00:22:52] Yeah. And I've sold, I've sold in so many situations that are just inappropriate to sell in. which is, which is the, like, which is the antithesis. Let me think of, of, of good user research stuff.
Erin: [00:23:03] Yeah. Gotta replace. Always be closing with, always be learning or something like that.
Jonathan: [00:23:08] Yeah, and that is a model that we, that we live by. Absolutely.
Erin: [00:23:11] Yeah. I'm curious. You talked about, you know, kind of getting better at figuring out this ideal profile of your participants in terms of people who are really engaged in the problem space. I know that's something that our listeners. are thinking about when they're trying to find, especially in B2B, right?
participants that are going to be good and what does that look like and how do you screen for that? Did you learn anything over the months of being able to sort of predict better over time? Who is going to end up being a better participant? Are there any like sort of intangibles or things you wouldn't have thought of to look for when you first got started?
Jonathan: [00:23:53] I mean, I don't know if this is a secret not, but by far, the easiest way to know that is to do a user interview and look at the, how long the notes are. it just, just quantity is actually quite good. in terms of what we found, even more so than quality. I will say that one thing that we do look for is.
Someone who's used tools to solve the problem is actually quite helpful, particularly if they've used a set of them already. and then it, particularly if they've been in sort of the same type of role at multiple companies is also quite helpful. you're really looking for the person who's like, worn many hats.
It's very helpful. We found.
Erin: [00:24:26] Right. And is that because people who have sort of SAS geekery you know, in their blood are going to be better customers for you? Or are there better at talking about software or why was that important?
Jonathan: [00:24:41] Yeah, I think, and it's not specific to, they have to have worked in SAS. It's more that they've had, that they've tried to solve this problem in different environments, I think is the more important problem to think, to look for because then they can relate ideas. and having, I think some day to day experience, like, okay, not how would I solve this, but how have I solved
this is what makes the biggest difference.
Okay.
Erin: [00:25:02] Great. When you were at, you said you didn't know how to do a user interview in the beginning, and you're kind of always, you know, selling and too enthusiastic. Did you, did you have to learn the, you know, don't ask hypotheticals or what other, what other mistakes did you sort of learn by doing
Jonathan: [00:25:20] I have made every mistake possible. I mean, even like little things like, when someone wants to click a button, immediately clicking the button as opposed to asking, okay, what would be the next screen that would come up. It's a really un-intuitive question. It's so obvious. You just click the click the goddamn button, sorry. But the a, that is, it's not how it works. that's not, that's not what you're looking for. yeah. My co founder had a lot of hard conversations with me around, chilling out and then actually
observing him later views, being the note taker for, for a lot of the insurance before I was comfortable
it myself.
Erin: [00:25:48] Was he chill?
Jonathan: [00:25:50] yes. Oh, yeah. Very neutral. Extremely neutral. Yeah. Yeah.
JH: [00:25:53] Yeah. This is, this conversation is really running me like a year, year and a half ago when we were building a new workflow and it's kind of solution in our product for scheduling stuff with your own user base. we kind of did a very similar thing of, we were calling them like design partners and found people who were like the right profile and we'd go back to that same group at some frequency and it was super helpful.
And all the stuff you're saying is just like bringing back so many memories of. how we found those people. And one of the best things I think we did was just asking on, like, the initial kind of survey we sent around was like, how do you solve this problem currently? And you could just, you could learn so much from like the people who wrote like a paragraph or two about like, we use Google sheets, but then we try to air table and then like that, dah, dah, dah, dah.
You're like, I want to talk to that person. And then the person would be like, I don't know. Like I just find people in my base are like, all right, you, you might not have thought about this as much or if solid as well. And, Yeah. So that advice just is really a ringing true to me. it was super helpful to find people that way.
Jonathan: [00:26:46] Yeah. One of my favorite actual, this is actually a sales question, not a user interview question, but I find them are both quite helpful is what is the process for doing something, because when someone has a PR, when someone's thought enough to actually create a process, first of all, processes are painful.
Uh, things. Are easy. They don't require a process. but it, it has a little bit of, implies that there's some friction along the path and that they've had to have thought through what is the right way to solve this problem because I've seen it with some regularity. so that actually is a pretty good, it's actually a great sales question and a good user interview question.
But, just one of the very, very small Venn diagram questions that offended both camps.
JH: [00:27:17] Yeah. And now that you, uh, you mentioned launching on Monday, is the plan to just keep this up and definitely. Or is the cadence going to shift a little bit now that there's a product out in the wild? Like what? What's next for you all?
Jonathan: [00:27:28] Yeah, you good. You have to do certain things like maintain that product. You notice, uh, this last phase of, of really building for mockups has been super, super fun. I think the team is actually ready to kind of see the thing live now. I think there's like a different, a different world there.
And then also like, fundamentally, the, you know, you start getting actual data, right? People are actually using the product, and, or can't use the product. so that kind of changes the way the interviews are done, pretty significantly. So we're excited for the next phase of, of research.
Erin: [00:27:56] Yeah. Do you have a plan for how that's going to work once you go live and you have, I don't know, your product analytics, cooking and seeing what, what's happening? Do you anticipate keeping daily interviews or, seeing what happens or.
Jonathan: [00:28:11] Yeah. We're, we're still in a phase where we're still, we're still going through the daily interviews. I think we'll probably can track them down. I used to have more input sources for, for how to build things, right? when you have users using the product, you can look what they can create or where they spend time.
you know, we use FullStory, we use amplitude to sort of gather that information. And I'm excited just to compliment the. They use your interview with some of the, Muslim, the actual data on usage. So
Erin: [00:28:33] Yeah. All right. So thinking back memories on the last year, are there any moments that jump out where. you know, a user interviewer, a handful of them, really changed your trajectory or where you really, you know, had to where that humility had and let go of one of your babies or, or anything that just stands out as being like, wow, you know, this would be a, a different and worse product without, without having that insight.
Jonathan: [00:29:04] Yeah. So there was one really big fundamental shift that we made as a company. originally we basically created a, an overlay product, a little bit like a walkthrough guide, like a Pendo or an accuser or walking. it turns out those are really good solutions, uh, that already exist in the world.
and we don't really need another one of them. And I think we've had a number of interviews where you would ask someone about a problem and you have this idea for, Oh, we can solve it this way. And they would say, Oh, let me show you how I'm solving it. and I think that was, great for the user and for the user research for these interviews.
They're like, Oh, great, I have a great way of solving this problem, but, when something is good enough already, I think that's what's very important for us to kind of, figure out how to disentangle. That doesn't tell you necessarily how to do it. what the right, you know, innovative solution is that at least tells you that, this maybe isn't a problem that's on fire.
so we, we had a bunch of those conversations, so
Yeah,
JH: [00:29:56] Yeah, totally. I know there's a, there's a framework for that type of stuff. I forget what it is, but, just that notion of, you know, it's really important to ask, like, you know, how acute is this pain point? And someone might say like, you know, a scale of one to 10, a nine or a 10. But if you don't follow it up with how well is that solved for you today?
And they say like, you
Jonathan: [00:30:10] Right.
JH: [00:30:11] and you're like, well, there's not a lot opportunity there. whereas if they say something's only like a six pain point, but the current solution for them is a zero, like, that's actually a
pretty wide, you know, margin to attack.
Jonathan: [00:30:20] Yeah, exactly. Yeah. And I think, again, this is like a really big difference from sales because in sales we really trying to do is highlight a pain point and then sort of explain the consequences of not fixing that. So for example, it's not that you. somebody takes a long time. Right? That's actually not a pain point.
It's a pain point because you could have spent that time doing something else that is valuable. That's what makes it painful. and so in sales, you're trying to like uncover these pain points and then exacerbate them or kind of give them a sense of their scale and use the research. You're actually doing that as well.
but you just need to be a lot more. It's not, you're trying to convince the user interview and it's painful. It's right that you're trying to convince yourself as painful.
Um.
JH: [00:30:58] That's good frame.
Erin: [00:30:59] I like that.
so I think what's was much more expensive for us was, was building what we thought made sense and then finding out that it actually wasn't the right thing to build.
Erin: [00:00:37] Hi everybody, and welcome back to awkward silences. Today we're here with Jonathan Anderson. He's the CEO and cofounder of Candu that's c, a, N, D. U. They have just launched on Monday, which is very exciting after a year of daily user interviews. So we want to talk about the process that that got Jonathan and team from sort of an idea to a launched product. It's the first editor ever for your SAS app. So Jonathan, thanks so much for joining us,
Jonathan: [00:01:10] Thank you so much for having me here
Erin: [00:01:12] and we've got JH here too.
Jonathan: [00:01:14] and JH as well.
JH: [00:01:15] Yeah, yeah. I'm fascinated by the, the interview a day, so I'm curious to see how, how you pulled that off.
Jonathan: [00:01:20] yeah. So we, it actually, it didn't start that way. We didn't plan on doing an interview a day, but it ended up being by far the best way for us to gather information and then kind of adapt the product. so it just ended up being, we learned so much more on these interviews than we would otherwise.
So, I spend, have spent most of my time sourcing good interviews, running them, and then working with our designer and our engineers to. To put those ideas into practice, into our app.
JH: [00:01:44] Yeah. How literal are we talking? Like literally every day you were, you were talking to somebody or like most days.
Jonathan: [00:01:50] We probably do four or five interviews a week. they started in an hour. We shortened to 30 to 45 minutes, but yeah, no. Yeah. And we're very serious about the amount of user research we do.
JH: [00:01:59] Yeah. It's super impressive.
Erin: [00:02:01] So you said that you didn't start out with this idea of right. Cause research is a means to an end, although we do like sort of doing it as a habit. so how did you get to that place where it became this thing that you were doing that was valuable? This daily habit.
Jonathan: [00:02:17] I actually, I love this question. We, I think when you first of all, starting a company is probably the least efficient thing you can do. literally you're trying to take a, take an idea, and bring it to market. And the problem with that is that the idea is always works is wrong. You just don't know why it's wrong yet.
and so for us, it was really, The reason I started doing more and more user interviews was because, our first MVPs ended up, the ones that we had intuition about the ones that we believe were correct. the ones that we built just ended up not being, wanted by other people. So we had to flip it on its head and said, okay, what do people actually want?
Well, the best way to do that is to actually just ask them so.
Erin: [00:02:51] Great. And so what, what you, you said your initial MVPs was the, sort of. Essence of the idea of the company, the same all throughout, or where did you start and where did you end up and you know, a simple terms
Jonathan: [00:03:06] Yeah. So my background's in, in like the, in the SAS world, and especially like on the B2B side at business business, and I love software adoption. I love the idea of helping people just do better at their work by using the tools that are actually useful to them. and so we really came in with like, what's the problem we want to solve?
the, I think it's a great starting frame for starting a company, but you have to iterate through a lot of ways of solving that problem before you land on one that you actually believe in.
Erin: [00:03:32] Yeah. And so the problem that can do is solving is.
Jonathan: [00:03:36] The problem that we were solving is that, we help you change the user interface to match your user's needs. So we're really working on how do you make, B2B software usable by everybody? Our mission is to leave no customer behind, and it's really this idea that, today applications are kind of one size fits most on.
They do tons of things, lots of features. the problem with that is that there's lots of users who look at that horrible, really different needs sets. you know, user one might have nothing, nothing in common with user two. and so really what we do is we try to actually change the interface itself to match the, to match the, what the user is actually here to do.
Erin: [00:04:12] Cool.
JH: [00:04:13] Cool. and like how just practically like day to day, like would you talk to somebody on Monday, learn some things, iterate on that feedback, and then in the conversation on Tuesday. You know, it's a little different. Or like what'd you do? A couple in a row before you incorporated stuff like how, how real time and continuous was like the adjustments you're making between these conversations.
Jonathan: [00:04:34] Yeah. So we are blessed with having a really fantastic, I'd say, squad of folks who do these interviews so. it's, basically a small team, but really I can credit to, our product designer. and the way we actually work, as we would sort of show mockups, we would talk to users about those mockups.
And then split the, we can talk more about how the structure is interviews, but effectively you've moved through a series of designs, within about a week. So you do about four or five interviews on that set of mockups before sort of earrings. The next set,
JH: [00:05:02] Gotcha. Gotcha. And was it just, it was just easier for you all scheduling wise to, you know, do the hour a day basically instead of doing like, you know, one day with four hours or five hours? Like I guess I'm just geeking out on like the process here because obviously we're big fans of like continuous and frequent interviewing, but I feel like the way we often hear it from folks or teams is.
Kind of like the weekly batch of like, Tuesdays are our user, you know, day. And we'd go out and talk to people and then we process and iterate, and next Tuesday we'd do it again. so the daily thing is just for me, it's like a real trip and I'm just super curious about it.
Jonathan: [00:05:36] I think that actually has more to do with stamina than anything else. I don't know about you. I find if I do more than three interviews in a day, I'm just exhausted. and you just get less curious about what the users, the users are saying. So it's actually, it's been really nice about having. A set period.
You know, we always do ours between, like nine and 11 o'clock in the morning, where you can actually talk to the user about what they, what they're interested in. And of course it's going to end up looking along. Theater life is not a perfectly scheduled thing, especially when you're dependent on other people who are busy and are doing great work.
But, but yeah, we found that we had to cap it at three, otherwise we asked. We just, yeah, we didn't care about the answers enough. I think.
Erin: [00:06:11] I love that. I think it's such an undervalued aspect of being effective at work is, you know, playing to your strengths in the reality of, you know, does this plan actually hold up? If I actually do it, do I actually want to do a full day of interviews every week, or, you know, once a month or once a sprint or whatever it is, which might be great for some people in some teams, but there's no, you know, rule that says that's what you have to do right?
Jonathan: [00:06:38] Yeah. I'm also just not sure if I'm extroverted enough or, you know, have the, the willpower Degas go through them all. So for us it was just a lot easier to break them up. So.
JH: [00:06:47] Yeah, no, it seems really smart. I think, after, you know, the sprint methodology kind of took off and there's the, it ends with the user testing day. I think people really got into the batch of like, this is our day for testing. And, there's a lot of merits to it. I think it's a really cool idea in a lot of ways, but.
I don't think people explore it from the other side of like, well, what are the downsides of doing all these conversations in one day? so it's really interesting to hear about those kind of pieces you're bringing up of, yeah. You know, he get burned out. Like, I can't, I can't talk to five people back to back to back to back.
And then also it's a lots of process and everything comes out of that. did you find those easier to deal with the outputs of each conversation on like a smaller batch size.
Jonathan: [00:07:22] Yeah. And actually I feel like I am having trouble, a little trouble with the question because for us, so we work in a scrum methodology as well. But the, it's, I actually think of user and user reviews as an input to making progress and iterative progress. Not as a, Hey, we have this great demo. Let me show off what we have.
We do that as well. That's a, that's a great fun Friday activity where we show off what we've made. But the interview itself is actually about these iterative designs for changing day to day. so it's, it's, I think we think of it more as an input than an output.
Erin: [00:07:51] When you say day to day, I'm just back to that. So you're doing, you know, well, first of all, okay, so let's talk about the process. You said you'd tell us a little bit more about how you structure the interviews so you have some sort of mock up or a prototype or something that you're talking to people about.
tell us more about like, what's happening in those interviews.
Jonathan: [00:08:09] Yeah. So, the interviews are, are pretty, we've gotten pretty specific about them. there's a pattern. so the way it works would be someone who's going to lead the interview, and they are going to show the mockup, as well as somebody taking notes and then somebody to help coordinate and schedule.
And fill in, like basically just be like a helper for the interviewee. and we basically have one question which we always ask, which is, what would you expect? and that's really because we're, we're busy trying to be as neutral as possible in the interview process.
and then really what we're looking for is the reaction. So we split up our, the feedback that's given into. You know, happy moments or things, the questions or concerns or, you know, opportunities for new growth or even pain points. but the question itself is supposed to be very neutral. what we're really looking for is the reaction to the, interview with you has to, what we've made.
Erin: [00:08:56] So you show them the thing and then what? What do you expect? What happened? What would this do?
Jonathan: [00:09:02] Yes. Or like, Hey, where would you click first? Great. okay, if I were to click that button, what would be the next screen you would see? And then you'd click it and you'd say, and you'd hear the reaction about saying, Hey, Oh, great. Or, Oh shit, this is
Erin: [00:09:14] yeah,
Jonathan: [00:09:15] And that's how you basically suck out All of those unfiltered reactions, which I have to admit as a research subject, is not super fun because you're constantly feeling like you're being tricked. so I think there's like a, it's not fun to, to have to like uncover our biases, but that, that, that those sort of unfiltered reactions are, I think what's so powerful about running a good user interview.
Erin: [00:09:37] Yeah, absolutely.
JH: [00:09:39] who from the team was in these sessions? Is it the whole squad that you referenced, or a subset.
Jonathan: [00:09:44] Yeah, so the way it works is that we have, our, our designer usually leads them, the person who's creating the designs. And then we have a designated note taker who's usually on the customer team. and then, my cofounder and I are often, In the group as well, to listen to it. And then we actually decided to open it up to the whole team that they allowed to.
Everyone on the engineering and product side, I can listen in, a sort of a, a lurker on the line. so that's where we run them
Erin: [00:10:08] How big is your company? How many folks are there all together?
We're
Jonathan: [00:10:11] a team of eight.
Erin: [00:10:13] Cool. That's about where I got . Something like that. but you have all these departments. I'm like, wow, this can be really a lot of people. Okay. So you've got about eight people and everybody's sort of participating, you know, to a different level from time to time,
Jonathan: [00:10:25] Yeah. So like if they're working on the editor, for example, they might join those interviews. If we're focused on maybe segmentation, maybe the data side of our product engineers might come in for that. Yeah.
Erin: [00:10:36] And these are all remote interviews or in person or both?
Jonathan: [00:10:40] Yeah, we've, we've tried blended as well, so I should say that our company is remote my cofounder is in London, along with most of our engineering team. we also have folks in San Paulo, in Seville, Spain, and in San Francisco. I'm based in LA, so yes, we do a lot of remote stuff. I actually found that, Blended interviews where you have some people in the rooms and people out are really hard to run.
I've also found that, it, we really need one research subject at a time, so we have some rules that it's really helpful to have a video interview. I forgot to be all remote. Everyone do remote and, and yeah, only one person can be sort of a, as the interviewee.
JH: [00:11:15] So say you, you, you pick a part of your flow, right? And, you're. Talking to people every day about it. You're making it better, you're improving it. that's great. Right? I think I always feel in product that's hard is it never really feels like you're done. Like something always could be better. and I imagine at some point you start to hit a little bit of diminishing returns where you've kind of gotten over that 80 20 hump.
You've made a lot of the obvious improvements and the really important stuff that came out in the early conversations. How do you know when like maybe that part of the app or that workflow is good enough and you need to like pivot and start getting feedback on a different part of the experience? Like.
It just feels like once you're pulling on a thread and you keep talking to people and they keep pulling out some other small thing, you could like stay on it forever. Like how did, how did you navigate that aspect.
Jonathan: [00:11:54] Yeah. I think this is the, this is the Promise of scrum, right? You say, particularly for a new app, you have so many different aspects that have to be built. and so for us to be just have these pull up points that are, you know, every week or every two weeks, and I really credit my cofounder, who was a product manager at TransferWise and at Coursera to sort of, do the marching orders on the ship.
Because the truth is that yes, your point you, you definitely hit diminishing returns. it's incredibly obvious when UX does not work correctly. It's a little bit less obvious when UI could be improved. but I think it's, unless there's like a glaring reason why you should move on, you should really think about the milestones.
It's like, okay, we're going to tackle this V, this flow of screens within this week, and then move on to the next set.
Erin: [00:12:32] Yeah. fixed fixed time variable scope. We've been, we've been
Jonathan: [00:12:36] I'm not trying to reinvent the wheel over here. We're just trying to, you know, stay on the track, you know? So,
Erin: [00:12:41] Right. No, totally. Cool. Okay. So, back to these, these, artifact mock up things that you're showing are these all kind of the same level of fidelity? Do you like to do, are they interactive prototypes, envision balsamic mockups, PA pen and paper drawings? Like what are, what are we showing people.
Jonathan: [00:12:57] Yeah. This, madness, we've been through so many different versions of this. So I guess concretely, when we think about artifacts, we really use a lot of Figma. there's a couple of limitations with Figma's prototyping tool in that you can't have an endless amount of things to, to test, but it's very fast to create.
so it makes it very easy to throw something together. and also the rest of it together does not, is not the right language for the amount of work that goes into. Creating low fidelity design, so let's give credit where credit's due. but yeah, we found that that having something to look at is super powerful for user interview.
It can be as simple as, you know, you drawing boxes on top of a Figma file with zoom, like, that's totally fine. but yeah, having some sort of visual repetition is very useful. I will say that there's a completely separate type of user interviews, which is the not how to build this, but should we build this or what
should we build? and those are actually really, those are actually, I think we would argue much harder conversations to have. and those are mostly done verbally, with ideas not so much with, with prototyping. So.
Erin: [00:13:53] Right. And so how do you know when you're planning out your different sprints and cycles, whatever you call them, how far ahead do you know like what you're building versus like, hold the phone, we need to do some generative stuff and figure out what to do next. You have a cadence of that or do you sort of get through.
The known unknowns and then kind of reset, like how does that work in this year of getting to launch?
Jonathan: [00:14:19] I think the, the way I, I feel like this, we think about this is often, you know, you think about via feature, you know, okay, we want to figure out some way to group users. How are we going to do that? What are the right, you know, is it via file? Is it, is it via, you know, user traits? Like what's the right way to you?
The problem, the, the reality is actually much more, simple, when you're early on in a product's life cycle, you're doing a lot more of the generative stuff, and then the closer you get to launch, the more concrete you get it. So it's, it's, I think it's, well, I'd like to think that it would be, I feature the realities that it's been, very much driven by our viral launch schedule.
Yeah.
JH: [00:14:55] Cool. I feel like there's going to be a lot of people like nodding their heads and be like, this sounds amazing. I would love to talk to a user, you know, every day or most days. But you're, you're all in, you know, an early team and like sourcing all these people and scheduling and planning for the calls and processing what you learned for inputs into the next work.
Like there's a lot of costs there, both like in time and, you know, in dollars in, in other, aspects of it. Like, how did you all decide to make this a priority? It was just something from the start that was like, this is important and we're doing it. Or, was it like a thoughtful choice? Or like how did it actually become like, you know, get sign off in that sense.
Jonathan: [00:16:08] Well, so I guess I, I sign off on my own time so that, that makes us lot of processes easier. But, yeah, I mean, this is, I think this is to the point of that I'm starting companies, just a hugely inefficient endeavor. this is just not an efficient way to, to. Yeah, right on something. It is, it's an extremely efficient way to figure out what it is you should be building.
so I think what's was much more expensive for us was, was building what we thought made sense and then finding out that it actually wasn't the right thing to build. Software has gotten so cheap, but it's still expensive for a team, especially with the opportunity cost of what you could be building.
So. it was a super deliberate choice after the third MVP built, turned out not to be that useful. I feel like that, and I think the, the key thing was that we built out a, I think a lot of startups do this, but we built, had a design partner program, which was basically a series of folks who gave really good feedback, who are currently working on the problem that we're trying to solve, who wanted to be part of this product.
Yeah, this new product, process. And so for them, it was really an opportunity to sort of come in and join an early stage team. And they really became full time partners for us. And so the way the actual scheduling would work was that we would talk to them about once a month. and so we had a group of, you know, 10 or so companies who we had an on retainer basically.
And then we complemented those with, with new folks throughout.
JH: [00:17:26] Gotcha. Gotcha. Okay, cool. Yeah, the, the partner piece seems, really helpful in terms of how to do something like this because you at least have a group that you can go back to you consistently. And you mentioned keeping them on retainer. Like was there an actual, like financial relationship with those people or those companies or were they just so invested in this problem space that they were happy to help?
Jonathan: [00:17:42] Yeah, no, I think, no, there wasn't a financial relationship. This was purely a, you know, please come help us. We need your help. but I think to your, to that point, like I think there's something really, really, really fun. I'm seeing a product evolve over time and seeing how your input, even if it's interactive, is causing these sort of changes.
I think people got very invested in sort of the outcome. And as we moved toward launch.
JH: [00:18:05] Cool. It seems like a good market signal too, right? Of like, this problem for me is so cute. I'm willing to spend some my time to help with this potential solution to it. That that feels like a meaningful signal that like you're, you're solving something that really needs to be solved.
Jonathan: [00:18:18] Oh, I feel like you're, I wish, I, you, I wish I have to agree with you, but I have to actually strongly disagree. I, so I think that this is actually the. Biggest problem that we face, which is that people want to be helpful. they love, like, you know, they, they want to be agreeable. They want to, they want to help out folks they think are smart and interesting and fun to talk to.
And so they tend to be like, Oh yeah, I would be, you know, happy to sort of join this endeavor. trying to figure out what would actually be value to that valuable to them in their jobs. That took a lot longer, I think getting someone to agree to an initial interview and then agree to the next interview.
so I think that. That was actually a big insight for us was, was how do you get past the, Hey, let me be as helpful as possible to, Oh, yeah, we, I, this is what I need. That's a, that's a big, big jump.
Erin: [00:19:06] And also too, what are you willing to pay for? Right, the
Jonathan: [00:19:09] Yeah, exactly. Yeah,
JH: [00:19:11] yeah. yeah. The willingness to pay. I get, I get off my track, I guess. Just, you know, this is my own bias, right? But like, I couldn't spend time on something if I didn't care about it. Like I would want to be helpful, but I'd also be like, Hey, like I'm busy. I'm not, you know? So, that's
Jonathan: [00:19:22] Yeah. Very
JH: [00:19:22] That's an interesting part.
Erin: [00:19:25] So, just to back up to repeat what you said, so basically you found these like 10 or so companies who are these ongoing partners in terms of delivering interviewees to you?
Jonathan: [00:19:36] actually, no, sorry. The, just to quickly clarify, it was
the, uh, the way that design partner program worked is that we would find a company who was, who specifically was in the space that we're looking for. but then it really came down to finding the person, who had, specifically, in our case, worked on activation or user onboarding and then actually working with them directly.
so it was really a personal commitment, not a company commitment. And that's, that was super valuable for us. And then also, cause it's not like I, the whole company knows what we're working on. It's, it's much easier if you know, specific, if Sarah knows exactly what we worked on one month ago and then can sort of reflect on the changes to the product and then give more candid feedback.
Erin: [00:20:12] Got it. So you sort of build up your own panel that happened to be comprised of representatives from 10 or so companies and would talk to each of those people like once a month or so. Okay, cool. Yeah, that's interesting. How did you get at did once a month feel like, I dunno. we, you know, we talked to people in terms of how often is too often to hit up the same person in terms of annoying them or in terms of they get too familiar with what you're building and then they're, you know, feedback isn't as valid versus the benefits of what you're talking about there is that longitudinal and being invested.
How did you kind of come up with the right amount of. You know, time to talk to each of these people,
Jonathan: [00:20:55] Yeah. I think there's also just the more fundamental problem, like is it a busy time for them?
Are they, you know, are they on maternity leave or are they is it just like, you know, picks up a new project and are running with it so. Um, you know, over the course of this year, we've had people switch jobs.
We've had people, you know, have kids. We've had people, you know, come in and out for all kinds of reasons. so I think for us it was more around finding out who gives good feedback, because in theory, I think everyone should be able to, but it really takes someone who's like really engaged with a problem, to give the best feedback and then trying to sort of fit them into this, framework and beg and plead for their, for their insights.
So.
JH: [00:21:34] If you know for, I'm an early stage team, there's a couple of us we are know, just trying to figure out this idea. Do you have like what would like your playbook be if, if I wanted to get this thing set up and do it ourselves, like what would be like the best lessons learned and recommendations you'd have from doing it yourself?
Jonathan: [00:21:50] Yeah. I guess it's only through experience of doing it wrong. I really didn't know how to do a user interview. I would get too excited about what I was talking about, which is a super important quality in anyone starting any kind of thing. because you need to inspire people. You need to get people excited about this.
Vision, you're painting for the world, that is completely at odds with building something that's actually valuable. so, you kind of need both that optimism. but also the, the realism of. I'm okay actually. Okay. If you would use it, why would you use it? what, okay. How long would it take you to use it?
Why wouldn't you be able to use it? I think, for me it was, it was really working with my cofounder B Kelly who had done product before. That was very helpful. And then also just, you know, things like reading the mom test a great book about, how to get past the, the bias to agree.
JH: [00:22:38] Hmm. Yeah, I've heard that before. That's really hard for founders to turn off the selling instincts cause they're always just in the mode of, you know, selling the vision and trying to be, to your point, like inspirational. so it's, it's a good reminder that in this context, that's actually not the right mode to be in.
Jonathan: [00:22:52] Yeah. And I've sold, I've sold in so many situations that are just inappropriate to sell in. which is, which is the, like, which is the antithesis. Let me think of, of, of good user research stuff.
Erin: [00:23:03] Yeah. Gotta replace. Always be closing with, always be learning or something like that.
Jonathan: [00:23:08] Yeah, and that is a model that we, that we live by. Absolutely.
Erin: [00:23:11] Yeah. I'm curious. You talked about, you know, kind of getting better at figuring out this ideal profile of your participants in terms of people who are really engaged in the problem space. I know that's something that our listeners. are thinking about when they're trying to find, especially in B2B, right?
participants that are going to be good and what does that look like and how do you screen for that? Did you learn anything over the months of being able to sort of predict better over time? Who is going to end up being a better participant? Are there any like sort of intangibles or things you wouldn't have thought of to look for when you first got started?
Jonathan: [00:23:53] I mean, I don't know if this is a secret not, but by far, the easiest way to know that is to do a user interview and look at the, how long the notes are. it just, just quantity is actually quite good. in terms of what we found, even more so than quality. I will say that one thing that we do look for is.
Someone who's used tools to solve the problem is actually quite helpful, particularly if they've used a set of them already. and then it, particularly if they've been in sort of the same type of role at multiple companies is also quite helpful. you're really looking for the person who's like, worn many hats.
It's very helpful. We found.
Erin: [00:24:26] Right. And is that because people who have sort of SAS geekery you know, in their blood are going to be better customers for you? Or are there better at talking about software or why was that important?
Jonathan: [00:24:41] Yeah, I think, and it's not specific to, they have to have worked in SAS. It's more that they've had, that they've tried to solve this problem in different environments, I think is the more important problem to think, to look for because then they can relate ideas. and having, I think some day to day experience, like, okay, not how would I solve this, but how have I solved
this is what makes the biggest difference.
Okay.
Erin: [00:25:02] Great. When you were at, you said you didn't know how to do a user interview in the beginning, and you're kind of always, you know, selling and too enthusiastic. Did you, did you have to learn the, you know, don't ask hypotheticals or what other, what other mistakes did you sort of learn by doing
Jonathan: [00:25:20] I have made every mistake possible. I mean, even like little things like, when someone wants to click a button, immediately clicking the button as opposed to asking, okay, what would be the next screen that would come up. It's a really un-intuitive question. It's so obvious. You just click the click the goddamn button, sorry. But the a, that is, it's not how it works. that's not, that's not what you're looking for. yeah. My co founder had a lot of hard conversations with me around, chilling out and then actually
observing him later views, being the note taker for, for a lot of the insurance before I was comfortable
it myself.
Erin: [00:25:48] Was he chill?
Jonathan: [00:25:50] yes. Oh, yeah. Very neutral. Extremely neutral. Yeah. Yeah.
JH: [00:25:53] Yeah. This is, this conversation is really running me like a year, year and a half ago when we were building a new workflow and it's kind of solution in our product for scheduling stuff with your own user base. we kind of did a very similar thing of, we were calling them like design partners and found people who were like the right profile and we'd go back to that same group at some frequency and it was super helpful.
And all the stuff you're saying is just like bringing back so many memories of. how we found those people. And one of the best things I think we did was just asking on, like, the initial kind of survey we sent around was like, how do you solve this problem currently? And you could just, you could learn so much from like the people who wrote like a paragraph or two about like, we use Google sheets, but then we try to air table and then like that, dah, dah, dah, dah.
You're like, I want to talk to that person. And then the person would be like, I don't know. Like I just find people in my base are like, all right, you, you might not have thought about this as much or if solid as well. And, Yeah. So that advice just is really a ringing true to me. it was super helpful to find people that way.
Jonathan: [00:26:46] Yeah. One of my favorite actual, this is actually a sales question, not a user interview question, but I find them are both quite helpful is what is the process for doing something, because when someone has a PR, when someone's thought enough to actually create a process, first of all, processes are painful.
Uh, things. Are easy. They don't require a process. but it, it has a little bit of, implies that there's some friction along the path and that they've had to have thought through what is the right way to solve this problem because I've seen it with some regularity. so that actually is a pretty good, it's actually a great sales question and a good user interview question.
But, just one of the very, very small Venn diagram questions that offended both camps.
JH: [00:27:17] Yeah. And now that you, uh, you mentioned launching on Monday, is the plan to just keep this up and definitely. Or is the cadence going to shift a little bit now that there's a product out in the wild? Like what? What's next for you all?
Jonathan: [00:27:28] Yeah, you good. You have to do certain things like maintain that product. You notice, uh, this last phase of, of really building for mockups has been super, super fun. I think the team is actually ready to kind of see the thing live now. I think there's like a different, a different world there.
And then also like, fundamentally, the, you know, you start getting actual data, right? People are actually using the product, and, or can't use the product. so that kind of changes the way the interviews are done, pretty significantly. So we're excited for the next phase of, of research.
Erin: [00:27:56] Yeah. Do you have a plan for how that's going to work once you go live and you have, I don't know, your product analytics, cooking and seeing what, what's happening? Do you anticipate keeping daily interviews or, seeing what happens or.
Jonathan: [00:28:11] Yeah. We're, we're still in a phase where we're still, we're still going through the daily interviews. I think we'll probably can track them down. I used to have more input sources for, for how to build things, right? when you have users using the product, you can look what they can create or where they spend time.
you know, we use FullStory, we use amplitude to sort of gather that information. And I'm excited just to compliment the. They use your interview with some of the, Muslim, the actual data on usage. So
Erin: [00:28:33] Yeah. All right. So thinking back memories on the last year, are there any moments that jump out where. you know, a user interviewer, a handful of them, really changed your trajectory or where you really, you know, had to where that humility had and let go of one of your babies or, or anything that just stands out as being like, wow, you know, this would be a, a different and worse product without, without having that insight.
Jonathan: [00:29:04] Yeah. So there was one really big fundamental shift that we made as a company. originally we basically created a, an overlay product, a little bit like a walkthrough guide, like a Pendo or an accuser or walking. it turns out those are really good solutions, uh, that already exist in the world.
and we don't really need another one of them. And I think we've had a number of interviews where you would ask someone about a problem and you have this idea for, Oh, we can solve it this way. And they would say, Oh, let me show you how I'm solving it. and I think that was, great for the user and for the user research for these interviews.
They're like, Oh, great, I have a great way of solving this problem, but, when something is good enough already, I think that's what's very important for us to kind of, figure out how to disentangle. That doesn't tell you necessarily how to do it. what the right, you know, innovative solution is that at least tells you that, this maybe isn't a problem that's on fire.
so we, we had a bunch of those conversations, so
Yeah,
JH: [00:29:56] Yeah, totally. I know there's a, there's a framework for that type of stuff. I forget what it is, but, just that notion of, you know, it's really important to ask, like, you know, how acute is this pain point? And someone might say like, you know, a scale of one to 10, a nine or a 10. But if you don't follow it up with how well is that solved for you today?
And they say like, you
Jonathan: [00:30:10] Right.
JH: [00:30:11] and you're like, well, there's not a lot opportunity there. whereas if they say something's only like a six pain point, but the current solution for them is a zero, like, that's actually a
pretty wide, you know, margin to attack.
Jonathan: [00:30:20] Yeah, exactly. Yeah. And I think, again, this is like a really big difference from sales because in sales we really trying to do is highlight a pain point and then sort of explain the consequences of not fixing that. So for example, it's not that you. somebody takes a long time. Right? That's actually not a pain point.
It's a pain point because you could have spent that time doing something else that is valuable. That's what makes it painful. and so in sales, you're trying to like uncover these pain points and then exacerbate them or kind of give them a sense of their scale and use the research. You're actually doing that as well.
but you just need to be a lot more. It's not, you're trying to convince the user interview and it's painful. It's right that you're trying to convince yourself as painful.
Um.
JH: [00:30:58] That's good frame.
Erin: [00:30:59] I like that.
