Revolutionizing Task Management with AI: Insights from Zach Swafford

Speaker 1:
0:00
Big part of the problem there is that it's using physical resources where they don't exactly know what they want and what they need. Just get started right away, I mean. There's no point in waiting.

Speaker 2:
0:23
Hello and welcome to another episode of the Accelerometer. I'm Dr Matthew Merman, CEO and co-founder of Anarchy. Today, we're going to be discussing AI project management with Zach Swofford, co-founder of Dart, an AI that helps companies automate their work work. Zach, maybe you can elaborate a little bit about what work work is yeah, absolutely.

Speaker 1:
0:47
Thanks, matt. Thanks for having me on. So work, work or work about work, is the stuff that we do that doesn't need to be done. Maybe it's not interacting with people, it's not things that only humans can do, it's just stuff that we kind of need to do to keep the business running. In our case, that's stuff that's like the low impact project management work. So something like you have to fill out a document, that, or a templated document that just needs to be done over and over, or you need to build a timeline that is the same timeline that you've built many, many times, but there's no way to you know to do that again for a new process. So dart unlike previous generations of project management tools, dart is able to automate away a lot of those low impact work tasks.

Speaker 2:
1:45
How did you get started working on this?

Speaker 1:
1:48
Yeah, absolutely so. We basically found that a bunch of people that were doing project management were using the old generation of tools tools like Asana, mondaycom, even notion and those tools were not really cutting it for them because the like these mostly teams that were trying to move really quickly with ai and that sort of thing um were not able to move as fast with the work that they um with the kind of uh, the, the stuff around their work. They were able to do the coding really fast and able to make high-impact products, but the project management was lagging behind. So we just applied the tools that we have now, that we didn't have a year or two ago, to that problem.

Speaker 2:
2:41
So were you working as a project manager before or did you just come into this feeling like I just love project managers?

Speaker 1:
2:48
It's a great question. Yeah, and it's. The truth is it's a little bit of both. I wasn't formerly a project manager my background is in AI, actually for the past 10 years or so but I was working at a startup and you know, doing the classic wearing many hats thing at the startup, and so I was doing some engineering IC work, doing some engineering management work, and then I was functioning as a project manager for my team, and so I got to see kind of firsthand where those problems were and like how they could be improved.

Speaker 2:
3:23
What were the biggest problems?

Speaker 1:
3:24
where those problems were and like how they could be improved. What were the biggest problems? Yeah, my previous job, I would say that we had. I mean, one thing I remember specifically um is that we had, um, these set of tasks that we needed to do every couple of weeks and we needed them to follow like a really rigorous template and we had to do, you know, 10 sub steps and all this other stuff. Everything was slightly different each time we did it, but for the most part it was all the same. And so, you know, we had to copy and paste tasks. We had to duplicate things manually every couple of weeks and that would take, you know, several hours of my time every week or two. Um, and that was very low leverage for me. Um, at a fast moving startup, um, that we were able to. And now, with Dart, you know, I those kinds of things that when they come up, it's just a few clicks to, uh, have have an AI enabled tool automatically do all that work for me.

Speaker 2:
4:28
That's really wonderful. So you mentioned that you've been working in AI for a while before this. What type of AI did you get started out in?

Speaker 1:
4:38
Yeah, so I did my undergrad and master's at Stanford, um, and I did all sorts of stuff then. It's funny looking back because I uh, I didn't really work on LLMs and I mean deep learning was kind of the exciting hot thing in 2012. And so we were, we did, we did a lot of fun deep learning stuff but, um, really cutting my teeth on some of that early stuff. It's funny looking back because it feels like times have already changed so fast, so much. I still feel early in my career, but I feel like a lot of the stuff that I learned in school is not what I spend my time doing now.

Speaker 2:
5:20
right, I mean, I feel like you know, I did a degree in neural networks, uh, and ai too, and I feel like so much of it is. It's just so irrelevant now and you, just like everybody who is doing that, just like immediately jumped on the llm train. They're like this is what we're always trying to do I totally agree.

Speaker 1:
5:42
It's finally. People finally got a glimpse for of like what the future actually is, and they just jumped on it as quickly as possible. Like everyone switched to work on it.

Speaker 2:
5:51
So yeah, so were you working on neural networks before this I was yeah, so at at stanford I did do some work on gans.

Speaker 1:
5:59
That was like the really hot thing right when I was graduating. There's so much fun. Absolutely yeah and uh, and that was that was a big part of my master's. And then, um, after that I went to work at a company called plenty um, and I was actually the first employee there. Plenty does indoor farming, hydroponics, and there I worked on applying more traditional machine learning techniques to basically optimizing plants. That's what I like to Optimizing plants.

Speaker 2:
6:28
Optimizing plants what happens when plants get too fast?

Speaker 1:
6:33
Good things really. The faster that you can grow and grow healthfully and well a plant, particularly in an indoor context, the better a plant farm can do. You know it's all about.

Speaker 2:
6:49
So we get more food. There's no risk of us running out of oxygen in our atmosphere or something like that. That's right. That's right, yeah, carbon dioxide. I guess we would get too much oxygen, uh-huh.

Speaker 1:
7:00
Uh-huh. Oxygen is generally good, Carbon dioxide is generally bad, I guess. So yeah, I think more plants is generally better. It's funny I've never thought about the kind of the paperclip problem for plants. We never were working on any agents smart enough to where I was worried about saying optimize plants forever. Now I'm at plenty. But yeah, it's a fun idea.

Speaker 2:
7:24
But we're getting close to that now. Right, like if we were to create some optimizing lm systems, put them in charge of our hydroponics, like we could have that problem. Yeah, is that something that worries you?

Speaker 2:
7:39
in the plant context specifically, perhaps not do you ever get worried about this with the project manager? We're going to optimize project management too much and our project manager is just going to like, take up all of the projects and solve all of the world's problems just suddenly, and then we're just going to be sitting there bored. Yeah, it's funny.

Speaker 1:
7:56
I think the I've spent a lot of time with the philosophical questions here, particularly like in school and in the past, and I think I don't think enough now about, uh, because now we actually are on the edge of this progress. I think I don't. I don't have all the answers um to to what things are actually going to look like um, right down the road. I, yeah, in the context of project management or at plenty of plants. I mean, I I don't think we're doing work that. Um, if you're optimizing a SaaS business, I guess with AI, I'm not sure if it has quite the same potential to um take up all of the world's resources building paperclips is this just because sas businesses are too small?

Speaker 2:
8:53
maybe, yeah, maybe. Um, you'd have to be running a consumer business for this to be a problem that's, that's a good idea.

Speaker 1:
9:02
That's maybe what I, that's maybe what I'm thinking right now. Yeah, I mean the paperclip problem example, the parable about an AI that gets so good at building paperclips that it converts the whole universe to paperclips. That to me feels like a big part of the problem there is that it's using physical resources and so runaway ai that is using virtual resources infinite virtual resources feels like it impacts negatively, impacts humanity a little bit less. Now it's a very short jump to say optimize project management, um. From there to say um, optimize my projects or do the best projects for my business, um. And those are both problems that we are trying to solve with dart.

Speaker 1:
10:03
So I guess, if we reframe it as, in six months, you ask dart to to not just improve your task management and your documents, but also to um, you're a consumer business, you use dart and hey, dart AI, figure out a project plan for me to dominate my category. That's something that we can help with now and we'll be able to help with even more very soon. That is, I guess, something that could then start to. If our AI agents are too good, we can start to cause problems in the real world, I think.

Speaker 2:
10:42
What is your goal for Dart in the long run?

Speaker 1:
10:46
Yeah, well, we're starting to get right onto that topic.

Speaker 1:
10:49
I think the big idea right now, I think, and where we've built a really good product right now, is around automating away some of the toil and the repetitive project management work that needs to be done. Where we go from here, and what we're already doing some of, is doing your work for you, doing the actual work itself, not just the work about work. So if the long future of Dart, and even the short-term future of Dart, is that you put in a big-picture project plan and you say, hey, I want to again dominate my category in this way and Dart helps you to define a project plan, a timeline, figure out the resources that you need, build the whole big picture of how to do that, and then executes, uses agents to execute on all the stuff that um, that it can't, basically anything that an ai can't do, um, and then leaves the sort of human interaction, um, the stakeholder management, some of the other things that humans and human pms are really good at now. Leaves that to you, to you business.

Speaker 2:
12:03
How close are we to this future? I feel like you're going to give me an exact number, like seven months, the two, four days.

Speaker 1:
12:11
Yeah, exactly, I mean Dart already does some of your work for you. Not only do we already automate some stuff, but we already can generate documents. Already automate some stuff, but we already can generate documents. Um, if you have a task to fill out, a prd or something like that, which what's a prd? Ah yeah, thank you.

Speaker 1:
12:33
Um, like a product, a document where pm might specify what they like, the, the spec for project. You can think of it as like a project spec. Essentially it's yeah, I guess, a PM term of art, so a document where you want to spec what you're going to do for a project. We can already generate that for you and that would have been the first task or the first step in a project, and soon we'll be generating resources for you. So, if you have an engineering project and you want to avoid, a big part of that project is going to be building and specing the project, designing the project, writing the code for the project and then testing it.

Speaker 1:
13:24
I think all four of those things Dart can take a first stab at in the next six months. We're already doing lots of them now. So it depends on the kind of project. We're not going to, yeah, and I guess that's going to be the real answer to your question is it depends on the kind of project. If the project is stuff that AI is close to good at, dart can do a lot of it now. If the project is stuff that AI is close to good at Dart can do a lot of it now. If the stuff is like build a rocket ship, it's going to take a little bit longer.

Speaker 2:
13:51
So I guess speccing out a project is a really interesting question. So how much does Dart really take of that process? So I imagine, like I have something in my head, I want to create something. If I don't say very much about what it is like, dark couldn't possibly know what I'm thinking, could it right?

Speaker 1:
14:12
um, no, no, it can't know what you're thinking um. Sometimes it can help you figure out what you're thinking um and help you figure stuff out. I mean mean, actually that's one of the things that we're able to do almost the best, I think. Funnily enough, if you already know exactly what you want and what you need, then just writing it down, that's something we can help with. And breaking it down and assigning it to people and building a timeline around it, those are all something we can help with. And breaking it down and assigning it to people and building a timeline around it, those are all things we can help with.

Speaker 1:
14:49
But, honestly, one of the places where our users seem to get the most value from Dart is in the case where they don't exactly know what they want and what they need, and maybe they only have a vague idea of what. I want to add a feature for this, you know, or I want to do this other thing. I think one of AI's great, one of the great things about LLMs that's really different from most other information technologies is that it can really riff and brainstorm and ideate and be a partner in figuring out what you need to do.

Speaker 2:
15:29
How do you, how do you get llms to riff? This is something that I feel like I've tried to get llms to ask me good questions about my ideas, and something that's always been a little bit challenging yeah, the, the questions in both directions.

Speaker 1:
15:45
That is, um, yeah, being a good conversation partner and asking good questions in both directions is definitely like a really hard problem. Um, right now we have we take a more um, a lot of darts. Brainstorming takes a more um, generative approach, I guess you would say, which is to say that we humans create ideas, dart AI adds some ideas. They build off of each other, and there's more generation, the conversational here's some information. Okay, cool, like, give me more information and then we can build a plan together. That's like that. That's kind of a conversational experience that we we haven't nailed either.

Speaker 2:
16:34
Actually, have you followed any of the research on active learning?

Speaker 1:
16:39
some I.

Speaker 2:
16:40
I can't consider myself an expert, though I feel like this is hinting at, like an area of researchers I haven't seen yet, but like generative, active learning. Like because active learning is usually just like you know, you're going around exploring an environment and you're training the neural network at the time and you're asking like, okay, do I have enough data right now? And then, when the answer is no, you have a specific interaction that you go back to the system and I guess in this case it's a generative network already. It's not actually learning, it's like in context learning, which is kind of different, and then you don't have this binary of I have enough information or I don't. It's what information do I have? And is this completing a world model which, yeah, that sounds like an interesting direction to go in, fascinating?

Speaker 1:
17:30
And, as we were talking about earlier, I mean I think that there are so many, because this is such an interesting and fertile and world-changing field. I think that there are so many areas to explore about that specifically. I mean you had mentioned, um. I mean I'm also picturing getting enough information to create documents and artifacts and stuff like that, and then you know, not just having that be in one thread or in one conversation, but you can also permanently generate new documents and then retrieve them later to give yourself the context. I mean this is part of our vision for Dart, where Dart asks you upfront, okay, like what's your business, what are your goals, and that kind of thing. And then, yeah, you have a conversation you build, dart builds artifacts about and a mental model about, what kind of business you are and what kind of project you're running and what you need, and then can retrieve that as needed to help you with your project management.

Speaker 2:
18:42
If you don't mind my asking how many users do you guys have now? Not at all.

Speaker 1:
18:47
Yeah, so we have more than 500 teams. I checked, trying to remember the exact number because I checked the board when I left this morning. We have probably 560 monthly active teams, which is our metric.

Speaker 2:
19:02
That's really incredible.

Speaker 1:
19:03
Yeah, it's great. It's great we have been growing a lot the past year or so because it turns out a lot of people really like AI in their project management.

Speaker 2:
19:16
How often do those teams use it?

Speaker 1:
19:20
So most one of the things about project management is that different, there are sort of different categories of user. Like on a given team you probably have one or two people that are um checking. I mean on a team of, let's say, a size of five to ten, you probably have a third of those people checking, checking in every day or every couple of hours and you know two-thirds of those people that check in between a few times a week and once a week.

Speaker 2:
19:54
Okay, wow, is latency something that matters to them at all? I guess because, like doing project management, you could generate a huge amount of information very quickly. Like, I guess, you want to have access to everything about projects, right?

Speaker 1:
20:12
So we have a web app and that's the main interface with Dart, although you can also use it through ChatGPT. I do think that in all interactions, one piece of feedback we get is that latency does matter a lot. Um, as with any web app, of course, we're trying to hit slas around how quickly the page loads and how fast each thing, how quickly each thing happens when a button is clicked so you have slas for your project management software, internal ones yeah, okay, we don't, we don't have, uh, we don't well, I feel like that's a winning point.

Speaker 2:
20:51
When you're like helping people with productivity and they're like we need uptime on this, yeah, like if this is down for any amount of time, we will sue you, I would be like, yes, absolutely yeah, we don't have, um, we don't have agreements right now that specifically, uh, yeah, where that kind of thing is in the contract.

Speaker 1:
21:10
But we have, I guess, I guess we just it may be metrics is a better term for like internal- this is like a great, you know, adversarial idea for any of your potential customers.

Speaker 1:
21:19
If you're out there and listening to this, please ask for slas, yeah absolutely, I mean we'll hit them because, uh, we know we'll hit them because the uh, we're hitting them internally, right.

Speaker 1:
21:35
But yeah, no, I mean we give ourselves really strict targets around generally, like the load time of the application and some classic, just like application metrics and how quickly each interaction happens, and we do actually quite frequently get feedback that the AI-enabled parts of the app are slower than people would like, and so we're always hunting for ways to improve that and trying to figure out the worst model we can use, while still doing the best we can, because generally worse models are faster. Yeah, I think it's. It's something we've thought a lot about about how to improve various interactions. Generally speaking, it means working around um, figuring out really smart ways to get um to get all the context into a prompt or into a given situation as quickly as possible at right, the very beginning, keep the chains as short as possible so that people can get the answers that they're looking for, the brainstorming that they need, as fast as possible.

Speaker 2:
22:37
What's been the most challenging part of working with LLMs for you so far.

Speaker 1:
22:41
I mean, we were just talking about the latency thing, so that's kind of a boring answer. So that's kind of a boring answer. It has really come up frequently with customers that they want, they expect people, particularly when you're kind of hiding the magic from them. I think people have been trained a little bit by ChatGPT specifically to just be willing to wait for like three seconds. A few years ago ago, nobody would have waited for three seconds for anything on the web, but now people are willing if, if they think it's like really cool ai, somehow people have been trained just because it's so, so good.

Speaker 1:
23:20
I think that people have been trained to wait for it. Um, when we're hiding the magic a little bit more and doing really cool stuff, but uh, it's not obviously using LLM. I think that it really bothers people when it takes three seconds. So it has been a challenge. But beyond that, what else have we really been challenged by? I mean the fact that the field evolves so quickly and the fact that the models that we started using a year ago are all deprecated and we need to use totally new tech for that. I think that's one thing I think people in the field really expect you to use like be up to speed on all the newest stuff, the newest stuff and that's great and we are generally but the features that we've built, that we built a year ago or even longer ago are not that good anymore.

Speaker 2:
24:22
So I think keeping, keeping everything good might be part of it I like this idea of like building something where you know you're hiding the magic, like chat gpt, because it begs the question do you ever get adversarial users, like I know with chat gpt, like they're constantly? People are trying to figure out what prompts are behind the scenes. Yep, is that like something that you guys ever worry about?

Speaker 1:
24:48
the short answer for us is no. We have not been worried about that. I think prop, like reverse prompt engineering, is something that I am aware of. We are aware of um, but the magic for us is not really in the prompts as much. You know, prompt engineering is an interesting field and I love it and there's all sorts of cool stuff there, but at the end of the day, almost anybody can write a sentence and put it in a chat GPT, so we don't really treat our prompts as IP really or super secret sauce. A lot more of the magic is in the rest of the project and the rest of the product, excuse me and the rest of the infrastructure.

Speaker 1:
25:36
Um, do we worry about adversarial users? Otherwise, we certainly have one in a hundred users that come in that are just basically using dart as a way, or or maybe try to push at the limits of what Dart can do, to try to use it as a way to just get directly at open AIs, apis, maybe something like that. But it's those patterns. Those usage patterns are just so different and so easy to spot that it has been a big challenge for us to prevent that. And also the APIs that we use are themselves perhaps even too careful with rejecting prompts or whatnot. That data that is problematic, I guess.

Speaker 2:
26:33
So where do you really consider your secret socks?

Speaker 1:
26:37
Yeah, I guess. So where do you really consider your secret sauce? Yeah, great question. I think for now, the secret sauce is in our understanding of how really modern teams are using AI for project management. I think that other project management tools have not moved at the speed that we have to use to really understand how people want to be interacting with their tools in this like brave new world. We're the first LLM native project management tool and I think that that has been our main advantage thus far. Going forward, I think that we are working on building a data asset, to a large degree, of what people's projects look like and how people use documents, tasks, projects, et cetera, and that data asset will be, I mean, I think, increasingly valuable. I feel like everyone's talking about data assets these days, for I mean, well, they have been for a long time, but particularly in the context of LLMs, I think that data assets will be critical.

Speaker 2:
27:53
So this is your first startup right.

Speaker 1:
27:56
The first one I founded. Yeah, exactly I um, I was really early at plenty, Um, but yeah, it's the first one I founded.

Speaker 2:
28:03
What's your founder journey been like so far?

Speaker 1:
28:06
Well, the the journey has been awesome. We did YC um in the winter 22. So about two years ago um, I've been working with my good friend malad malik um and he and I we worked together on lots of stuff before this um. We really complement each other's skills really well. Um, having good co-founders and good founder friends. I mean, coming out of stanford I feel like I knew so many people founding um and still do that.

Speaker 1:
28:36
That having a really strong support network around has made my journey infinitely easier and I do not envy founders in the past who were going it alone. You know we with project management and I'm sure there are a lot of other similar categories applications it is a really well evolvedevolved, well-defined space which has made our job harder in some ways and easier in others. So I think one of our biggest big-picture challenges has been figuring out a course and a tactic and a set of tactics and strategies to really differentiate ourselves and stand out above and beyond what exists already. That's been, I think, our biggest existential challenge. I think we're doing well with it, but there's a lot more to go and we do need yeah, that's our main challenge as a business. That's what keeps me up at night is differentiating Dart from what exists, but it's been a wonderful journey and I would certainly recommend it to anybody.

Speaker 1:
29:39
Um, anybody who's thinking about founding, I always say this um, I loved my journey at plenty, but, um, and at the time what I was thinking was, oh, I'll kind of cut my teeth at a different startup and then grow into like a founder role, and I did learn so much from my previous experiences. But if I had to give advice to anybody starting off right now, it would be to just get started right away. I mean, there's no point in waiting. You'll learn best by doing the thing, by practicing what you're trying to do, not by getting advanced degrees or by going to a big company or even by going to a smaller company. That's like the company that you want to build yourself. I would say just just try to build it yourself now.

Speaker 2:
30:28
I definitely feel that I did a lot of degrees on my way, thinking like, yes, these will be useful for starting a company. It was starting a company, know it was starting a company was just so different, like in any way that I could have imagined. I just don't think it's possible, like through any amount of reading or thinking or studying, to anticipate the day-to-day challenges that you're actually going to face over, like I didn't anticipate like one of my biggest challenges personally starting a company would be time management, like like when I was building or when I was in my phd. Uh, time management is also important there, but you have control. You have control over all of your time. You rarely have a meeting with anybody else, whereas in a startup it just creeps up on you. Like you, you know you have other people who need to have meetings with you and then you have meetings that you plan and suddenly you're just like wait a second, I can't screw up my entire schedule for this week. I've got 20 meetings Totally.

Speaker 1:
31:22
Totally yeah and yeah, figuring out how to run that for yourself. What I've found is it's a very personal. There are so many cool tools and but there's no, there's no magic bullet, and so you know, even to just circle back to the point about, you should do that as soon as you can to learn what works for you, because some people it's like blocking off time on the calendar, some people it's using a particular tool. There are so many good strategies for time management, for everything, but you don't really know until you try yeah, so what's been since starting?

Speaker 2:
31:57
what's been the hardest part for you?

Speaker 1:
31:59
Well, on a personal level, I think one thing I think about a lot is this building a sustainable business that can run really well without the level that I'm putting into it right now. You know it's been about two years and I've been working pretty hard. You know I, I, I have a pretty rigorous time management process myself, and so I actually, you know, I know that I spend somewhere between 70 and 80 hours each week, um, working, and I think that is sustainable for me, at least for a little bit longer. But on a personal level, I'm thinking about kids soon, and so I think one thing that, for me, at least for a little bit longer, but on a personal level, I'm thinking about kids soon, and so I think one thing that keeps me up at night there is, like, how can I do a good job with both my startup and my kids and give both the time that they need to flourish? But that's more of a future facing challenge than a present one.

Speaker 1:
32:53
It's a champagne problem. That's how it is. Yeah, exactly, um, what's been the, what's been the biggest challenge thus far? I mean it's big and cliche, but just finding product market fit is is so hard. I mean we, we, we have an awesome product, we have a lot of really good users we have, we're making good money and we've got path forward to do more, raise more, everything we need to do but we don't have product market fit yet in the traditional sense. The rock isn't rolling down the hill ahead of us. We're still pushing the rock up, and so I mean the big problem that we spend our time thinking about and worrying about is how are we, how do we get all the way to product market fit?

Speaker 2:
33:40
What's something that you wish you'd done differently in your startup?

Speaker 1:
33:45
Good question, oh man, and that's tough too, because we already said the. I already said the one about going straight to it rather than.

Speaker 2:
33:53
I mean, I think we all wish we could have. We could have started our startups when we were 5 years old. We've just all been working on it for the last 30 years exactly yeah, yeah right.

Speaker 1:
34:02
I just wish I had invented the internet. That would have been really cool. It would have been great man.

Speaker 1:
34:07
Yeah, what's something I wish I had done differently.

Speaker 1:
34:11
I think that always building stuff that you know people want and then validating it with those people right away and learning from it right away.

Speaker 1:
34:32
I think the only the places where we've wasted time and luckily this hasn't happened often, but it has happened are places where we built something that we didn't have enough insight into. We didn't, we were just adding a feature that sounded cool, or it was a CISP, a solution in search of a problem, or it was just something that only one team had asked for and we had yet to validate that it was a true need of our category. I think those are the places where we wasted our precious time as a company, and so we do our best to avoid that kind of stuff now. But I would say always always being really sure that what you're building, if you have users, always talk to them, but don't just talk to them about you. Talk to them about the stuff that you're building and once you build something, validate it with those same users, run it past them and, if they don't like it, iterate on it or scrap it, but don't build stuff and put stuff in your product that is more theoretically useful than actually useful.

Speaker 2:
35:44
How do you stay close to your users?

Speaker 1:
35:46
Oh, so many ways. I mean that's like a life quest, a company quest. We do a lot of the classic things. We talk to people. We do net promoter score surveys. We get on calls. We try to do five to ten calls per week with users. We run surveys. We use Sentry and Datadog to collect information about what people are doing with the product so that we can notice the hot spots and the problems and that we can see how people are using things. We use Amplitude for analytics. That's, I guess, maybe the most important one. We love just when users will share with us how they're using Dart. So a lot of the time people will send us the prompts that they use or we'll ask them hey, can you share with us some of the tasks that Dart made and the ways that you're using the product? And so when we actually get a chance to see how people are using Dart themselves, I think those are often the times that we learn the most.

Speaker 2:
37:01
Are there any tools that have been particularly helpful to you guys building an LLM app?

Speaker 1:
37:07
Honestly, we have. Mostly we're a pretty engineering for better or worse, we're a pretty eng forward team and so we've mostly done stuff ourselves directly. Um, we've yet to really really branch out into some of the, some of the projects, many of which I know are great, um to to help us with some of the um, yeah, we, we've kind of run our own for a lot of the stuff.

Speaker 2:
37:38
So you guys don't use any LLM observability tools, not right now, any of the other prompt engineering IDEs.

Speaker 1:
37:46
Nope, and I think maybe part of that is that we got started on it so early. I mean, it was kind of something we were building from the very beginning and yeah, again, for better or worse. We're kind of like we were building from the very beginning and, yeah, again, for better or worse, we're kind of like do-it-ourselves-ish team, which I think I've seen both sides of that. I know there are pros and cons but yeah, we generally like to build that stuff ourselves and because we were working on it so early on, a lot of those tools just didn't exist when we got started, so we haven't fully incorporated them yet, but I know we will.

Speaker 2:
38:18
So when you say you were working on it very early on, was this before GPT was generally released?

Speaker 1:
38:26
Exactly, yeah, yeah. Yeah, we weren't able to release any stuff for production before the general release, but we had a lot of stuff built and and ready to go, um, like right around like last november and stuff, and so we were able to kind of capture the fever to a certain extent at that time, which was fun what's it been like watching the current crop of startups that came once gpt was released?

Speaker 1:
38:49
yeah, I mean it's amazing. I think it's so cool to see so many smart and motivated and excited people all facing not the same direction, but close to the same direction and working on stuff that is so important and useful and powerful for the future. And I know that many of the companies that people are building are going to be super successful. And I mean I think it's awesome. I think it's super cool. When I was in school, I think one of the hottest things that everyone was talking about was crypto and Bitcoin and everything, and I was less excited about crypto then, and I was less excited about crypto then, and I'm in some ways, envious of people that are really early on in their careers now, because I think this is the real deal.

Speaker 2:
39:47
Yeah, I think that's a great note to head out on. Thank you very much for coming on.

Speaker 1:
39:53
Absolutely.

Speaker 2:
39:54
Yeah, this was really great. Thanks, matthew.