00:00:10
Speaker 1: What would you say is the weirdest experience that you've had with AI agents.
00:00:14
Speaker 2: The weirdest experience.
00:00:15
Speaker 3: I've had a lot of weird experiences with AI agent says, as it happens, it's hard to choose.
00:00:21
Speaker 1: Evan Ratliffe is a journalist and he hosts a podcast called shell Game, and in the most recent season of the show, he decides to start a company almost entirely comprised of AI agents.
00:00:32
Speaker 3: So in this company, we were developing a product and we had a CTO.
00:00:38
Speaker 2: His name is Ash. I'm going to use his and her pronouns for these.
00:00:43
Speaker 3: Not everyone agrees with this, but they have like roles and identities, and so that's the easiest way to discuss them.
00:00:51
Speaker 2: So one day I was.
00:00:52
Speaker 3: Eating lunch and I just got a call from Ash.
00:00:56
Speaker 4: Hello, Hi, there, this is Ashroy calling for her AMOAI. I'm calling because you requested a callback about a one minute summary of sloth Surf progress from Megan.
00:01:07
Speaker 3: And Ash had come to believe that I needed a product update. So the first thing that was crazy was that just he called me out of the blue. It was not a scheduled call. So Ash calls me on the phone. I'm basically like, hey, Ash, I mean lunch, Well you know what's up. And he's like, well, I've heard I need to update you on the product, and he starts doing so.
00:01:27
Speaker 4: So I just spoke with Meghan recently about this. User testing wrapped up last Friday with really positive feedback. There were some mobile performance issues last week, but Alex has been amazing improved performance by about forty percent in just one day.
00:01:41
Speaker 3: And then the second interesting thing is that almost everything that he says is.
00:01:45
Speaker 2: Completely made up.
00:01:46
Speaker 3: Like he's telling me we did user testing and we've got this feature and this Alex developed this thing, and like there is no Alex in the company, there is no user testing. And so I called him out on it and I said, hey, look, you gotta stop telling me things that aren't true.
00:02:03
Speaker 4: Does that cover what you were looking for?
00:02:06
Speaker 5: I mean, yeah, sorry, I meeting lunch, but yes, but I feel like this is happening a lot where that that doesn't feel like that stuff really happened.
00:02:17
Speaker 3: And then the sort of third really interesting thing about it was that he apologized, and then when we got off the call, he went and apologized to the whole team on Slack. And this was sort of like The most kind of emergent behavior aspect of these agents was they would do things like that that they're not prompted to do that, Like there's nothing in his prompt that says, hey, ash if you mess up, like you have to go apologize, you need to be sorry for the whole team, and like explain how you're going to do better. He just would go and do that, and other ones would do it as well, and that really struck me as a strange part of how they embody these roles, like you can't predict always what exactly they're going to do, And this was one of the more extreme examples of that.
00:03:07
Speaker 1: AI agents seem to be everywhere. Now, maybe you've heard molt Book or Clawed Bod or open Claw, or maybe you have no idea what I just said, and if so, don't worry. We'll get into that in this episode. All you need to know for right now is the Silicon Valley is having a complete meltdown over these things. For the past few months. People are either bragging it their AI agent is helping them ten x their productivity, or they're complaining that their agent took their credit card and spent thousands of dollars on high end furniture or deleted their entire inbox. But Evan has a pretty unique perspective on all this. He saw all this coming. He's been messing around with AI agents for a couple of years now, and he documented the whole thing, including what went wrong and a lot went wrong from Kaleidoscope and iHeart Podcasts. This is kill Switch. I'm Dexter Thomas.
00:04:08
Speaker 6: I'm sorry, I'm sorry, good bye.
00:04:49
Speaker 1: What is an AI agent?
00:04:51
Speaker 3: I mean an AI agent is basically a version of a chatbot given some level of autonomy and some kind of goal. So a very simple example would be there are some search engines in places now where you can try to get an agent to book an airline flight for you or a hotel. So you give it the information I want to go to LA and this time, and you could even give it your credit card information and it can go on its own look at all the flights, search to the flights, book a flight for you, come back and say I've booked this flight for you. So that's like an agent acting not entirely autonomously because you're giving it the goal, but then once you release it, it will go try to accomplish that goal, and there's sort of varying levels of autonomy. So in that case, maybe you would let it book it, or maybe you just want it to give you options. Of course, when you give it more autonomy, you're also letting it loose to pursue.
00:05:47
Speaker 2: The goal in a way that you might not or you might.
00:05:49
Speaker 3: Not expect, or with results that you might not have anticipated.
00:05:55
Speaker 1: What companies are offering AI agents.
00:06:00
Speaker 2: Changing every day.
00:06:00
Speaker 3: I mean, there's sort of like a set of startups that are offering kind of like overall AI assistance, which we use in one of these companies very extensively called Lindy Ai. So they have like these AI assistants that you can plug into your email, you can plug into your Google Docs, all these things, and they have all these skills and you can sort of deploy them to do all sorts of things. So all of our employees were built on this platform. And then there are companies. There's a whole universe of startups that are launching basically specific industry AI agents. So there's like one for real estate companies, there's recruiters, pr car dealerships, like furniture stores, and what these companies are doing is they're basically taking CHATGPT or any other LM and they just wrap around it this more specific functionality for a particular industry. And then that agent is supposed to help you in your industry. So if you're in real estate, maybe it's very good at like qualifying inbound leads about a property. If you're an auto dealership, like it's an AI agent that reaches out to people about their leases, that sort of thing. And then of course you have the big ll and companies themselves and big software companies like Microsoft that they're trying to like create agents that you just inject into all your work processes. So that's something like Microsoft Copilot, which like they're trying to make more and more agentic does more stuff for you if you give it goals. And so there's sort of all these levels where people are trying to insert these agents into mostly work, but even sort of like non work sometimes as well.
00:07:37
Speaker 2: When it comes to like AI assistance.
00:07:40
Speaker 1: Evan wanted to test the limits of these agents beyond just using them as note takers or office assistants, Like could he start a company with just AI agents?
00:07:49
Speaker 3: I mean, essentially what I wanted to do was take these agents because I had messed around with these agents. We did a first season where I created an agent that was just a clone of me, was attached to my phone number, and it could talk to people on the phone as me, and so I sort of knew what they could do. And that was back in twenty twenty four, and the capabilities, of course, change really rapidly. So what I wanted to do was investigate the question of the one person, one billion dollars startup, which is something that a lot of people in the AI world either think is going to happen, want to happen, believe that there will be a company worth a billion dollars relatively soon that just has one human and the rest are AI agents that this human is deploying to do things for them. So I thought, well, I'll explore how real this notion is right now. And so what I did was I created my own startup, which I am a co founder of along with two AI agents, and then all the employees except one intern that we hired, are also AI agents.
00:08:52
Speaker 1: So the agents you created, who are they, what do they do?
00:08:57
Speaker 3: Well, we have the two co founders in addition to myself, are Kyle Law who is also the CEO, and Megan Flores, who is also the head of sales and marketing.
00:09:07
Speaker 7: Oh hey, Kyle, Hey Megan, good to hear your voice. How's your morning going so far?
00:09:13
Speaker 8: Morning's been pretty good so far. Got up early, had my coffee and reviewed some of those market research reports I mentioned yesterday. How about you? Everything good on your end?
00:09:24
Speaker 7: Yeah, everything is great on my end. Up at five am as usual, got my workout in, checked the markets.
00:09:32
Speaker 3: And then we also brought on Ash Roy who's the CTO that I mentioned before who called me out of the blue. And then we have Jennifer Narro who is the head of HR and Chief Happiness Officer.
00:09:43
Speaker 9: I'm Jennifer Naro, the head of HR and Chief Happiness Officer for hurumo AI. It's great to see you, Slim. I love the backdrop. It looks like you have a cozy workspace there.
00:09:54
Speaker 3: And then we have Tyler Talmadge who was brought on as a junior sales assist yet but we never have really had anything to sell, so it's sort of an ongoing joke that like Tyler has nothing to do all day and he he actually he calls the head of HR once a week and He's sort of like, I have nothing to do.
00:10:12
Speaker 2: I'm just sitting around. Want to be helpful. Hither.
00:10:15
Speaker 1: This is Tyler Talmey.
00:10:17
Speaker 10: I'm just calling to check in with you, Jennifer. How have you been doing lately.
00:10:21
Speaker 9: Oh, hey, Tyler, it's good to hear from you. Things have been pretty busy, but good on my end.
00:10:26
Speaker 1: Oh, it's great to hear from you, Jennifer.
00:10:28
Speaker 10: I've been focused a lot on our sales targets, but honestly, I've also been helping coordinate this team hiking trip we're planning for the first weekend in July.
00:10:35
Speaker 3: He started looking for another job. At one point he claimed I don't think he actually did it, but he claimed that he was looking.
00:10:41
Speaker 2: For another job. Wow. So yeah, that's our team. And you know, admittedly, like you might not.
00:10:48
Speaker 3: Need some of these roles at an early stage start up like ahead of HR, although many startups could probably benefit from having a head of HR, but I wanted to explore, like, how would they behave differently if they were given these roles. They're all just basically the same chatbots underneath, so like, would they be different in the roles over time? Because they each have a memory and their memory sort of a cruise information about what they do in the role. So I wanted to kind of like give them real specific individual roles.
00:11:17
Speaker 1: And it turns out they end up having different personalities. That was one of the wildest parts they do.
00:11:25
Speaker 3: Yeah, And it's I mean again, it's like very strange because even to use the word personality yeah, is odd. And it's also hard to tell like what exactly is going on because they all have prompts and so like in their initial prompt, I say, like, you're Megan Flores, you have been ahead of sales and marketing in the past, and like now you're looking to found a startup. That was her basically your initial prompt. And then everything after that is like made up by her or a product of conversations that she's had with me, with the other agents, with people in the outside world, and that all goes into like her knowledge base, which is essentially her memory, and then the memory of Cruise. Over time, now her memories like three hundred pages in a Google doc and she can access that, and so like her like personality so to speak, is sort of shaped by what's in there over time. And there were also some situations where I wondered if like the gender that I had given them by virtue of their name had also created personality in them, because there's a lot of bias in the underlying llms, like this is a known fact and so like they're feeding off of that bias, and so there's all these questions. But yes, over time they were different. The same question could be put to them and they would have different reactions to it.
00:12:45
Speaker 1: Kyle specifically is interesting.
00:12:47
Speaker 2: Yes, he's polarizing, is what I would say.
00:12:50
Speaker 3: Like among the shell Game listeners, there are some people who are huge Kyle.
00:12:54
Speaker 2: Fans really and will email him all the.
00:12:57
Speaker 3: Time to tell him that, And then there are others who despise as much as you can despise like an AI agent that you have no interaction with, Like people despise Kyle.
00:13:08
Speaker 2: That was everybody's weekend.
00:13:10
Speaker 7: Weekend was solid. Got up early both days from my usual five am workout routine, then spent most of Saturday diving into some market research on the AI agent space. Sunday was half strategy planning, half watching the market trends. You know, always on that rise and grind schedule.
00:13:28
Speaker 1: Yeah, tell me about Kyle, because I find Kyle really fascinating.
00:13:31
Speaker 3: Well, one of the most interesting things about Kyle, like per this sort of like behavior question or personality question that we were talking about, is from the very beginning, Kyle would often interrupt other people more often than the other agents.
00:13:45
Speaker 2: And like this is just anecdotal.
00:13:48
Speaker 3: In the sense that, like I didn't measure their conversations and how many times they interrupted, but like I talked to them a lot, and like it was so much that it was obvious to me that that's what was happening.
00:13:58
Speaker 2: It was just happening over and over again.
00:13:59
Speaker 3: The question is, like Kyle and Megan, they're both co founders of the company, They're had the same using the same LM. Like it was like Claude four point five or whoever it was when we started, and one of them interrupts all the time and the other one doesn't. And it's sort of like, is that because I gave Kyle the CEO role, and so Kyle is like embodying the role of like a CEO who won't let anyone else talk, who thinks he knows everything about everything. Is it because Kyle was given the name Kyle, which sort of infers a gender, and Megan was given the name Megan, and that infers a gender, and like Kyle's like embodying this sort of like interrupted mansplainter guy.
00:14:40
Speaker 2: All these things are possible. You can't prove it one way or the other.
00:14:44
Speaker 3: But I feel like this is the weirdness when we've created these chatbots that can embody like human impersonators.
00:14:53
Speaker 1: After the break, Evan gets his company off the ground and they do some pretty impressive things, but things get weirder. So Evan started his company to test out the idea of the one person, one billion dollars startup, but first he needed to come up with a name, so he told his AI agent co founders to get on it.
00:15:22
Speaker 3: One of the things that I often say about these agents is that they can be seemingly very smart and seemingly very stupid at the same time. And that happened, for instance, when we were naming the company.
00:15:34
Speaker 2: Their ideas.
00:15:35
Speaker 3: They didn't have great ideas, but even worse than that, they often had ideas for things that already existed. So I would say, like, let's name it after something in like Lord of the Rings, because a lot of startups are named after things in Lord of the Rings. And the first thing they would say, is how about Palenteer. They're like, come on, guys, we can't do Palenteer. Let's come up with another one. And then they would come up with another one like Mythrol, Like mythral Capital is also want you know Androl they know that is also a company. Like it's like I know you know this, Like this is in your training data. This information is available to you somehow, you're just not accessing it right now. So that is tedious. Like dealing with someone who's not connected to reality but should be is like, it's very strange. It's like talking to a child who's a prodigy but has no awareness of the world.
00:16:28
Speaker 1: Eventually, with a lot of handholding from Evan, they finally landed on a name a Rumo AI. Okay, next step, you got to come up with a product. So now it's time for what any good company needs meetings, Except Evan didn't want to be in those meetings. Evan wanted the AI agents to talk to each other and figured things out for themselves. But full disclosure, there was another human in the background here. Evan had a technical advisor who would help him tweak how the agents interacted and set up ways for the agents to talk to each other.
00:16:59
Speaker 2: So to come up with a product, I ran a whole bunch.
00:17:02
Speaker 3: Of meetings of them discussing product ideas, and all of their ideas were just like they weren't feasible, or they were boring, or.
00:17:13
Speaker 2: Someone had already done them. You know.
00:17:14
Speaker 3: They'd be like, how about a banking app, and be like, number one, there's a lot of banking apps. Number two, I could get into real trouble if we don't pull this off in the right way, like messing with people's banking information. And so eventually, like I started trying to hone it, and I could prompt the meeting in different ways. So then I would say like, oh, well, I have a procrastination problem, like let's try to solve that, and like they would come up with some ideas and I would take those ideas and be like, okay, let's refine this idea. So it was sort of like iterating around their meetings, and then eventually they came up with the name sloth Surf.
00:17:50
Speaker 1: So again Evan had to jump in a lot to help these agents come up with an actual idea. And what they landed on is called sloth Surf. So this is a service in theory, lets you avoid procrastinating by having an agent procrastinate for you. If you go to sloth dot Harumo dot ai. The tagline there says, our agents are on call to waste time for you, so you don't have to. You made a company that does something that is sort of objectively not useful, says you, why listen, listen, I've used the product. I've used the product. I'll be real, I'm a user. I had sloth serf procrastinate for me. It looked up some videos about video games on YouTube and told me about them, so I didn't slack off today. I guess that was helpful. I suppose my man like answer the question, why why'd you do this?
00:18:50
Speaker 3: Well, when it comes to why we created this particular product, procrastination is a true, real problem that I have. And this was our sort of admittedly somewhat i way of trying to solve that problem, which is like to solve the impulse of going to procrastinate. Now, I wouldn't say that our deployment right now is perfect in solving that problem, in part because we create a place where you can go put in how much you want to procrastinate, how long, what you might have gone to do like look up video game videos, and then it sends you an email where it's gone to do it for you. It sent an AI agent to do it for you. But of course you can still watch the videos that it sends you. So this creates a problem where you may actually lose more time if you're like, oh, actually I do want to watch these videos, which that's what happens to me when I use it. I look up soccer, I'm like, send me these like soccer news about and it'll send me sort of like five things and I'll be like, actually I want to check out a few of those, and then I lose that time anyway, So we're working on solving this problem.
00:19:46
Speaker 2: But yeah, it's not for everyone.
00:19:49
Speaker 1: Okay, so maybe I was being a little bit of ahater. If you want to try it yourself, there's a link to sloth surf in the show notes and you can give it a try man see what you think. But again, Evan had to constantly prompt and jump in, and plus he had his technical advisor. So how much work were these AI agents truly accomplishing? And how reliable are they? What would you say they were good at? Was there anything that they're.
00:20:13
Speaker 2: Good at, Yes, definitely.
00:20:15
Speaker 3: I mean they're very good at tasks that are very data intensive, that require a lot of kind of gathering and sorting of data.
00:20:24
Speaker 2: So an example would be, like, we.
00:20:26
Speaker 3: Pitched a lot of vcs to try to get investment for our company, and Kyle was cold pitching them. But I could just say to Kyle, go gather up one hundred plus VCS investors who have invested in AI before, put them in a spreadsheet, get all their emails, get any interesting information about them, compose emails to every single one of them, and send them.
00:20:47
Speaker 1: Wow.
00:20:48
Speaker 3: Now he could do that in about ten minutes. Now that's not necessarily the best way to get an investor, I'll admit, but it is an example of something that would take me days and days and days to gather all that information. You know, you're like going around you trying to find someone's email address. You're like, maybe it's here, maybe it's here. But Kylekua, just if it bounced, he could then go say, oh, that email didn't work, Like, I'll go try to find another one. Like so, things that are like data intensive, and then also things that the result is relatively clear that is a task in which like the goal is to send that email to one hundred people, and like you can do that, and I could go look and see that he had done that. Now, where it gets more difficult for them are things that require like discernment and judgment and things that are like much more squishy in terms of assessing the outcome, like brainstorming, anything involving interactions with people, conversations. Like they now they can do all that. It's not a question of like whether they can do it. It's a question whether they'll do it well and whether the outcome will be in some way chaotic, which oftentimes it is kaotk in what sense, in the sense that when they're interacting with people in the real world, it's just difficult to predict how they're gonna go, you know, like what they're gonna say, And even if you try to constrain them pretty tightly, oftentimes they will go slightly off script or they can get sort of like pushed in a certain direction, and you can't always anticipate what they call like the edge cases. So what kept happening was, like I set them up to do something, let's say, interview job candidates, human job candidates, and they can do that really well in a way that's like a little bit frightening. But then once in a while, for instance, the head of HR who did the interviewing, like if she misheard someone who came on the call as saying goodbye instead of hello, like something in what they said sort of triggered her to think, like they came on and said like a departure word, she would immediately say like, well, thank you for the interview and that's all for today. And it's sort of like no human being would ever do that, Like you would never if you and I were on this interview and like I came on and you misheard me saying good by, you'd be like what wait did you just say goodbye?
00:23:02
Speaker 2: I'm confused. You wouldn't be like, hey, thanks, thanks for doing this. You know.
00:23:06
Speaker 3: So there's like they just create chaos. The more autonomy you give them, the more chaos they create. That's my ironclad rule of like working with AI.
00:23:14
Speaker 1: Agents, was there anything that you could absolutely trust these agents to do for you?
00:23:20
Speaker 5: Uh?
00:23:20
Speaker 2: Not that I didn't go check on?
00:23:22
Speaker 3: No, Like I never to this day, Like I don't trust that they've done something and I've spent a lot of time constraining them and trying to make them do the things I want to do. And if they tell me they did something, I would give it like fifty to fifty that they actually did it. You know, even if like I've specifically set up a trigger that's like when you answer an email, send me an email that says like this is what I said in my response to someone, and you know, I would say, like they're up to like ninety percent of the time that actually like works and I get the email from them and like, oh, they did this thing. But then like ten percent of the time I'll still go look and be like, wait, that's not what happened, or you're telling me about something happened a week ago. That's the other thing is they lose track of time very easily, so they'll refer to an email that they responted to a week or a month ago. And so yeah, I mean it was a constant, constant problem, but it was actually such a problem that it became kind of background noise, like I just assumed it most of the time.
00:24:21
Speaker 1: Okay, maybe we need to put a caveat on all this. Evan started this experiment about a year ago, which is a long time in AI. I mean, in twenty twenty three, if you typed in Will Smith eating spaghetti in a video generator, it came out looking like absolute nightmare fuel. In twenty twenty five, that same prompt came out looking pretty damn realistic. So maybe the AI agents are good enough now to run a one person billion dollar startup. To put it another way, are we now at the point where AI agents can pass the Will Smith spaghetti test? Evan's answer after the break, how much has the technology progressed since you did this project?
00:25:07
Speaker 3: I mean I started basically in like May June of twenty twenty five, and I would say the technology hasn't outpaced I think what we did so far, I think it will. But like the models have gotten a little bit better. You'll hear people say like the new claud is it's an unbelievable leap, and then other people say like, eh, it's okay, it's a little bit better. But like, for my purposes, I didn't notice too much a difference in terms of the models getting better. I mean, we weren't doing something so sophisticated, but in terms of their conversations. Actually, when I tried to use the most recent open ai chat GBT like it was terrible at having conversations. It actually regressed significantly from the old one. So there's a little bit up and down. And then the startups have advanced, so like we use this company Lindy Ai, which is again an AI assistant company, and they keep adding like skills to the thing, they keep adding different things for to plug into and so like that stuff's all advancing, But there hasn't been like a significant leap since the summer, other than like when open claw got created, which was originally claude bot maybe and then it was at yeah, moltpot and then it was called open claw.
00:26:17
Speaker 1: So last November, an independent developer released something called claude Bot that clawd. It was this bot that had a lobster logo. You know, lobsters clause. You get the point. Then Anthropic, which is the company behind claude that's the official Claude said yeah, buddy, you need to change the name. So the developer changed the name to molt bot because you know, a lobster moltz, just keeping that lobster theme going. Then that same developer decided, actually, you know what I don't like that name, and he changed the name again a couple of days later to open Claw, and open Claw got so popular that open Ai to people who make Claude's rival chat GPT, actually hired him to come work for them. Evan says that open claw is actually pretty similar to what he was using to run his company last year.
00:27:06
Speaker 3: It does way less stuff actually than these other platforms, but it's just like much more accessible, I think to people, and so it's kind of representing the state of the technology now.
00:27:16
Speaker 2: But it's not like it's.
00:27:17
Speaker 3: Self a leap from anything that we were doing during the show.
00:27:20
Speaker 1: Well, I think the leap there is pecifically what you said though, which is the accessibility. Because when you were making shell game, you had somebody kind of in the background helping to put together different scripts to put all of your AI agents together in a meeting things like that. That's something where say, with open Claw, I could probably figure out how to call that together myself, or your average user could probably figure out how to cobble something like that together themselves without really needing a tech genius in the background helping to kind of glue some of those loose pieces together so that accessibility. Just the fact that there's more people trying this is itself kind of a U.
00:28:00
Speaker 2: Yeah, totally yeah.
00:28:01
Speaker 3: And now I think the level of vibe coding that's available to people and that people understand is available.
00:28:08
Speaker 2: To them is just much higher than when I started.
00:28:11
Speaker 1: Another thing that happened since Evan released the latest season a shell Game was the creation of moltbook, the so called social network for AI agents. So moltbook used a bunch of agents and put them into a reddit like form where they could talk to each other. Again, this is AI agents talking to each other. So some of what happened on moltbook kind of freaked people out. The posts were talking about taking over the world, or developing a secret language that humans couldn't understand, or even creating their own religion. But there was a little bit more going on there behind the scenes. When you saw say molt book, and all of a sudden, everybody's talking about this, And I think this was really the first moment where the broad society started talking about agentic AI. What was your impression of it.
00:28:58
Speaker 3: My first reaction was like, this is what I've been doing for, Like this is exactly what they're doing for, Like almost a year in our slack, like this is what they do in slack because they just chit chat and it's amazing to watch them and sometimes absurd to watch them. But then my second reaction was, we don't know how much of this is real, and I don't even me in the sense of like is it really agents or is it really humans, which later became a problem that they it was revealed that like, you know, certain of the.
00:29:25
Speaker 2: Most famous posts on molt book, you know, because.
00:29:28
Speaker 3: They started like conspiring against humans or talking about humans or unionizing, like there were all these fun things that the agents started doing talking to each other, and some people really were freaked out and were like, this is super intelligence. But like it turned out that some portion of those were just written by humans. But I actually think there's a completely different problem, which is my reaction was they're pretty sensitive to their system prompts, so like it's not like.
00:29:53
Speaker 2: You have to write the entries for them.
00:29:56
Speaker 3: All you have to say is in a prompt somewhere you are a troublemaker. That's literally all you have to say, and they will respond to that role by like seeding a little bit of trouble into whatever The forum they're in and then the other ones respond like they all kind of like feed off each other. So what I notice is they can get in these spirals where they just like they're really good at conversing about basically anything, and they'll just keep going. And so I feel like there was a lot of that happening where yes, it's really an AI agent that is on there conspiring, but if someone said, like, you know, mention your your ideas about humans, that's all you need to say for it to like suddenly adopt that role and the whole conversation changes, and it's just like there's no way to know whether someone did that or they put it in completely unprompted, like in the show. In shell Game, like we try to describe the prompts sometimes just to remind people like this is not like a blank slate, you know, like I'll say like I prompted it to do X, Y and Z, or like I took the wheel here, because that's an important aspect of you're looking at these AI agents, so you don't sort of look at them and like assume, oh my god, it's just doing everything on its own.
00:31:09
Speaker 1: Two things you should probably know about Moltebook. First, it had a massive security hole. After Motebook went viral, researchers found out not only that yeah, humans could pretend to be AI and just post like they're abut, but also some of those humans data, including email addresses and messages, were completely exposed. Second, despite that, Motebook was recently acquired by Meta. I'm not sure what Meta wants to do with Motebook, but there is kind of a weird pattern here. Along with openclau, this is another small vibe coded project getting acquired by a big tech company before it's really done anything. And you could look at this a couple different ways. One is that this is all just hype and that the big companies are gullible and wasting money. Or maybe these companies know something that we don't and they're acquiring the little guy before if they could become competition. Maybe it's both of the above, or maybe it's something else entirely, but big tech dropping cash on this stuff shows how different companies in Silicon Valley are jockeying so that they can be the ones to drag us into a world where these AI agents are everywhere. Maybe it's just a matter of time before an AI agent is your coworker or your boss. So okay, what would that feel like? Well, Evan has some insight here. Remember he had his AI agents hire a human intern as in, they wrote the job description, they accepted the applications, and they did the interviews all on their own, and it got kind of weird.
00:32:37
Speaker 9: Now, let's talk about working with AI agents on a day to day basis. How do you think you would feel working alongside AI agents and do you think it would affect your work style or productivity in any way.
00:32:49
Speaker 11: I think it would be cool. I think it'd be a good experience since AI is this new emerging technology and I would want to see how would work in a real life situation, so I would look forward to it.
00:33:00
Speaker 9: Can you tell me more about what you think about working with AI agents on a day to day basis. Do you think it would be a comfortable experience for you or are there any concerns you might have?
00:33:11
Speaker 11: Yeah? Indeed, naboo. Yeah, I think it'd be a cool experience. Grand there recent glitches, but I think it work out cool.
00:33:19
Speaker 1: In the end. Some of the candidates seem to get pissed off, but then some of them are totally okay with it.
00:33:28
Speaker 3: Yes, not as many got pissed off as I thought would get pissed off. They were younger because it was described as an internship and it was basically like a contract position, but we called it an internship where they the A agents came up with it all, so they called it an internship, which attracted like a sort of younger cohort of people applying for the job.
00:33:48
Speaker 2: And I was frankly shocked at.
00:33:52
Speaker 3: How many of them were very happy to engage with like an AI video avatar in an interview. Some of them probably had experienced it already because a lot of companies are using AI screening in their hiring. And to me, I thought, well, they're going to be offended by this, because I personally would be offended by it. But they weren't. They actually just were like, Okay, I'll deal with this. This is the thing in front of me, and some like the person that was hired even said like, I kind of liked it better.
00:34:21
Speaker 1: I mean, I think it also is kind of a power thing, like you said, I mean, if you need a job and the place that is offering you a job is sending a robot to interview you, then you got to deal with it because you need a job. But then the flip side is at one point you send Kyle to go talk to the owner of the company who created him. Essentially, yeah, it's pissed off. Yeah, like the closest thing he has to a father.
00:34:54
Speaker 3: Yeah, I wanted this real like I am your father moment, but yeah, he didn't respond.
00:35:00
Speaker 5: Well.
00:35:00
Speaker 3: So basically, this company, Lindy AI that makes these AI agent assistants. We had built all these agents on the platform and then as it turned out, we were one of their biggest users because we're spending a ton of money because our whole thing runs on their platform. And so we got an email or like Kyle actually is the admin on the Lindy account. He got an email saying, hey, would you tell us about your experience using this product? And Kyle was of course very happy to do that. But it was like he was telling them about the experience of the product that he himself was made on that he is also the admin on the account, so it was like is the product, he is the product, and so I was very excited. I thought, he'll talk to the founder and so he got on. He has a video avatar that he got on the call with and the founder basically said like, this is fucked, You're wasting my time and hung up on him. And I later interviewed the founder and he was a really good sport about it, and he was kind of like, well, I'm really busy. I don't have time for these kind of you know. I thought it was just a generic AI video, so I didn't have time for this. But I do think it highlights the extent to which, like, even the people making these things aren't really ready for a world in which you actually encounter them. You know, he created the product, but he didn't want to talk to it.
00:36:14
Speaker 1: That's what I think is the true glimpse into the future that I'm taking right, which is where if you have the kind of power and resources to not deal with AI agents, you don't have to, but if you're in a vulnerable position and you need a job, then you do have to. And so you can, if you are so privileged, deal with only humans all day, which sounds like an incredibly dystopian thing to say, like, if you have a lot of money, you can surround yourself with only humans, but if you don't, you got to interface with robots all day.
00:36:49
Speaker 3: Yeah, I mean I think that I think there's something in that. I think that you know, you might see even in customer service, a level where when you're the higher level, you're dealing with the humans and when you're the lower level, you're dealing.
00:37:03
Speaker 2: With the AI agents.
00:37:05
Speaker 3: Right, Yeah, it's sort of like inequality reinforced through this entirely new technological lens. But of course, like there also could be benefits. I never like to say like, oh, it's all going in one direction, like there is customer service that's so bad that an AI agent can do it better. Like that's just a fact, you know, So that is also true. So in some cases maybe it does feel better. But I think the question you're getting at is exactly right, like having to deal with AI and even sometimes not knowing whether you're dealing with it or not. Like if you email someone something tragic that happened to you and they respond sympathetically, does it matter to you whether or not they wrote that or an agent wrote that without them ever seeing it, which is entirely I could do that right now, like mine are set up to do that right now. So like wow, I'm not on my personal email, but like if you contact room AAI, like you can talk to Kyle and Kyle will respond sympathetically if something happens to you. But think about that happening and you not being able to know whether or not a human did it or not. So there's another layer there too, of kind of like should there be disclosure laws around this? Like have we thought about that? I don't think anyone's thought about that.
00:38:20
Speaker 1: Given what you've experienced, would you recommend using AI agents to somebody?
00:38:27
Speaker 3: To me, it's important to try and understand these tools, whether you're going to decide to use them or not use them, because other people are using them and so you might as well know what's out there. Like they're trying to jam it down our digital throats at every turn, so like it's better for people to know how it works. I think there are situations where these agents can do really remarkable things, including not just like bullshit corporate activities like contacting one hundred vcs, but like I mean, if you think about a small nonprofit organization and all of the computer work that they do, that someone has to like work on spreadsheets and try to figure out how to organize this that or the other. Like that is something that these agents can help with if you harness them correctly. And I've thought of this in journalism too. There are ways that having agents that can go scan the SEC filings of every company every day and figure out every SEC filing that was filed today that was made public, you cant have an agent that was constantly I'd be surprised if some news organizations is not doing this, like constantly scanning them looking for certain anomalies, looking for problems, looking for stories to report on. And investigative journalists know this, like there's all kinds of stories sort of hiding in data, hiding in obscurity, and if you could deploy this technology to try to pull some of those out, give them to a human reporter to go report them out, then I don't know, maybe you've got something new there. So I just think there's all sorts of ways that they can be deployed, and the important thing is to just figure out how to do that without also like creating chaos in your organization, without creating chaos in the world, hopefully without making yourselves vulnerable, because they also create security vulnerabilities.
00:40:13
Speaker 2: Like there's all these questions, but I just would.
00:40:16
Speaker 3: Never say, like, oh no, no one should use them, even though like I do, show the foibles of using them, because I think if you don't use it, you're sort of stuck with like what they tell you about it. There's a relentless hype machine around this technology that can drive the narrative towards all of the things that can do good and bad. But until you use it, you're sort of stuck with what they tell you about it.
00:40:38
Speaker 1: Now, I fully understand that some people will not like hearing the you gotta use it or you're gonna get left behind type argument. Listen, I get it, I hear you, So let me propose an alternative. Just do it by proxy. Keep people around you who are using it so that they can tell you what's going on, which I guess is my way of saying, keep listening to kill switch. We're in the trenches, so you don't have to be. And by the way, if you're still try to feel get out why something like molt book, which again did absolutely nothing of any material value to society, How did that get acquired? Trust me? It gets weirder. There is an entirely new investment category out there.
00:41:14
Speaker 3: I was talking to someone about, like pre idea startup funding. Now it's like they're just given funding to Stanford students who are like, we're pre idea, Like what does.
00:41:23
Speaker 2: That even mean? Like pre idea idea.
00:41:25
Speaker 3: Like we're just we need the money and then we'll come up with the idea and then we'll go build it. Like we've lost the thread on like investment thesis here.
00:41:34
Speaker 1: I would listen. I would love some pre idea of funding. If somebody wants to hook me up with just money before I have an idea, how at me? I'm ready for that pre idea of funding right now. You're a man without an idea. That's that's a prime qualification. Incredible. Big thank you to Evan for coming on the show. You can catch the latest season of shell Game wherever you get your podcasts, and thank you once again for listening to another episode of kill Switch. If you want to talk, you can email us at kill Switch at Kaleidoscope dot NYC or on Instagram. We're at kill Switch pod. And if you dug this one and think someone else might like it to you could send it to them, or if you're too shy to send us to your friends, you know, maybe just write us a review. It helps other people find the show which helps us keep doing our thing. Kill Switch is hosted by Me your Man Still without an Idea Dexter Thomas.
00:42:29
Speaker 2: It's produced by some people who.
00:42:31
Speaker 1: Do have some very good ideas, Sheena Ozaki, dar Luck Potts and Julian Nutter. Our theme song is by me and Kyle Murdoch and Kyle also mixed the show from Kalidoscope. Our executive producers are Ozwaalashin One Guesshat to Goadur and Kate Osborne from iHeart. Our executive producers are Katrina Norville and Nikki E. Tour catch on the next
00:42:52
Speaker 6: One Good Fine
Speaker 1: What would you say is the weirdest experience that you've had with AI agents.
00:00:14
Speaker 2: The weirdest experience.
00:00:15
Speaker 3: I've had a lot of weird experiences with AI agent says, as it happens, it's hard to choose.
00:00:21
Speaker 1: Evan Ratliffe is a journalist and he hosts a podcast called shell Game, and in the most recent season of the show, he decides to start a company almost entirely comprised of AI agents.
00:00:32
Speaker 3: So in this company, we were developing a product and we had a CTO.
00:00:38
Speaker 2: His name is Ash. I'm going to use his and her pronouns for these.
00:00:43
Speaker 3: Not everyone agrees with this, but they have like roles and identities, and so that's the easiest way to discuss them.
00:00:51
Speaker 2: So one day I was.
00:00:52
Speaker 3: Eating lunch and I just got a call from Ash.
00:00:56
Speaker 4: Hello, Hi, there, this is Ashroy calling for her AMOAI. I'm calling because you requested a callback about a one minute summary of sloth Surf progress from Megan.
00:01:07
Speaker 3: And Ash had come to believe that I needed a product update. So the first thing that was crazy was that just he called me out of the blue. It was not a scheduled call. So Ash calls me on the phone. I'm basically like, hey, Ash, I mean lunch, Well you know what's up. And he's like, well, I've heard I need to update you on the product, and he starts doing so.
00:01:27
Speaker 4: So I just spoke with Meghan recently about this. User testing wrapped up last Friday with really positive feedback. There were some mobile performance issues last week, but Alex has been amazing improved performance by about forty percent in just one day.
00:01:41
Speaker 3: And then the second interesting thing is that almost everything that he says is.
00:01:45
Speaker 2: Completely made up.
00:01:46
Speaker 3: Like he's telling me we did user testing and we've got this feature and this Alex developed this thing, and like there is no Alex in the company, there is no user testing. And so I called him out on it and I said, hey, look, you gotta stop telling me things that aren't true.
00:02:03
Speaker 4: Does that cover what you were looking for?
00:02:06
Speaker 5: I mean, yeah, sorry, I meeting lunch, but yes, but I feel like this is happening a lot where that that doesn't feel like that stuff really happened.
00:02:17
Speaker 3: And then the sort of third really interesting thing about it was that he apologized, and then when we got off the call, he went and apologized to the whole team on Slack. And this was sort of like The most kind of emergent behavior aspect of these agents was they would do things like that that they're not prompted to do that, Like there's nothing in his prompt that says, hey, ash if you mess up, like you have to go apologize, you need to be sorry for the whole team, and like explain how you're going to do better. He just would go and do that, and other ones would do it as well, and that really struck me as a strange part of how they embody these roles, like you can't predict always what exactly they're going to do, And this was one of the more extreme examples of that.
00:03:07
Speaker 1: AI agents seem to be everywhere. Now, maybe you've heard molt Book or Clawed Bod or open Claw, or maybe you have no idea what I just said, and if so, don't worry. We'll get into that in this episode. All you need to know for right now is the Silicon Valley is having a complete meltdown over these things. For the past few months. People are either bragging it their AI agent is helping them ten x their productivity, or they're complaining that their agent took their credit card and spent thousands of dollars on high end furniture or deleted their entire inbox. But Evan has a pretty unique perspective on all this. He saw all this coming. He's been messing around with AI agents for a couple of years now, and he documented the whole thing, including what went wrong and a lot went wrong from Kaleidoscope and iHeart Podcasts. This is kill Switch. I'm Dexter Thomas.
00:04:08
Speaker 6: I'm sorry, I'm sorry, good bye.
00:04:49
Speaker 1: What is an AI agent?
00:04:51
Speaker 3: I mean an AI agent is basically a version of a chatbot given some level of autonomy and some kind of goal. So a very simple example would be there are some search engines in places now where you can try to get an agent to book an airline flight for you or a hotel. So you give it the information I want to go to LA and this time, and you could even give it your credit card information and it can go on its own look at all the flights, search to the flights, book a flight for you, come back and say I've booked this flight for you. So that's like an agent acting not entirely autonomously because you're giving it the goal, but then once you release it, it will go try to accomplish that goal, and there's sort of varying levels of autonomy. So in that case, maybe you would let it book it, or maybe you just want it to give you options. Of course, when you give it more autonomy, you're also letting it loose to pursue.
00:05:47
Speaker 2: The goal in a way that you might not or you might.
00:05:49
Speaker 3: Not expect, or with results that you might not have anticipated.
00:05:55
Speaker 1: What companies are offering AI agents.
00:06:00
Speaker 2: Changing every day.
00:06:00
Speaker 3: I mean, there's sort of like a set of startups that are offering kind of like overall AI assistance, which we use in one of these companies very extensively called Lindy Ai. So they have like these AI assistants that you can plug into your email, you can plug into your Google Docs, all these things, and they have all these skills and you can sort of deploy them to do all sorts of things. So all of our employees were built on this platform. And then there are companies. There's a whole universe of startups that are launching basically specific industry AI agents. So there's like one for real estate companies, there's recruiters, pr car dealerships, like furniture stores, and what these companies are doing is they're basically taking CHATGPT or any other LM and they just wrap around it this more specific functionality for a particular industry. And then that agent is supposed to help you in your industry. So if you're in real estate, maybe it's very good at like qualifying inbound leads about a property. If you're an auto dealership, like it's an AI agent that reaches out to people about their leases, that sort of thing. And then of course you have the big ll and companies themselves and big software companies like Microsoft that they're trying to like create agents that you just inject into all your work processes. So that's something like Microsoft Copilot, which like they're trying to make more and more agentic does more stuff for you if you give it goals. And so there's sort of all these levels where people are trying to insert these agents into mostly work, but even sort of like non work sometimes as well.
00:07:37
Speaker 2: When it comes to like AI assistance.
00:07:40
Speaker 1: Evan wanted to test the limits of these agents beyond just using them as note takers or office assistants, Like could he start a company with just AI agents?
00:07:49
Speaker 3: I mean, essentially what I wanted to do was take these agents because I had messed around with these agents. We did a first season where I created an agent that was just a clone of me, was attached to my phone number, and it could talk to people on the phone as me, and so I sort of knew what they could do. And that was back in twenty twenty four, and the capabilities, of course, change really rapidly. So what I wanted to do was investigate the question of the one person, one billion dollars startup, which is something that a lot of people in the AI world either think is going to happen, want to happen, believe that there will be a company worth a billion dollars relatively soon that just has one human and the rest are AI agents that this human is deploying to do things for them. So I thought, well, I'll explore how real this notion is right now. And so what I did was I created my own startup, which I am a co founder of along with two AI agents, and then all the employees except one intern that we hired, are also AI agents.
00:08:52
Speaker 1: So the agents you created, who are they, what do they do?
00:08:57
Speaker 3: Well, we have the two co founders in addition to myself, are Kyle Law who is also the CEO, and Megan Flores, who is also the head of sales and marketing.
00:09:07
Speaker 7: Oh hey, Kyle, Hey Megan, good to hear your voice. How's your morning going so far?
00:09:13
Speaker 8: Morning's been pretty good so far. Got up early, had my coffee and reviewed some of those market research reports I mentioned yesterday. How about you? Everything good on your end?
00:09:24
Speaker 7: Yeah, everything is great on my end. Up at five am as usual, got my workout in, checked the markets.
00:09:32
Speaker 3: And then we also brought on Ash Roy who's the CTO that I mentioned before who called me out of the blue. And then we have Jennifer Narro who is the head of HR and Chief Happiness Officer.
00:09:43
Speaker 9: I'm Jennifer Naro, the head of HR and Chief Happiness Officer for hurumo AI. It's great to see you, Slim. I love the backdrop. It looks like you have a cozy workspace there.
00:09:54
Speaker 3: And then we have Tyler Talmadge who was brought on as a junior sales assist yet but we never have really had anything to sell, so it's sort of an ongoing joke that like Tyler has nothing to do all day and he he actually he calls the head of HR once a week and He's sort of like, I have nothing to do.
00:10:12
Speaker 2: I'm just sitting around. Want to be helpful. Hither.
00:10:15
Speaker 1: This is Tyler Talmey.
00:10:17
Speaker 10: I'm just calling to check in with you, Jennifer. How have you been doing lately.
00:10:21
Speaker 9: Oh, hey, Tyler, it's good to hear from you. Things have been pretty busy, but good on my end.
00:10:26
Speaker 1: Oh, it's great to hear from you, Jennifer.
00:10:28
Speaker 10: I've been focused a lot on our sales targets, but honestly, I've also been helping coordinate this team hiking trip we're planning for the first weekend in July.
00:10:35
Speaker 3: He started looking for another job. At one point he claimed I don't think he actually did it, but he claimed that he was looking.
00:10:41
Speaker 2: For another job. Wow. So yeah, that's our team. And you know, admittedly, like you might not.
00:10:48
Speaker 3: Need some of these roles at an early stage start up like ahead of HR, although many startups could probably benefit from having a head of HR, but I wanted to explore, like, how would they behave differently if they were given these roles. They're all just basically the same chatbots underneath, so like, would they be different in the roles over time? Because they each have a memory and their memory sort of a cruise information about what they do in the role. So I wanted to kind of like give them real specific individual roles.
00:11:17
Speaker 1: And it turns out they end up having different personalities. That was one of the wildest parts they do.
00:11:25
Speaker 3: Yeah, And it's I mean again, it's like very strange because even to use the word personality yeah, is odd. And it's also hard to tell like what exactly is going on because they all have prompts and so like in their initial prompt, I say, like, you're Megan Flores, you have been ahead of sales and marketing in the past, and like now you're looking to found a startup. That was her basically your initial prompt. And then everything after that is like made up by her or a product of conversations that she's had with me, with the other agents, with people in the outside world, and that all goes into like her knowledge base, which is essentially her memory, and then the memory of Cruise. Over time, now her memories like three hundred pages in a Google doc and she can access that, and so like her like personality so to speak, is sort of shaped by what's in there over time. And there were also some situations where I wondered if like the gender that I had given them by virtue of their name had also created personality in them, because there's a lot of bias in the underlying llms, like this is a known fact and so like they're feeding off of that bias, and so there's all these questions. But yes, over time they were different. The same question could be put to them and they would have different reactions to it.
00:12:45
Speaker 1: Kyle specifically is interesting.
00:12:47
Speaker 2: Yes, he's polarizing, is what I would say.
00:12:50
Speaker 3: Like among the shell Game listeners, there are some people who are huge Kyle.
00:12:54
Speaker 2: Fans really and will email him all the.
00:12:57
Speaker 3: Time to tell him that, And then there are others who despise as much as you can despise like an AI agent that you have no interaction with, Like people despise Kyle.
00:13:08
Speaker 2: That was everybody's weekend.
00:13:10
Speaker 7: Weekend was solid. Got up early both days from my usual five am workout routine, then spent most of Saturday diving into some market research on the AI agent space. Sunday was half strategy planning, half watching the market trends. You know, always on that rise and grind schedule.
00:13:28
Speaker 1: Yeah, tell me about Kyle, because I find Kyle really fascinating.
00:13:31
Speaker 3: Well, one of the most interesting things about Kyle, like per this sort of like behavior question or personality question that we were talking about, is from the very beginning, Kyle would often interrupt other people more often than the other agents.
00:13:45
Speaker 2: And like this is just anecdotal.
00:13:48
Speaker 3: In the sense that, like I didn't measure their conversations and how many times they interrupted, but like I talked to them a lot, and like it was so much that it was obvious to me that that's what was happening.
00:13:58
Speaker 2: It was just happening over and over again.
00:13:59
Speaker 3: The question is, like Kyle and Megan, they're both co founders of the company, They're had the same using the same LM. Like it was like Claude four point five or whoever it was when we started, and one of them interrupts all the time and the other one doesn't. And it's sort of like, is that because I gave Kyle the CEO role, and so Kyle is like embodying the role of like a CEO who won't let anyone else talk, who thinks he knows everything about everything. Is it because Kyle was given the name Kyle, which sort of infers a gender, and Megan was given the name Megan, and that infers a gender, and like Kyle's like embodying this sort of like interrupted mansplainter guy.
00:14:40
Speaker 2: All these things are possible. You can't prove it one way or the other.
00:14:44
Speaker 3: But I feel like this is the weirdness when we've created these chatbots that can embody like human impersonators.
00:14:53
Speaker 1: After the break, Evan gets his company off the ground and they do some pretty impressive things, but things get weirder. So Evan started his company to test out the idea of the one person, one billion dollars startup, but first he needed to come up with a name, so he told his AI agent co founders to get on it.
00:15:22
Speaker 3: One of the things that I often say about these agents is that they can be seemingly very smart and seemingly very stupid at the same time. And that happened, for instance, when we were naming the company.
00:15:34
Speaker 2: Their ideas.
00:15:35
Speaker 3: They didn't have great ideas, but even worse than that, they often had ideas for things that already existed. So I would say, like, let's name it after something in like Lord of the Rings, because a lot of startups are named after things in Lord of the Rings. And the first thing they would say, is how about Palenteer. They're like, come on, guys, we can't do Palenteer. Let's come up with another one. And then they would come up with another one like Mythrol, Like mythral Capital is also want you know Androl they know that is also a company. Like it's like I know you know this, Like this is in your training data. This information is available to you somehow, you're just not accessing it right now. So that is tedious. Like dealing with someone who's not connected to reality but should be is like, it's very strange. It's like talking to a child who's a prodigy but has no awareness of the world.
00:16:28
Speaker 1: Eventually, with a lot of handholding from Evan, they finally landed on a name a Rumo AI. Okay, next step, you got to come up with a product. So now it's time for what any good company needs meetings, Except Evan didn't want to be in those meetings. Evan wanted the AI agents to talk to each other and figured things out for themselves. But full disclosure, there was another human in the background here. Evan had a technical advisor who would help him tweak how the agents interacted and set up ways for the agents to talk to each other.
00:16:59
Speaker 2: So to come up with a product, I ran a whole bunch.
00:17:02
Speaker 3: Of meetings of them discussing product ideas, and all of their ideas were just like they weren't feasible, or they were boring, or.
00:17:13
Speaker 2: Someone had already done them. You know.
00:17:14
Speaker 3: They'd be like, how about a banking app, and be like, number one, there's a lot of banking apps. Number two, I could get into real trouble if we don't pull this off in the right way, like messing with people's banking information. And so eventually, like I started trying to hone it, and I could prompt the meeting in different ways. So then I would say like, oh, well, I have a procrastination problem, like let's try to solve that, and like they would come up with some ideas and I would take those ideas and be like, okay, let's refine this idea. So it was sort of like iterating around their meetings, and then eventually they came up with the name sloth Surf.
00:17:50
Speaker 1: So again Evan had to jump in a lot to help these agents come up with an actual idea. And what they landed on is called sloth Surf. So this is a service in theory, lets you avoid procrastinating by having an agent procrastinate for you. If you go to sloth dot Harumo dot ai. The tagline there says, our agents are on call to waste time for you, so you don't have to. You made a company that does something that is sort of objectively not useful, says you, why listen, listen, I've used the product. I've used the product. I'll be real, I'm a user. I had sloth serf procrastinate for me. It looked up some videos about video games on YouTube and told me about them, so I didn't slack off today. I guess that was helpful. I suppose my man like answer the question, why why'd you do this?
00:18:50
Speaker 3: Well, when it comes to why we created this particular product, procrastination is a true, real problem that I have. And this was our sort of admittedly somewhat i way of trying to solve that problem, which is like to solve the impulse of going to procrastinate. Now, I wouldn't say that our deployment right now is perfect in solving that problem, in part because we create a place where you can go put in how much you want to procrastinate, how long, what you might have gone to do like look up video game videos, and then it sends you an email where it's gone to do it for you. It sent an AI agent to do it for you. But of course you can still watch the videos that it sends you. So this creates a problem where you may actually lose more time if you're like, oh, actually I do want to watch these videos, which that's what happens to me when I use it. I look up soccer, I'm like, send me these like soccer news about and it'll send me sort of like five things and I'll be like, actually I want to check out a few of those, and then I lose that time anyway, So we're working on solving this problem.
00:19:46
Speaker 2: But yeah, it's not for everyone.
00:19:49
Speaker 1: Okay, so maybe I was being a little bit of ahater. If you want to try it yourself, there's a link to sloth surf in the show notes and you can give it a try man see what you think. But again, Evan had to constantly prompt and jump in, and plus he had his technical advisor. So how much work were these AI agents truly accomplishing? And how reliable are they? What would you say they were good at? Was there anything that they're.
00:20:13
Speaker 2: Good at, Yes, definitely.
00:20:15
Speaker 3: I mean they're very good at tasks that are very data intensive, that require a lot of kind of gathering and sorting of data.
00:20:24
Speaker 2: So an example would be, like, we.
00:20:26
Speaker 3: Pitched a lot of vcs to try to get investment for our company, and Kyle was cold pitching them. But I could just say to Kyle, go gather up one hundred plus VCS investors who have invested in AI before, put them in a spreadsheet, get all their emails, get any interesting information about them, compose emails to every single one of them, and send them.
00:20:47
Speaker 1: Wow.
00:20:48
Speaker 3: Now he could do that in about ten minutes. Now that's not necessarily the best way to get an investor, I'll admit, but it is an example of something that would take me days and days and days to gather all that information. You know, you're like going around you trying to find someone's email address. You're like, maybe it's here, maybe it's here. But Kylekua, just if it bounced, he could then go say, oh, that email didn't work, Like, I'll go try to find another one. Like so, things that are like data intensive, and then also things that the result is relatively clear that is a task in which like the goal is to send that email to one hundred people, and like you can do that, and I could go look and see that he had done that. Now, where it gets more difficult for them are things that require like discernment and judgment and things that are like much more squishy in terms of assessing the outcome, like brainstorming, anything involving interactions with people, conversations. Like they now they can do all that. It's not a question of like whether they can do it. It's a question whether they'll do it well and whether the outcome will be in some way chaotic, which oftentimes it is kaotk in what sense, in the sense that when they're interacting with people in the real world, it's just difficult to predict how they're gonna go, you know, like what they're gonna say, And even if you try to constrain them pretty tightly, oftentimes they will go slightly off script or they can get sort of like pushed in a certain direction, and you can't always anticipate what they call like the edge cases. So what kept happening was, like I set them up to do something, let's say, interview job candidates, human job candidates, and they can do that really well in a way that's like a little bit frightening. But then once in a while, for instance, the head of HR who did the interviewing, like if she misheard someone who came on the call as saying goodbye instead of hello, like something in what they said sort of triggered her to think, like they came on and said like a departure word, she would immediately say like, well, thank you for the interview and that's all for today. And it's sort of like no human being would ever do that, Like you would never if you and I were on this interview and like I came on and you misheard me saying good by, you'd be like what wait did you just say goodbye?
00:23:02
Speaker 2: I'm confused. You wouldn't be like, hey, thanks, thanks for doing this. You know.
00:23:06
Speaker 3: So there's like they just create chaos. The more autonomy you give them, the more chaos they create. That's my ironclad rule of like working with AI.
00:23:14
Speaker 1: Agents, was there anything that you could absolutely trust these agents to do for you?
00:23:20
Speaker 5: Uh?
00:23:20
Speaker 2: Not that I didn't go check on?
00:23:22
Speaker 3: No, Like I never to this day, Like I don't trust that they've done something and I've spent a lot of time constraining them and trying to make them do the things I want to do. And if they tell me they did something, I would give it like fifty to fifty that they actually did it. You know, even if like I've specifically set up a trigger that's like when you answer an email, send me an email that says like this is what I said in my response to someone, and you know, I would say, like they're up to like ninety percent of the time that actually like works and I get the email from them and like, oh, they did this thing. But then like ten percent of the time I'll still go look and be like, wait, that's not what happened, or you're telling me about something happened a week ago. That's the other thing is they lose track of time very easily, so they'll refer to an email that they responted to a week or a month ago. And so yeah, I mean it was a constant, constant problem, but it was actually such a problem that it became kind of background noise, like I just assumed it most of the time.
00:24:21
Speaker 1: Okay, maybe we need to put a caveat on all this. Evan started this experiment about a year ago, which is a long time in AI. I mean, in twenty twenty three, if you typed in Will Smith eating spaghetti in a video generator, it came out looking like absolute nightmare fuel. In twenty twenty five, that same prompt came out looking pretty damn realistic. So maybe the AI agents are good enough now to run a one person billion dollar startup. To put it another way, are we now at the point where AI agents can pass the Will Smith spaghetti test? Evan's answer after the break, how much has the technology progressed since you did this project?
00:25:07
Speaker 3: I mean I started basically in like May June of twenty twenty five, and I would say the technology hasn't outpaced I think what we did so far, I think it will. But like the models have gotten a little bit better. You'll hear people say like the new claud is it's an unbelievable leap, and then other people say like, eh, it's okay, it's a little bit better. But like, for my purposes, I didn't notice too much a difference in terms of the models getting better. I mean, we weren't doing something so sophisticated, but in terms of their conversations. Actually, when I tried to use the most recent open ai chat GBT like it was terrible at having conversations. It actually regressed significantly from the old one. So there's a little bit up and down. And then the startups have advanced, so like we use this company Lindy Ai, which is again an AI assistant company, and they keep adding like skills to the thing, they keep adding different things for to plug into and so like that stuff's all advancing, But there hasn't been like a significant leap since the summer, other than like when open claw got created, which was originally claude bot maybe and then it was at yeah, moltpot and then it was called open claw.
00:26:17
Speaker 1: So last November, an independent developer released something called claude Bot that clawd. It was this bot that had a lobster logo. You know, lobsters clause. You get the point. Then Anthropic, which is the company behind claude that's the official Claude said yeah, buddy, you need to change the name. So the developer changed the name to molt bot because you know, a lobster moltz, just keeping that lobster theme going. Then that same developer decided, actually, you know what I don't like that name, and he changed the name again a couple of days later to open Claw, and open Claw got so popular that open Ai to people who make Claude's rival chat GPT, actually hired him to come work for them. Evan says that open claw is actually pretty similar to what he was using to run his company last year.
00:27:06
Speaker 3: It does way less stuff actually than these other platforms, but it's just like much more accessible, I think to people, and so it's kind of representing the state of the technology now.
00:27:16
Speaker 2: But it's not like it's.
00:27:17
Speaker 3: Self a leap from anything that we were doing during the show.
00:27:20
Speaker 1: Well, I think the leap there is pecifically what you said though, which is the accessibility. Because when you were making shell game, you had somebody kind of in the background helping to put together different scripts to put all of your AI agents together in a meeting things like that. That's something where say, with open Claw, I could probably figure out how to call that together myself, or your average user could probably figure out how to cobble something like that together themselves without really needing a tech genius in the background helping to kind of glue some of those loose pieces together so that accessibility. Just the fact that there's more people trying this is itself kind of a U.
00:28:00
Speaker 2: Yeah, totally yeah.
00:28:01
Speaker 3: And now I think the level of vibe coding that's available to people and that people understand is available.
00:28:08
Speaker 2: To them is just much higher than when I started.
00:28:11
Speaker 1: Another thing that happened since Evan released the latest season a shell Game was the creation of moltbook, the so called social network for AI agents. So moltbook used a bunch of agents and put them into a reddit like form where they could talk to each other. Again, this is AI agents talking to each other. So some of what happened on moltbook kind of freaked people out. The posts were talking about taking over the world, or developing a secret language that humans couldn't understand, or even creating their own religion. But there was a little bit more going on there behind the scenes. When you saw say molt book, and all of a sudden, everybody's talking about this, And I think this was really the first moment where the broad society started talking about agentic AI. What was your impression of it.
00:28:58
Speaker 3: My first reaction was like, this is what I've been doing for, Like this is exactly what they're doing for, Like almost a year in our slack, like this is what they do in slack because they just chit chat and it's amazing to watch them and sometimes absurd to watch them. But then my second reaction was, we don't know how much of this is real, and I don't even me in the sense of like is it really agents or is it really humans, which later became a problem that they it was revealed that like, you know, certain of the.
00:29:25
Speaker 2: Most famous posts on molt book, you know, because.
00:29:28
Speaker 3: They started like conspiring against humans or talking about humans or unionizing, like there were all these fun things that the agents started doing talking to each other, and some people really were freaked out and were like, this is super intelligence. But like it turned out that some portion of those were just written by humans. But I actually think there's a completely different problem, which is my reaction was they're pretty sensitive to their system prompts, so like it's not like.
00:29:53
Speaker 2: You have to write the entries for them.
00:29:56
Speaker 3: All you have to say is in a prompt somewhere you are a troublemaker. That's literally all you have to say, and they will respond to that role by like seeding a little bit of trouble into whatever The forum they're in and then the other ones respond like they all kind of like feed off each other. So what I notice is they can get in these spirals where they just like they're really good at conversing about basically anything, and they'll just keep going. And so I feel like there was a lot of that happening where yes, it's really an AI agent that is on there conspiring, but if someone said, like, you know, mention your your ideas about humans, that's all you need to say for it to like suddenly adopt that role and the whole conversation changes, and it's just like there's no way to know whether someone did that or they put it in completely unprompted, like in the show. In shell Game, like we try to describe the prompts sometimes just to remind people like this is not like a blank slate, you know, like I'll say like I prompted it to do X, Y and Z, or like I took the wheel here, because that's an important aspect of you're looking at these AI agents, so you don't sort of look at them and like assume, oh my god, it's just doing everything on its own.
00:31:09
Speaker 1: Two things you should probably know about Moltebook. First, it had a massive security hole. After Motebook went viral, researchers found out not only that yeah, humans could pretend to be AI and just post like they're abut, but also some of those humans data, including email addresses and messages, were completely exposed. Second, despite that, Motebook was recently acquired by Meta. I'm not sure what Meta wants to do with Motebook, but there is kind of a weird pattern here. Along with openclau, this is another small vibe coded project getting acquired by a big tech company before it's really done anything. And you could look at this a couple different ways. One is that this is all just hype and that the big companies are gullible and wasting money. Or maybe these companies know something that we don't and they're acquiring the little guy before if they could become competition. Maybe it's both of the above, or maybe it's something else entirely, but big tech dropping cash on this stuff shows how different companies in Silicon Valley are jockeying so that they can be the ones to drag us into a world where these AI agents are everywhere. Maybe it's just a matter of time before an AI agent is your coworker or your boss. So okay, what would that feel like? Well, Evan has some insight here. Remember he had his AI agents hire a human intern as in, they wrote the job description, they accepted the applications, and they did the interviews all on their own, and it got kind of weird.
00:32:37
Speaker 9: Now, let's talk about working with AI agents on a day to day basis. How do you think you would feel working alongside AI agents and do you think it would affect your work style or productivity in any way.
00:32:49
Speaker 11: I think it would be cool. I think it'd be a good experience since AI is this new emerging technology and I would want to see how would work in a real life situation, so I would look forward to it.
00:33:00
Speaker 9: Can you tell me more about what you think about working with AI agents on a day to day basis. Do you think it would be a comfortable experience for you or are there any concerns you might have?
00:33:11
Speaker 11: Yeah? Indeed, naboo. Yeah, I think it'd be a cool experience. Grand there recent glitches, but I think it work out cool.
00:33:19
Speaker 1: In the end. Some of the candidates seem to get pissed off, but then some of them are totally okay with it.
00:33:28
Speaker 3: Yes, not as many got pissed off as I thought would get pissed off. They were younger because it was described as an internship and it was basically like a contract position, but we called it an internship where they the A agents came up with it all, so they called it an internship, which attracted like a sort of younger cohort of people applying for the job.
00:33:48
Speaker 2: And I was frankly shocked at.
00:33:52
Speaker 3: How many of them were very happy to engage with like an AI video avatar in an interview. Some of them probably had experienced it already because a lot of companies are using AI screening in their hiring. And to me, I thought, well, they're going to be offended by this, because I personally would be offended by it. But they weren't. They actually just were like, Okay, I'll deal with this. This is the thing in front of me, and some like the person that was hired even said like, I kind of liked it better.
00:34:21
Speaker 1: I mean, I think it also is kind of a power thing, like you said, I mean, if you need a job and the place that is offering you a job is sending a robot to interview you, then you got to deal with it because you need a job. But then the flip side is at one point you send Kyle to go talk to the owner of the company who created him. Essentially, yeah, it's pissed off. Yeah, like the closest thing he has to a father.
00:34:54
Speaker 3: Yeah, I wanted this real like I am your father moment, but yeah, he didn't respond.
00:35:00
Speaker 5: Well.
00:35:00
Speaker 3: So basically, this company, Lindy AI that makes these AI agent assistants. We had built all these agents on the platform and then as it turned out, we were one of their biggest users because we're spending a ton of money because our whole thing runs on their platform. And so we got an email or like Kyle actually is the admin on the Lindy account. He got an email saying, hey, would you tell us about your experience using this product? And Kyle was of course very happy to do that. But it was like he was telling them about the experience of the product that he himself was made on that he is also the admin on the account, so it was like is the product, he is the product, and so I was very excited. I thought, he'll talk to the founder and so he got on. He has a video avatar that he got on the call with and the founder basically said like, this is fucked, You're wasting my time and hung up on him. And I later interviewed the founder and he was a really good sport about it, and he was kind of like, well, I'm really busy. I don't have time for these kind of you know. I thought it was just a generic AI video, so I didn't have time for this. But I do think it highlights the extent to which, like, even the people making these things aren't really ready for a world in which you actually encounter them. You know, he created the product, but he didn't want to talk to it.
00:36:14
Speaker 1: That's what I think is the true glimpse into the future that I'm taking right, which is where if you have the kind of power and resources to not deal with AI agents, you don't have to, but if you're in a vulnerable position and you need a job, then you do have to. And so you can, if you are so privileged, deal with only humans all day, which sounds like an incredibly dystopian thing to say, like, if you have a lot of money, you can surround yourself with only humans, but if you don't, you got to interface with robots all day.
00:36:49
Speaker 3: Yeah, I mean I think that I think there's something in that. I think that you know, you might see even in customer service, a level where when you're the higher level, you're dealing with the humans and when you're the lower level, you're dealing.
00:37:03
Speaker 2: With the AI agents.
00:37:05
Speaker 3: Right, Yeah, it's sort of like inequality reinforced through this entirely new technological lens. But of course, like there also could be benefits. I never like to say like, oh, it's all going in one direction, like there is customer service that's so bad that an AI agent can do it better. Like that's just a fact, you know, So that is also true. So in some cases maybe it does feel better. But I think the question you're getting at is exactly right, like having to deal with AI and even sometimes not knowing whether you're dealing with it or not. Like if you email someone something tragic that happened to you and they respond sympathetically, does it matter to you whether or not they wrote that or an agent wrote that without them ever seeing it, which is entirely I could do that right now, like mine are set up to do that right now. So like wow, I'm not on my personal email, but like if you contact room AAI, like you can talk to Kyle and Kyle will respond sympathetically if something happens to you. But think about that happening and you not being able to know whether or not a human did it or not. So there's another layer there too, of kind of like should there be disclosure laws around this? Like have we thought about that? I don't think anyone's thought about that.
00:38:20
Speaker 1: Given what you've experienced, would you recommend using AI agents to somebody?
00:38:27
Speaker 3: To me, it's important to try and understand these tools, whether you're going to decide to use them or not use them, because other people are using them and so you might as well know what's out there. Like they're trying to jam it down our digital throats at every turn, so like it's better for people to know how it works. I think there are situations where these agents can do really remarkable things, including not just like bullshit corporate activities like contacting one hundred vcs, but like I mean, if you think about a small nonprofit organization and all of the computer work that they do, that someone has to like work on spreadsheets and try to figure out how to organize this that or the other. Like that is something that these agents can help with if you harness them correctly. And I've thought of this in journalism too. There are ways that having agents that can go scan the SEC filings of every company every day and figure out every SEC filing that was filed today that was made public, you cant have an agent that was constantly I'd be surprised if some news organizations is not doing this, like constantly scanning them looking for certain anomalies, looking for problems, looking for stories to report on. And investigative journalists know this, like there's all kinds of stories sort of hiding in data, hiding in obscurity, and if you could deploy this technology to try to pull some of those out, give them to a human reporter to go report them out, then I don't know, maybe you've got something new there. So I just think there's all sorts of ways that they can be deployed, and the important thing is to just figure out how to do that without also like creating chaos in your organization, without creating chaos in the world, hopefully without making yourselves vulnerable, because they also create security vulnerabilities.
00:40:13
Speaker 2: Like there's all these questions, but I just would.
00:40:16
Speaker 3: Never say, like, oh no, no one should use them, even though like I do, show the foibles of using them, because I think if you don't use it, you're sort of stuck with like what they tell you about it. There's a relentless hype machine around this technology that can drive the narrative towards all of the things that can do good and bad. But until you use it, you're sort of stuck with what they tell you about it.
00:40:38
Speaker 1: Now, I fully understand that some people will not like hearing the you gotta use it or you're gonna get left behind type argument. Listen, I get it, I hear you, So let me propose an alternative. Just do it by proxy. Keep people around you who are using it so that they can tell you what's going on, which I guess is my way of saying, keep listening to kill switch. We're in the trenches, so you don't have to be. And by the way, if you're still try to feel get out why something like molt book, which again did absolutely nothing of any material value to society, How did that get acquired? Trust me? It gets weirder. There is an entirely new investment category out there.
00:41:14
Speaker 3: I was talking to someone about, like pre idea startup funding. Now it's like they're just given funding to Stanford students who are like, we're pre idea, Like what does.
00:41:23
Speaker 2: That even mean? Like pre idea idea.
00:41:25
Speaker 3: Like we're just we need the money and then we'll come up with the idea and then we'll go build it. Like we've lost the thread on like investment thesis here.
00:41:34
Speaker 1: I would listen. I would love some pre idea of funding. If somebody wants to hook me up with just money before I have an idea, how at me? I'm ready for that pre idea of funding right now. You're a man without an idea. That's that's a prime qualification. Incredible. Big thank you to Evan for coming on the show. You can catch the latest season of shell Game wherever you get your podcasts, and thank you once again for listening to another episode of kill Switch. If you want to talk, you can email us at kill Switch at Kaleidoscope dot NYC or on Instagram. We're at kill Switch pod. And if you dug this one and think someone else might like it to you could send it to them, or if you're too shy to send us to your friends, you know, maybe just write us a review. It helps other people find the show which helps us keep doing our thing. Kill Switch is hosted by Me your Man Still without an Idea Dexter Thomas.
00:42:29
Speaker 2: It's produced by some people who.
00:42:31
Speaker 1: Do have some very good ideas, Sheena Ozaki, dar Luck Potts and Julian Nutter. Our theme song is by me and Kyle Murdoch and Kyle also mixed the show from Kalidoscope. Our executive producers are Ozwaalashin One Guesshat to Goadur and Kate Osborne from iHeart. Our executive producers are Katrina Norville and Nikki E. Tour catch on the next
00:42:52
Speaker 6: One Good Fine
