Fuld transskription af 1. AI Coding for Teams | Sohil Shah

proud of. And that is something that is the biggest differentiator between what we could accomplish individually before versus what we can do now with the power of assistance of AI. Pete Bishop: Do you feel like that there is a way to do code and some of this AI is skipping over some of those practices? Or do you think it doesn't matter as long as you get to the right destination? Sohil Vinod Shah: So would say that it helps you get to the destination a lot faster. In terms of doing the right thing or the wrong thing, I don't think AI is there yet. You still need supervision. You still need to read the code that LLM is spitting out. And you to do a review of the code before it goes to production. So that part hasn't been eliminated yet. What has happened is we have shifted more towards code reviewers than code writers. So the writing part is a lot easier now compared to what it was back in the day. But the reviewing part is more difficult now, because there is a lot of code that is output from the LLM and ⁓ a lot of lines of code that you just have to review it. And sometimes there certain patterns that LLM outputs, can be in some sense. So you need to simplify those things or ask the LLM to simplify those functions. or those code logic, and that's the way to go for now. Pete Bishop: Yeah, I love that idea of just code reviewer. Now, I think that's where a lot of things are going, like you're an editor versus a writer or some of these things that are changing. What I wouldn't mind digging into a little bit is we talked a little bit about working as a team. And I think that's a little different for people who are to having this one-to-one relationship with AI, right? So you go in, you ask questions or you're building things, but it tends to be fairly isolated, especially when you're working with an LLM. I'm just wanting to understand how does a team evolve with AI? How does that work? ⁓ You mentioned a bit about this review process, but has that changed the of a code team? Sohil Vinod Shah: In some sense, because a lot of focus, so pre-AI world, a of the engineers would be writing code and then there would be a subset of one or two engineers on the team who would actually be contributing to or doing the expert reviews before it production. So some senior engineer maybe a staff engineer on your team would be doing most of those reviews. So that was the pre-AI world, but now moving into the post-AI world a lot. lot of the assistance in terms of the writing code is done by the LLM. And then we have every engineer on the team reviewing all the code because there's so many pull requests coming in. most of us are just reviewing each other's code most of the time. And the that we get from LLM in terms of writing the code, that helps us put a better finished product at the end of it. So the time taken to review code is much longer than what it used to be before, just because the amount of code that we are trying to push out. but the amount of time to write the code is much constrained or smaller because of the assistance that we are getting from the LLM 1.1 site. So we have two dynamics here. I wouldn't say the work is getting any easier. What I would say is the pace and the velocity at which we can produce features or push out, that velocity has sort of increased. So ⁓ we are working on a product. But what is happening is that we are building a finished product at the end of it. We are not doing a POC. We are not shipping incomplete products anymore. We are building a full set of features and then releasing those out. Pete Bishop: Interesting. Yeah, I was going to ask you, is there a scenario where AI is helping you do the code review in the future? Or do you see that happening? Or is it sooner or later a human has to be in there checking things over? Sohil Vinod Shah: It does happen. So recently, Cloud released a new feature called UltraReview. So if you use Cloud Code, you can do a slash UltraReview on the Opus 4.7. And it does some review for you the scenes. So any of those basic coding logic errors and things like that, those could be caught using those things. But I don't think it does the whole. It doesn't understand the business side of things. So it doesn't know what the context is. it takes a human mind for them to be involved in the process to detect those issues. Pete Bishop: Yeah, that makes sense to me. feel like the business requirements, understanding kind of the nuances of what you're actually trying to achieve makes a ton of sense to have people involved in that. what are you doing when you're ⁓ PayPaling? Like what are the things that, do you have any side projects or anything that you're working on? Sohil Vinod Shah: So currently, I'm heavily involved in the Agent Development Kit project. This is like an open source project from Google. It you build agentic framework. And it's essentially an agent harness that allows you to run single agent, multi agent architecture in production. And you can manage those dedication of task, and all of those things. So it's a very simplistic. framework which allows you to manage agents in production. When ⁓ take an agent that is developed locally and you deploy it in production, there's a lot of governance and aspect that needs to be added to those agents for them to run safely in production, which is currently the biggest concern in the How do you productionize these agents that you're building locally? And that's one of the things that this framework allows you to do. And that's one of the things that I'm focusing on right now in my personal life. Pete Bishop: So you're saying, I think if I understand, is rather than using some of these common ones like just Gemini and ChachiPT and what, if you're taking basically an isolated LLM that you can spin up ⁓ a local server, those kind of the open source ones that they allow, right, like you mentioned, you can kind of keep that firewall from everything else. that, is that? Sohil Vinod Shah: So

there are two things, right? One is like there are two tracks right now that are going in terms of the AI development. So one track is the vibe coding path. So you have access to an LLM and everyone who has access to the LLM tries to vibe code and figures out a website that they need to develop or some idea or some POC that they need to build. So you don't need a lot of expertise to build those things. You just need access to the LLM to build something out for you that you can show to a consumer. of the end user as a POC, but it's not usable at the end of the day. So in order to production-ize it and functionalize it, you need to build a governance framework around it. And that's where the agentic part comes in. So the agentic part helps you communicate between different processes that you have within the system, and then make sure that you have sufficient guardrails that are built out. this agent framework that we're talking about, the agent development kit, ADK allows you to build that harness, that guardrail in place so that the agents can run seamlessly in production and execute the business functions that are needed. Pete Bishop: That's amazing. Okay, so you had something you can, can you quickly show us here? Sohil Vinod Shah: Yeah, sure. So I have a Travel Concierge agent that I built locally in the last few days. ⁓ And ⁓ this is initial UI for the ⁓ Agent Development It's pretty neat. It gives you all the agents, the prompts, and the tools that were invoked during the entire process. ⁓ For the sake of this podcast, I went ahead and ⁓ inputted all the prompt and ran through the demo just to make sure that we don't, we are not spending time trying to load through the responses. But if you look over here, I have a simple prompt ⁓ where I'm trying to ask the travel concierge agent to basically plan a three-day trip to Tokyo for two people. And the budget for that trip is $3,000. Pete Bishop: And hey, I'm just going to jump in. this is probably the first time we've shared a screen. But now that most podcasts are video, we're taking full advantage. So if you're driving at the moment, make sure your eyes are glued to the screen ⁓ as you'll see this interface ⁓ pop up, which is kind of cool to see. Sorry to interrupt, but let's keep going. Sohil Vinod Shah: No worries. And then ⁓ you can see that the supervisor agent, which is the travel concierge, handles the, it hands over the reference to the preference reader. What preference reader would do is it would read all the preferences that I have as a user. So my preferences would be different from yours. Like for me, what's important to me is there should be good vegetarian options because I'm a primarily vegetarian. I have a primary vegetarian diet. And then ⁓ I would have specific accommodation requests. Maybe I need accessibility and things like that. ⁓ Or maybe I need ⁓ a room in a hotel, which is at the first floor, for easy access to go in and out. Or I need a hotel in a specific location in

a city, like maybe a downtown area, because I prefer being closer to the action compared to being in some suburb where I have to drive 30 minutes out. So those preferences are stored in a preference reader. And what it does, it looks at your previous bookings. And then based on your previous bookings, it figures out what the preferences are. And then it loads those preferences in. ⁓ After that, it invokes the researcher agent. The researcher agent is basically going to look at all the point of interest that the bloggers have posted on World Wide Web. So it would read through most of those blogs, figure out what are the important points of interest that user needs to cover during this trip in Tokyo, what are the top most things, what are the second best things, and so on and so forth, and the amount of time it would take for each one of those. Pete Bishop: Just a quick question. why do you split these up into separate agents? I think I know the answer here, but why wouldn't you have one big agent that does everything? Sohil Vinod Shah: So one reason for that is very hard to scale. ⁓ So if you have one big agent that does everything and it has access to hundreds of tools, scaling becomes an issue. And what we have often seen, or what I have often seen in my experience, is that it hallucinates a lot. The other thing is the cost also goes up exponentially, because every time you have to invoke that agent, all the tools and the context for those tools have to be loaded in the context. And for every turn that you make with the agent or every prompt that you send to the agent, all of that text has to be sent in, and then you get a response back. So by splitting the agents up, you minimize the amount of tools that you send in the context. So you are saving cost on the tool context. That's number one. And the other thing is you also delegate the task. So tomorrow you can have other sub-agents that can join this team and then basically help you execute further functions if needed. Pete Bishop: Yeah, I imagine troubleshooting is a little easier too when you know something's going wrong. Yeah, okay. Sohil Vinod Shah: That's correct, yeah. And then from the researcher agent, you get back a summarized view of all the things that you need to cover while you're on this trip. And then let's say the researcher agent sends the response back to the travel concierge. The travel concierge finally invokes the itinerary builder. So it has all the information it needs to build an itinerary for you. So what it would do is it would send you an itinerary on a day-by-day basis. And it is doing this because I have specifically asked for this format in the instructions. So I don't want a full blog of entire itinerary in one go. I want to review each day and then make any adjustments if needed based on that specific day. At 9 o'clock, you go visit some temple. ⁓ At 2.30, you go for a river walk. And then at 9 o'clock at night, you take an evening stroll through some park. And it also gives you an estimated steps and some daily estimated value of how much you can expect to spend. ⁓ So once everything looks good on the first day itinerary, ⁓ mark the identity as approved for that particular day. And what it does, it will record the approval for that day in its database. And this is an important pattern which everyone needs to learn. This is the human in the loop pattern. So what happens is every time you want to make a decision, you want a human approver to make a decision, you can pause the agent execution at a specific time. or specific point in the session. And then you're waiting for a human to basically approve it or give some input from their side. And in this case, it's an approval to proceed with the next step. And then it will proceed with the next day of the itinerary. Pete Bishop: Yeah, I love this. I find it so interesting when you're setting up multi-agents like this, how you're almost using job descriptions to kind of manage this team of agents. It starts to feel like coworkers and you have the travel concierge, which is like what a travel concierge essentially does. ⁓ I think that's really neat. I saw this plugin. was, ⁓ someone had built it in... ⁓ Claude code, but it basically turns all your agents into like a Sims simulator. So you have a little man at a desk and he walks over and you can create new sub agents and they're just people. So they're just like personifying all these agents. But when they're done their job, they, they stand up and they go to the water cooler or something, cause they have nothing to do. And then you got to assign them stuff again. And it's just a neat way to look at how these things work together. Sohil Vinod Shah: Exactly, and the word is persona. So you need to define the proper persona for each of these agents and that goes inside the instructions for each of these agents. So if you want the agents to do slightly different from what I have designed over here, then you need to specify those in the instructions. Pete Bishop: Got it, got it, okay. So the ultimate goal for this is to create a travel planner, right, from start to finish. And you said it would eventually start to be able to book in real time. It doesn't do that right now, right? Sohil Vinod Shah: Yes. Yes. for now, ⁓ basically there's a mock MCP tool that mimics the booking option. And then it will just book the tickets ⁓ everything that you need to get done for that specific trip. But assuming that you have a Expedia MCP server behind the scene or some other travel agent ⁓ that has an MCP server available for you to make instant booking, you can plug that in to your agent. And then it will do real-time bookings for you as well. Pete Bishop: That's amazing. And just for the people that may not know what MCP server is, do you mind just quickly describing that? Sohil Vinod Shah: So MCP is Model Context Protocol. ⁓ Essentially in very layman terms, currently we have API endpoints. So they have a verb to it,

getRestPost, and things like that. ⁓ MCP is more of a streaming protocol. So you can put a layer of MCP servers on top of these REST endpoints. And it's a way for agents to communicate with these endpoints. Pete Bishop: Yeah, I think ⁓ it's almost like having a window into some software so it allows it to call, like you said, it's like an API, although that doesn't help explain things to too much. It's just a way to grab, ⁓ like for instance, I imagine, does PayPal have its own MCP server that allows other things to connect to it? Right. Sohil Vinod Shah: Yes, we do. So we have an MCP server for payment. So if you want to make a payment as an example to any enterprise or any business, then you would use a PayPal MCP server to make those payments. Other alternative is the way to think about it is, let's say you have a stash of tools or toolbox. And there are a bunch of tools that are there within the toolbox. And MCP is essentially that toolbox that has a bunch of tools in it that you can use to get a specific task done. Pete Bishop: Yeah, exactly. think like if you're in ChatGPT or Claude, you usually see their list of software that you can connect to. That's probably all using MCP servers to some degree to allow you to like book tasks in a Santa through Claude and those types of things. That's amazing. So is the plan for, ⁓ the plan to get this up and running anytime soon? How long do you figure this will take before it's actually up and live? Sohil Vinod Shah: I'm trying to get access to the MCP server for one of the travel concierge agents. So once I have that up and running, then it would great to actually productionize it then have it as a service that I can sell out. ⁓ Pete Bishop: Yeah. and I think it's so interesting now how software is becoming more approachable for a lot of people. Like not to you, you were already there, but I think for the average person to be able to build out tools and stuff has always a fair bit at arm's reach. And now these things are becoming easier. Although it's nice to hear that there's still the need for to come in and make sure that the code, like you said, is legit. And there's, There's things that still need to be worried about and fussed about before we have proper of commercial grade software out in the world. Sohil Vinod Shah: Yeah, I have seen examples of product managers at PayPal trying use LLM and vibe code most of the web pages that they expect to see, design docs and all of those without having any background software engineering or software development and they're able to figure all that stuff out. It's a relief and I can see the sense of accomplishment for them to be able to do that. without having to depend on the engineering teams for all these low hanging of concepts that they need to build out for the customers and things like that. Pete Bishop: Yeah, I imagine you're just like, now we don't have to build the next landing page or whatever. ⁓ Sohil Vinod Shah: Yes, exactly. Pete Bishop: mentioned you're using Google and Gemini, but given how these, these companies are kind of upping each other all the time. How you how do you choose which one to use at the moment just given how they're all changing and the next models beating the next model and so on and so forth. Sohil Vinod Shah: So I think lot has happened in terms of, so what I feel like the lines have been blurred between programming languages and tools that we use. Cause we don't need to learn a specific tool or a specific language in order to be productive in a specific project or a field. So that line has been blurred because of AI. So in my case, when I started working at PayPal, I had never done any Python programming before that. And to be very honest, I still don't understand a whole lot of Python, but everything I do ⁓ today's day and age is all built on Python. And is only happening because I have access to LLM and all the other tools. Otherwise it would take me a few years to learn the language, hone up that skill, become an expert in that particular ⁓ programming before I can start reviewing some code or giving positive feedback to anyone else. Pete Bishop: that interesting? Hey, it's just you're back in the day You would have to just go back to school and learn all that again. And you're just coding. Yeah, it's that's bananas. Well So hell is ⁓ wonderful having you the podcast. Thanks for sharing your work. That's super interesting And you're live, let me know and I'll I'll send out a link. We'll get everyone on it Sohil Vinod Shah: OK, I'll definitely post it out. Pete Bishop: Yeah. thanks very much. ⁓ yeah, we'll chat to you again. Sohil Vinod Shah: Sounds good. Thank you so much. Really appreciate it. Bye. Pete Bishop: All right, bye bye.