Yoni Leitersdorf: Hello everybody and welcome to another episode of Building with AI and Heartbreaks. Today I have Ritika with me from Thomson Reuters. Hello Ritika. ⁓
Rittika Jindal: Hello.
Yoni Leitersdorf: I'm so happy to have you here on the podcast. So for the audience who doesn't know, I saw Ritika present. at the Gartner DNA conference a year ago in Florida. Actually a year and two weeks I think ago. I was amazed by our presentation. ⁓ She talking about something called ⁓ OpenC2, we'll talk about in second. And then since then, I've been following Euridica and you've been building more and more interesting things around AI and I'm so happy to have you here. So maybe for our listeners first, let's start with some background. So you're Thompson Reuters, you're a principal engineer. So what does Thompson Reuters do for those who didn't hear the Joel Ron episode that I did? And then what do you do as a principal engineer?
Rittika Jindal: First of all, thank you, Yoni, for having me and thank you so much for attending my talk at Gartner. I think it was one of a kind experience. Now coming back to Thomson Reuters, coming back to what I am doing at Thomson Reuters. ⁓ let's start with ⁓ I think ⁓ it's important to maybe ⁓ the conversation that Joel started. So Thomson Reuters, we are mainly a ⁓ trusted content and technology company. ⁓ And when I say content and technology company, we mainly focus on empowering professionals in legal, tax, risk, compliance, and journalism. key focus is about ⁓ ⁓ AI products. So we have two main products, Westlaw and Practical Law, ⁓ are used by these professionals to their customers, to help their clients. ⁓ And I talk about what I do, at TR. I'm part of a team called Content Innovation. are the R &D team in the content space at TR ⁓ it's a mixture of ⁓ and research engineers and job is to design and build ⁓ innovative ⁓ to you know ⁓ to about how we process our content how we ⁓ deliver content into a product like Westlaw and Practical Law.
Yoni Leitersdorf: Okay, Thompson Reuters ⁓ tools for people in the legal space. And part of that, they research content, case law, history, etc. and you're part of the innovation in doing that. So give us maybe an example or two that you're comfortable with. What does innovation in content for law do? How does it look like?
Rittika Jindal: So I think we can look at it like today, most of the companies or more than enterprises are looking at building AI, building with AI, building AI. And our team also focuses on building agents or building these agentic workflows. you have, when you're processing content, you would have a pipeline that you will build to process content, to deliver content to a product. So our team takes care of enriching these pipeline processing.
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: this pipeline using AI, using the tools and technology that are available process and to enrich the content for a product.
Yoni Leitersdorf: Okay, so a year ago, yes, I got
Rittika Jindal: And I think... Yeah, I was just saying, just adding to this and what makes this really interesting is precision, ⁓ volume that's right? So when, let's say, ⁓ legal are using our products, they're using our products to file cases to their customers, which could impact their ⁓ lives, their careers, know, ⁓ all of that. So the impact is not about just delivering the content, it's about about delivering the content with accuracy, delivering the content faster, right? So our pipelines have to have all of these features that, you know, that takes care of the accuracy and the content currency as well.
Yoni Leitersdorf: So all those news articles we see where lawyers use AI to draft legal arguments ⁓ the AI quotes things that don't exist and hallucinates, cetera, you're helping them avoid that essentially, making sure that they don't make that mistake.
Rittika Jindal: Yeah. Exactly. Yeah, exactly. And this is not like a metric or a check that we have to have. It's like it's rooted in a product, right? It's like a professional trust bar that's built in into a product. And this sort of drives ⁓ that we do, right? All the designing, all the building of the product. Like accuracy is the key.
Yoni Leitersdorf: that makes sense. are, you a lot of times people talk about what's the level of accuracy that's required in different flows. You know, when reporting financials to the stock market, obviously, accuracy has to be 100%. When, you know, answering just simple questions may be less, but in your case, people's lives depend on this, and you need to make sure that you're accurate. So,
Rittika Jindal: Yeah.
Yoni Leitersdorf: Okay, so let's double click and zoom in a bit more. So you're using AI to accelerate innovation within that process of content generation. And you write lot online about the journey that you've gone through from, you know, a year plus ago ⁓ with OpenC2 you talked about at Gartner to ⁓ how do software teams like today? So maybe if I start with OpenC2, that was a year and a bit ago.
Rittika Jindal: Hey.
Yoni Leitersdorf: what was it? What was that project that you talked about at Gardner?
Rittika Jindal: Yeah, so OpenSETU, setu is a Sanskrit word. ⁓ means bridge. ⁓ OpenSETU is a data agent that helps the professionals in TR ⁓ ⁓ the data, right? So let's say if I am into marketing or HR or sales, ⁓ want to look at the data. ⁓ ⁓ now you would have ⁓ analyst in between, you know, so as a non-technical user, I would ask them the kind of data that I want. ⁓ these data analysts will write the
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: the Python code or the SQL code and then they will fetch the data, create these reports and then send these reports. So we wanted to bridge this gap so that our non-technical users can directly interact with the data. So OpenSETU was a data agent which allows a non-technical user to interact with the data. So you write a question in a natural language, what were my sales last year or how many employees are on leave this month? So you ask a question and converts this natural language question into Python or SQL depending upon the complexity of the question. So it has a list of tools at its access. So it writes the code and then it runs the code on the data and then gets the data the HR or to the marketing ⁓ So it starts to bridge the gap. ⁓ we called it the data analyst agent ⁓ it's like writing the code for you, it's bringing the data ⁓
Yoni Leitersdorf: And it.
Rittika Jindal: to you.
Yoni Leitersdorf: And we talk a bit about accuracy. is the accuracy of OpenC2? What was the accuracy of the tool? ⁓ did you measure it? Can you walk us through that a bit?
Rittika Jindal: Yeah. Absolutely. So it's a tool which is backed by the data. So the tool is not generating the data, it's generating the code for you. So the ⁓ generates the code and it runs the code. ⁓ then the runs on the data that's already there. ⁓ So you run the code on the data, it gets you the accurate data. ⁓ But didn't just stop and relied on how this code was generated. We of course went on and added ⁓ evaluation, We had the ground truth data. So we did a lot of evaluation about the data that it was generating at that time. like now ⁓ that we developed is sort of ⁓ driven and test driven. We make sure that we have the ground truth data. We are evaluating whatever we are building, starting from day one.
Yoni Leitersdorf: Okay, so essentially. you knew what the answers were to some questions. And then you said, OK, we'll run these questions. We'll see what answer OpenC2 gives us. And then we'll compare it to the actual answer that we expected see that they're the same, which makes sense. That's exactly how evaluations work. And we see it in a lot of places. Of course, we also have it in our own platform in Solid. But that's kind of the ⁓ as you're saying. ⁓ start from evaluation so that you can test that you can rely on this thing. ⁓
Rittika Jindal: Yeah.
Yoni Leitersdorf: So that was about a year plus ago. Since then, you posted a lot about how software development looks like And there were a couple of posts that ⁓ put on recently that I want to dig into those. So ⁓ one of them, you talked about how the work of software development has changed completely. The places that people are now putting their effort and emphasis in your own people, your own architects, your own software developers, have shifted their focus they're now using AI to write the code for them. So let's dig into that one first. So tell me a bit about what you wrote there.
Rittika Jindal: Yeah, think honestly it has changed even since the last time I wrote my blog. Like it has changed ⁓ much since and I think I need to write a new blog. But let's about that blog. I think it was about how our software engineers focuses more on ⁓ the planning the designing rather than writing code. So let's say would as a software developer, you would spend ⁓ 30 % of time designing and planning. But now you spend 70 % of your time on designing and planning where you're asking questions like, why am I doing this? For example, let's take a very simple example. You are converting, let's say, we are in content space. We do lot of formatting, a lot of conversions, et cetera. So you're converting from one format to another format. Earlier, you would maybe research or you will look about, how does this format work? But now our software engineers ask questions like, do I really need this format? So that's the design thinking that has changed in this life cycle. But you know, I think I wrote that blog maybe a month or two ago. Right now our focus has shifted, you know, slightly more in towards AI native engineering. So at that time we were focusing on using AI to write code to help us plan, implement, right? But now we look at this AI native engineering where we see the whole SDLC cycle is built using AI. Now, if you look at like the whole SDLC cycle, ⁓ you have planning, right? So you gather requirement, you do the plan, you do the design, then you develop, you test, you debug, you know, then you deploy to production, right? So it's like a full life cycle. So now we have AI at all of these stages, right? So we start with plan, you plan with AI, then you write code with AI, but we also now have built our skills, know, which allows us to test, which allow us to debug, right? So lot of us ⁓ our project, we use cloud code as our AI dev tool. ⁓ what we have done, ⁓ of course have tools or skills that allow us to write code, ⁓ the code, but we now all have ⁓ that helps us test the code. So we have built end-to-end testing using skills that allows us to test. So when I say test, I don't just mean the unit test or the integration test. I mean ⁓ actually the whole workflow ⁓ is built in Cloud Code, right? And then your logs, let's say we're using AWS, so our logs goes into CloudWatch. agents have LangFuse built in, right? So ⁓ we skills that gets us LangFuse logs. So everything sort of comes into the same terminal. We pull in all the logs onto the same terminal and then we debug, ⁓ like there is an error, we debug in the same terminal. So you see, that's how we bring the whole SDLC cycle into our same terminal.
Yoni Leitersdorf: So, okay, so let me see if I understood correctly. So let's say there is a type of content you wanna pull onto the platform. So your team needs to build essentially the pipeline that pulls it in, right? That pulls and adapts it and modifies, et cetera, and loads it into the platform.
Rittika Jindal: the papers. Yes.
Yoni Leitersdorf: Now, activity that, an item like that starts with, OK, we need a plan. We need a plan. What we're going to bring, ⁓ from what sources, how are we going to convert it, how are we going to prepare it for the designated platform, and of course, test it along the way. So there's the plan of that, of what we're going to build. Then there's the building of the code. So ⁓ actually writing code, of course.
Rittika Jindal: you.
Yoni Leitersdorf: Then there's the testing of it and to see is it working. It pulls the content. it, you know, is it making the conversions or the results the way that you expected? And if they're not the way that you're expected, how do you debug that? And your point is in every single phase AI is involved. So let's start maybe from the planning. What, what tools are you using? What's your methodology for using AI within the planning for, for night for a work item, like the one I just described.
Rittika Jindal: Okay, yeah, so if we start from, let's say, just plan, so all the developers ⁓ a plan document, right? So you write, ⁓ with Claude to create a plan document, ⁓ that plan documents get within the team, depending on ⁓ the of that story or impact of the feature that you're working on, right? So let's say if you are just fixing bug, ⁓ a work doesn't, the plan doesn't need to be reviewed by everyone, but if it's like a ⁓ bigger that you're working on, the plan needs to be reviewed. the plan reviewed by the reviewers or by the senior engineers. So now plan just doesn't sit in a software developer's laptop. The plan actually goes into a GitHub, right? So we have a where the plan sits. ⁓ ⁓ take the plan, you ⁓ the plan reviewed, then you go into writing code using that plan, right? So you have written the code. ⁓ you are going to create a PR ⁓ has your code and your plan.
Yoni Leitersdorf: Okay. Okay. Okay, so the original plan goes into GitHub, probably a markdown file, I assume. so it's a markdown file. Senior engineers review it. By the way, when the senior engineers review it, do they actually read through it or do they use their own AI to review the plan that you propose? Like, does that work? ⁓
Rittika Jindal: Yeah, that's a. That's a good question. think initially when we started, sort of struggled with, you know, like, do I start reviewing the plan also with AI? But over the time we have realized we have actually cut down what we write in the plan. So it's easy for me to like just quickly glance over it. So I usually read like, you know, maybe first 10 lines and I know what the thousand lines of code are about. Right. So the plan is too big, you would need AI to ⁓ faster. But ⁓ plan is like really ⁓
Yoni Leitersdorf: Yeah.
Rittika Jindal: It's really about what you are doing. So it becomes really easy to read the plan by a human as well.
Yoni Leitersdorf: Okay. Makes sense. I use Cursor a lot. I have it plan most features in advance and usually I read the top. I don't usually read the actual implementation. mean, in my case, the implementations are because I don't write production codes and my case, the implementation is less my interest, but it's more, it implementing the right logic or going to. So, okay. So they write the plan, plan gets reviewed by humans. Then the code gets written by cloud code based on the plan. And then when there's a pull request open, there's the
Rittika Jindal: Yeah. Yeah. Yeah.
Yoni Leitersdorf: code that Cloud Code generated, and then the plan that was agreed to in advance is also included. OK, so pull request gets open. Now, that you guys do is different. ⁓ now Cloud Code came out with the PR capabilities, then a lot more people are using it. But you started doing this before. And so PR gets open. What happens next?
Rittika Jindal: Yeah. Yes. Yeah. So now it's about reviewing the code, right? So we have two levels of reviewing the code. We have built an on-demand Claude skill that a developer runs when they are creating, or before they create a PR, they run this review skill, which looks at the plan, which looks at the code, which looks at the AI's law, which looks at the test cases that you have written. So it does ⁓ all of that evaluation. this is an on-demand run by a developer, right? ⁓ let's say as a developer, if you miss those, right?
Yoni Leitersdorf: Mm. Mm.
Rittika Jindal: also wanted to automate this. So we also have a Claude bot reviewer that runs when you submit a PR. So when you submit a PR, bot kicks in. It all of these checks about checking the plan, checking the code, checking the architecture alignment, all of that. So ⁓ it that and ⁓ the sits in the PR. So when as a reviewer, ⁓ I the PR, I see the plan, ⁓ I the code, but I also see the review which is done by this bot.
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: automatically. So now I have a lot of things at my ⁓ to review the code. I don't need to read every line of the code now. The reviews are much faster. I just read few lines of the plan, then I read the review done by Claude.
Yoni Leitersdorf: And then I think in your, in your post, you wrote that basically Claude picks up all the, stuff, like the, basic checks that you always have, it picks them up and immediately can push back on, on them. And then that allows the humans to focus on the higher level things in, in that work. by the way, if, Claude finds issues in comments on the pull request, does the developer then go ⁓ and their Claude with the comments that came from the PR ⁓
Rittika Jindal: Yeah. Hmm.
Yoni Leitersdorf: like how does that loop get closed at that point?
Rittika Jindal: Yeah, so when a developer creates a PR, the Cloudbot runs and it gives you the comments, right? Then the developer takes that PR, gives it to their Cloud and say, okay, go fix the comments which are from the reviewer. So it's mainly about how you use different personas of Cloud, right? So you have a developer has a developer persona, but the Cloudbot has a reviewer persona. So you get those comments into the developer terminal and they try to fix, they do a lot of
Yoni Leitersdorf: Mm-hmm. Mm-hmm.
Rittika Jindal: iteration there as well and then you know once they have fixed all the comments then the PR goes to the senior engineer.
Yoni Leitersdorf: Okay, now. So now what happens is PRs, the basic stuff has been flushed out and fixed. And now it's only the things that you need humans for. You wrote a tool that every morning you can see what PRs need your attention, right? I saw you vibe coded a tool for doing that. Why did you need that? Why not just go into GitHub in the morning and say, ⁓ okay, here are the seven PRs that are waiting my attention. Why you need this?
Rittika Jindal: you think there are a lot of things that I found I needed ⁓ at that time. It was about ⁓ do I rank different PRs, what I need to look at first. ⁓ in GitHub, you do see all the PRs. You can sort them by last modified, et cetera. ⁓ can see all of that. But what I cannot see without opening is the number of lines written in that, ⁓ of files changed in that, unless I click on every PR. ⁓ I wanted a dashboard to quickly approve PRs, which were short. ⁓ Like a bug fixed PR, I want to review it, know, like the first thing in the morning. So I built this dashboard that helps me, you know, look at, what are the top PRs that I can review while I'm having like a meeting, you know, in a, yeah, like I can do multitask with these tools. Smaller PRs maybe don't need 100 % of my attention. I can review them while I'm doing something, you know, attending a meeting or listening to something as well.
Yoni Leitersdorf: OK, at a, actually one thing I noticed, the more I use Cursor, the more I multitask. And I'm not necessarily very good at multitasking, but I bought another screen. So now I have my third screen here because I needed a screen for Cursor so that I can watch what it's doing. And ⁓ as on a Zoom, if I'm not the one talking, I can quickly take a look and give it another instruction, another instruction. So. ⁓ Sometimes it feels like my brain burns. I'm like, feel overloaded with all this multitasking, like talking to someone working on cursor, you know, handling things on the other screen over here. It's ⁓ feels feels like a lot. Does that happen to you at all?
Rittika Jindal: Absolutely. So I think there was a time when I was working on like four to five Git work trees. know, like, of course, like three different terminals, but also like in the same terminal, I have two, three different things going on. And I realized that I was mixing my prompt, right? So if I had to write a prompt onto the first terminal, I would rather put it in the second terminal, right? And I think over the time I've started to realize that I cannot work on more than three things and that's my limit. So now I have decided that I will not have more than ⁓ in parallel. ⁓ could be maybe two GitHub repo and then a meeting, but I'm not gonna do more than three things. ⁓ think that's a ⁓ or that's something as an individual ⁓ have to find for yourself, right? Like what's your limit? How ⁓ of multitasking tasks you can do ⁓ in go without burning out and without feeling ⁓ stress ⁓
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: say Tuesday or Wednesday of your week, right? Because you survive till Friday, right? So yeah, I think on every individual ⁓ find that bar, to find that limit.
Yoni Leitersdorf: Yeah, I'm probably on the two or three range too. I wonder if as you know, the next generation, so I have kids they're 14, 12 and 10. And I wonder if when they get into the work world, you know, the corporate life or whatever you want to call it. are they going to have more multitasking abilities than I do, for example, or than we do? Because if you think about it, if you took someone, you know, a generation above us to them, working on zoom, recording on Riverside, sending emails at the same time, what's happening with someone on another thing like to them that that looks like a lot. ⁓ And so and obviously the things that look normal to us today.
Rittika Jindal: Yeah.
Yoni Leitersdorf: looked like a lot to someone who worked in the space of generation before us. So I'm wondering if the generation that follows is going to be able to multitask a lot more than we can today. What do you think? I know it's very theoretical, but what do you think?
Rittika Jindal: really hard to speculate though, but think the next is even smarter than us. ⁓ I think find ways to... maybe not only multitask, but also do things that our previous generation used to do in a much better way. I felt like the generation before us, you said, didn't really do multitask, but they were still completing all the tasks that were required, right? And think the new generation is going to find ways to not burn out, ⁓ but still able to check all the boxes. ⁓ Yeah, I think I have high hopes on the new generation. And I think
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: they have ⁓ sort of clear boundaries ⁓ you like creating that for themselves. know, like if you talk to JSC today, they have a clear understanding of, you know, where they want to stop, right? How do they define the whole balance and...
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: although I don't know where it stands for me right now. I don't I feel like it's a mixture. Like on Sunday sometimes I'm thinking, ⁓ maybe I should build this to help on my Monday. And then Sunday night I'm sort of like, wipe coding, know, that ⁓ dashboard that talked ⁓ to you Now. I think the work that I'm doing on Sunday evening is going to help me, know, for the entire month. Right? So is I don't think that's too bad, right? Because if I spent like maybe two hours on Sunday, but it helps me ⁓ for a I mean, sure, ⁓ I can a relaxing month. I think that's okay, ⁓
Yoni Leitersdorf: Yeah, it's you're paying on a Sunday to be able to work maybe slightly less on a Wednesday on the following Wednesday, et cetera. Yeah, I wonder about that because I feel like at least in role, it feels like we're ⁓ on treadmill that keeps running faster and faster. Like somebody keeps pressing the up button ⁓ and.
Rittika Jindal: Yeah. All the Yeah.
Yoni Leitersdorf: And what you do is you try to do more, do faster by, you know, by building all these tools for yourself. Like there's literally decks, for example, for solid half the board deck is built by Claude, ⁓ because there was recurring things and it's okay, go and, know, pulls it from our CRM and from our Slack and our email and Google drive it on like it does prep work. So it's clear time for me.
Rittika Jindal: Mm.
Yoni Leitersdorf: But then what I did is I filled that time by other things. I don't know if I'm doing more or, I don't know. It's definitely interesting. ⁓ guess speaking about doing more, so now the development is done with AI. So the planning, the coding, the review, everything is done by AI.
Rittika Jindal: Yeah.
Yoni Leitersdorf: Have you measured the change to the results of the team? So are you delivering more work items as a team? Are you delivering them faster? ⁓ you delivering less bugs? How you measure all that?
Rittika Jindal: So the dashboard that I built earlier was not only to help me the PRs, but it was also about measuring ⁓ number of PRs that were approving and merging. So wanted to measure whether our has really increased. Are we merging more PRs? we pushing? Are we really faster? ⁓ So that dashboard tell me that before, I think if I looked at the previous repos, maybe six months, back when we were maybe not AI native engineers. looked at some of the repos where ⁓ data showed me that we were merging maybe ⁓ to six PRs ⁓ a week, right? And that could, the of ⁓ lines the number of code could vary, you know. So I'm not gonna go into ⁓ the of those PRs, but that was the number that I saw. ⁓ But we are more than that every day. data shows me that we are ⁓ 5 to 10 PRs ⁓ single day. ⁓ scale has changed ⁓ think we are ⁓ faster, we are developing faster. like the shows that. ⁓
Yoni Leitersdorf: Mm-hmm. Yeah, we've been experiencing a similar thing. So our sprints at Solid are three weeks long. And I listened today to the sprint demo that we had at the end of the sprint now. And the VP of Engineering said that we merged 170 PRs over the last three weeks.
Rittika Jindal: Hmm. Hmm. Hmm.
Yoni Leitersdorf: So call it 50-ish PRs per week across a team of 20 people. 2 and 1 well actually it's less than 20 people. So let's say 2 and 1 half to 3 PRs per week per person, which is the most we've ever done. That's the highest rate. Now the sizes of these PRs are the same sizes of the PRs we always had even a year ago. The number has increased. The number per person has increased.
Rittika Jindal: Yeah. Mm. Yeah. Yeah.
Yoni Leitersdorf: And I also listened to ⁓ Lenny's podcast and he was talking to the guy who built Cloud Code. And that guy said that the number of PR, basically I don't remember exactly the number he had of PRs that he ships, but we did the calculation and he ships 30 times more PRs the non AI assistant software developer, 30 times more. ⁓
Rittika Jindal: Thank there. Yeah.
Yoni Leitersdorf: So is that where we're going? Just talking about the whole, well, our previous conversation of what we're doing and balancing. So is that what's gonna happen? So the expectation from software developers will be, the standards will be, okay, you need to 10, 20, 30 times of what you would have shipped five years ago. Is that what you see was gonna happen?
Rittika Jindal: think it's already happening, right? That is the expectation we have today. you have all the ⁓ tools at your disposal. Now you are shipping faster, you are writing the code faster, so we are delivering faster. Like it's not something that I'll ⁓ an year from now, I think it's something that we are asking today. Yeah, and the data shows that we are actually doing that. Like you also said that you have 50 PRs a week. ⁓ the current stat that we're talking about. But also feel that.
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: Over the time, it's not just going to be about writing code. It's going to be about how you the whole life cycle around this. So when you look at a product development, it's about ⁓ the requirement. It's about writing the code. It's about testing. It's about evaluating. I think ⁓ focus right now is on the testing and is on the evaluation. So we are focusing a lot on EDD and TDD, where we say that ⁓ write the test cases, you understand the evaluation, you understand the ground truth data, so you understand what you're really going to build and then you make that a part of the development. So now the focus is more on, we are using tools like LangFuse, we are using tools like BrainTrust to ⁓ evaluate, to hook the ground truth data to review the output these agentic workflows are producing for us. So going to become ⁓ a key metric ⁓ for this year.
Yoni Leitersdorf: So. when you're measuring the productivity of the team. So there's the number of PRs, there's the quality, the EDD, the evaluation-driven design, and then the TDD, the test-driven design. So that helps you validate the quality of it. Do you find that AI is creating slop ⁓ a mess in your code that you need to clean up over time? Are we paying something on the other side where the AI is creating more work for us somehow?
Rittika Jindal: ⁓ Yes and no. Like today, yes, we do see a lot of AI slop. We see a lot of dead code, especially when you have a lot of people using AI, right? slop started to vary, right? Let's say I worked on a story and then somebody else is... adding an enhancement to it. I could created a slop. I created a dead code. Now other developers or other engineers Claude is not going to read the code that's not working. It's going to at what's working and then build on top of it. And it's going to create its own slop. But I also feel that with the evolution that's happening in the base models, like we now use Opus 4.6 or Sonnet 4.6 to write a code. And the way I think Anthropic is shipping you know their models and shipping cloud code the enhancements I think all of these will soon be taken care and it will be think of the past as well like today we look at the AI slop today we look at the dead codes you know all the unused imports etc and we have added these into our review skills you know which reviews the code so a lot of this is is done automatically like we don't have a human checking AI slop it's done by AI but I think over the time This will be built in into the cloud code to make sure that you know, it's not generating any slop It's not generating dead code as well
Yoni Leitersdorf: OK, so the dead code in the slop is not a big issue and will be solved on its own. What you care about more is testing the code functionality that it works. And if it works and it passes the evals and the TDD, et cetera, then some slop is acceptable because it's just part of it and you clean it over time. Is that a right way to think about it?
Rittika Jindal: Yeah. Yes, and I think slop was always there. Like even when humans were writing code, we can say, right? Like we didn't write slop free code. Slop has been there. Maybe now the pace at which the slop is written is a little higher, but slop has always been there.
Yoni Leitersdorf: Yeah, true. Yeah. Yeah. Yeah, that's true. I, My code has been, it was always messy. My personal code in my previous startup, the VP of engineering, who's the same guy, the same VP of engineering here at solid. name is Alon. He barred me from writing code. He said, you can't write any more code because your code is a mess. You don't follow guidelines. You don't always have unit tests and stuff like that. So he said, you can't write any code anymore. And I respect that. ⁓ So yeah, was probably a ⁓ human slop So. Okay, I listened to you talk at that talk a year ago. Now, can you imagine what your talk would be a year from now? So let's say March 2027, you know, it was from a, you know, data analyst, automated AI data analyst to now code is being written by cloud code and humans are only involved in the evaluation and checking the plan and all that. A year from now, if I had you here in the podcast again, what are we going to be talking about?
Rittika Jindal: Mmm. Well, it's really hard to imagine, I think, the pace, the things are changing ⁓ week. It's really hard to imagine. if ⁓ I to, think... So, ⁓ you know, try to connect a lot of work we are doing with ⁓ how we doing maybe a decade back or five years back, right? ⁓ So ⁓ we at decade back, there was a called RPA, Probiotic Process Automation, if I'm not wrong.
Yoni Leitersdorf: Mm-hmm. Yeah, robotic process automation.
Rittika Jindal: Right? Right, right. So when we started looking at RPA, it was about automating all the manual things that we were doing. ⁓ We started evaluating whatever we were doing manually and it was a process to automate a lot of these things. when I was working at that time on RPA, ⁓ idea was that anything that you are doing more than five times, it to be automated. You shouldn't be doing it manually. ⁓ So was a ⁓ basic ⁓
Yoni Leitersdorf: Mm-hmm. Mm-hmm.
Rittika Jindal: And now I think that has changed with the AI, with the AI tools. I think it has, the number has reduced from five to three or maybe even two. Right? So anything that you're doing more than three times or more than two times, it should be all done by AI. Right? So that, that number has changed. So I think that maybe is still going to evolve over time. that I'm doing ⁓ than two times a day, I'll put that in an agent. or in an automated AI system that will do. And then I think I've also talked about how this whole SDLC cycle is AI native. ⁓ we are talking about ⁓ just AI engineering, we are talking about ⁓ native engineering, right? So the freshers or the college grads who are joining ⁓ companies, they are I feel like they are more AI native than we are, right? because they don't know how to work without AI. So thinking process starts with AI. I think they're gonna come up with more and more ideas and more and more innovation ⁓ not ⁓ SDLC, the whole of how we think of a new product, how we come up with an idea of building a new product. For example, now, ⁓ I look at, okay, these are the problems I have at my hand I need to build the solutions ⁓ automate these using AI. ⁓ think after year, the problems are also going to vary. The problems are also going to ⁓ change lot and the way we ⁓ build is also going to evolve. ⁓ I know if that answers your question. ⁓
Yoni Leitersdorf: Yeah, it basically you're sort of so what you're saying is let's talk about the fresher the the those fresh out of college. I can maybe equate it to a world where workstations, personal computers, etc. entered the corporate world. And then in parallel, they entered the college and university world, academic world. And at some point, those coming out of college were already well versed in using a computer. And so they could do stuff with it that maybe the people that were working back then didn't know how to do or didn't think about doing because they grew up in a pre-computer world, pre-personal computer world. And then they adapted. those freshers back then could do things or think about things that the current workers couldn't. And so what you're saying is now there's going to be a point where people who come out of college, they did their entire computer science degree, let's say, or whatever degree using AI. They're so used to it when they enter the corporate world. they will put it to use that we can't even imagine. Like we can't think of right now. They will do it because it will be so native to them. So that's interesting. That will make it very interesting, obviously. There's a lot of talk about how those fresh out of college are having a hard time finding a job because a lot of what they can do...
Rittika Jindal: Yeah.
Yoni Leitersdorf: AI can do. So how do you reconcile these two things? On the one hand, the freshers will bring in new ways of doing things that we can't think of, but on the other hand, nobody's hiring them because AI can do the stuff that they can do. So how do you, how do you reconcile that?
Rittika Jindal: Yeah. I think in my personal opinion, the profiles, the roles that we are hiring for are also going to change time, ⁓ upon the problems that we are solving. ⁓ ⁓ year, was checking how, when I hired a developer or an engineer, was checking how are writing code. But today, not looking at how they are writing code. I'm looking at how they are thinking about ⁓ solving this problem. ⁓ So the is about ⁓ what kind roles we are looking
Yoni Leitersdorf: Hmm.
Rittika Jindal: looking at what kind of people we want to hire, I think that's also going to evolve. So let's say today we are hiring software developers. Maybe five years we'll hire software thinkers, you You never know. I feel like, you know, ⁓ ⁓ the kind of work we do is changing. ⁓ the profiles are also going to change and, you know, the way we hire ⁓ grads are also going to evolve. ⁓
Yoni Leitersdorf: Mmm. Yeah, you're right. I mean, we're not going to need developers anymore. We'll need thinkers, planners, dreamers maybe. ⁓
Rittika Jindal: Yeah, absolutely.
Yoni Leitersdorf: Yeah, it's a good point. mean, the profile will also change. I think for software development, you need a certain profile, like someone who's really good at math and their brain works in a very specific way, et cetera. But if you don't need that anymore, then maybe you have other people in the creation of software, you know, with other skills. It'll be really interesting. So I guess your point is...
Rittika Jindal: Yeah.
Yoni Leitersdorf: We don't know. A year from now, you don't know exactly what we're going to be talking about. you can see some trends and take some bets, but it's very hard for you to put the finger on it, which I agree. I can understand that, how complicated that is.
Rittika Jindal: you Yeah.
Yoni Leitersdorf: Okay, so that was a really interesting conversation. So we're nearing the end of the episode. And at the end of the episode, I always ask a few questions. So of them I basically did, you know, where would be a year from now? Usually I say five years, but where would be a year from now is a good one. And I probably know the answer to this, but ⁓ what an AI tool you can't live without?
Rittika Jindal: I do want to say cloud code or cloud.ai. Yeah, think today, yes. ⁓
Yoni Leitersdorf: Yeah. Yeah, I assume that was the answer. What is an AI myth you'd bust in one sense? Is something that people are saying about AI that you think is nonsense? What do think that is?
Rittika Jindal: Hmm, I'll have to think about that. I do want to say that the myth, no not a myth, that might be true. I don't know. So when I talk to a lot of people, think, who are trying to upskill in AI, ⁓ find it hard ⁓ upskill. think that's a myth like upscaling today is quite easy. We have so much content, so many tutorials available at our disposal to learn and to upscale. I think it's just about ⁓ ⁓ path, finding what you're looking for. there is just like even if you look at ⁓ OpenAI's you look at Claude's tutorials, Anthropic has published like an Anthropic Academy where you can learn a lot about ⁓ how we are using tools. So I think upskilling is not as as it may seem. ⁓ Like today, think upskilling is quite easy. ⁓ If have the ⁓ right if you have the time, ⁓ think ⁓ upskilling easy now.
Yoni Leitersdorf: Yeah, I hope everyone has the intention to do it. But yeah, I see your point. A lot of people were talking about how hard it was for them to learn AI, but basically you're saying there's all the tools now. There's no excuse not learning how to use these tools. Amazing. OK, ⁓ is there source? So first of all, for the listeners, if you're not familiar with Ritika's ⁓ medium, so ⁓ RIT.
Rittika Jindal: Thank
Yoni Leitersdorf: Ttikajindal.medium.com. So I really suggest following her there and reading ⁓ content or finding Ritika on LinkedIn and her there. But other than your medium, what is the source that you recommend for people to read? ⁓ Podcast, book? that you recommend people to read. And it could be an AI, and it doesn't have to be. It could be in a completely different space.
Rittika Jindal: I think there are several tools, one is you can just. ⁓ open any tool, any LLM, you can just start asking questions to LLM and sort of prepare about the topic that you are interested learn about. ⁓ also, I think there are already so many tutorials. If you look at ⁓ Engie's ⁓ ⁓ are just so many free courses available ⁓ anybody can learn through. ⁓ ⁓ say ⁓ you want to learn about how ⁓ is working,
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: you can look at Anthropic Academy. If you wanna learn about how OpenAI is building or how prompt engineering, all of that, I OpenAI also have their own learning documentation. ⁓ I would say if you wanna start from ⁓ maybe start by looking at the documentation that's published by all of these enterprises, like the foundation model enterprises, ⁓ then you can also learn from deeplearning.ai. There are ⁓ like plus courses. And I think Andrew Ng really picks up like, know, so I think last year he created a course which is about writing code with AI, I had not seen before. Like we were all doing it, but nobody was creating a tutorial or a course about it. So I think ⁓ a really interesting ⁓ you know, for ⁓ like a product manager or a product owner ⁓ just see, you know, how developers are writing code with AI. I ⁓ Andrew is creative.
Yoni Leitersdorf: Mm.
Rittika Jindal: creating like courses for all of us. So ⁓
Yoni Leitersdorf: Yeah. I agree. took a couple of the courses on his site. They build it really well. It's really easy to internalize and to understand. So I agree with you. OK. Amazing. Ritika, I'm so happy we got to do this. ⁓ I'm really actually looking forward to doing this again a year from now ⁓ seeing how things changed. Because obviously, things change a lot. And they change fast. They change quickly. But really, thank you very, very much.
Rittika Jindal: You
Yoni Leitersdorf: for being here Riddika and for those listening to the podcast please remember to like, subscribe, thumbs up, whatever may be on the platform that you're listening to this to and we are gonna see you next time. Bye Riddika!
Rittika Jindal: Thank you, bye bye.
Rittika Jindal: Hello.
Yoni Leitersdorf: I'm so happy to have you here on the podcast. So for the audience who doesn't know, I saw Ritika present. at the Gartner DNA conference a year ago in Florida. Actually a year and two weeks I think ago. I was amazed by our presentation. ⁓ She talking about something called ⁓ OpenC2, we'll talk about in second. And then since then, I've been following Euridica and you've been building more and more interesting things around AI and I'm so happy to have you here. So maybe for our listeners first, let's start with some background. So you're Thompson Reuters, you're a principal engineer. So what does Thompson Reuters do for those who didn't hear the Joel Ron episode that I did? And then what do you do as a principal engineer?
Rittika Jindal: First of all, thank you, Yoni, for having me and thank you so much for attending my talk at Gartner. I think it was one of a kind experience. Now coming back to Thomson Reuters, coming back to what I am doing at Thomson Reuters. ⁓ let's start with ⁓ I think ⁓ it's important to maybe ⁓ the conversation that Joel started. So Thomson Reuters, we are mainly a ⁓ trusted content and technology company. ⁓ And when I say content and technology company, we mainly focus on empowering professionals in legal, tax, risk, compliance, and journalism. key focus is about ⁓ ⁓ AI products. So we have two main products, Westlaw and Practical Law, ⁓ are used by these professionals to their customers, to help their clients. ⁓ And I talk about what I do, at TR. I'm part of a team called Content Innovation. are the R &D team in the content space at TR ⁓ it's a mixture of ⁓ and research engineers and job is to design and build ⁓ innovative ⁓ to you know ⁓ to about how we process our content how we ⁓ deliver content into a product like Westlaw and Practical Law.
Yoni Leitersdorf: Okay, Thompson Reuters ⁓ tools for people in the legal space. And part of that, they research content, case law, history, etc. and you're part of the innovation in doing that. So give us maybe an example or two that you're comfortable with. What does innovation in content for law do? How does it look like?
Rittika Jindal: So I think we can look at it like today, most of the companies or more than enterprises are looking at building AI, building with AI, building AI. And our team also focuses on building agents or building these agentic workflows. you have, when you're processing content, you would have a pipeline that you will build to process content, to deliver content to a product. So our team takes care of enriching these pipeline processing.
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: this pipeline using AI, using the tools and technology that are available process and to enrich the content for a product.
Yoni Leitersdorf: Okay, so a year ago, yes, I got
Rittika Jindal: And I think... Yeah, I was just saying, just adding to this and what makes this really interesting is precision, ⁓ volume that's right? So when, let's say, ⁓ legal are using our products, they're using our products to file cases to their customers, which could impact their ⁓ lives, their careers, know, ⁓ all of that. So the impact is not about just delivering the content, it's about about delivering the content with accuracy, delivering the content faster, right? So our pipelines have to have all of these features that, you know, that takes care of the accuracy and the content currency as well.
Yoni Leitersdorf: So all those news articles we see where lawyers use AI to draft legal arguments ⁓ the AI quotes things that don't exist and hallucinates, cetera, you're helping them avoid that essentially, making sure that they don't make that mistake.
Rittika Jindal: Yeah. Exactly. Yeah, exactly. And this is not like a metric or a check that we have to have. It's like it's rooted in a product, right? It's like a professional trust bar that's built in into a product. And this sort of drives ⁓ that we do, right? All the designing, all the building of the product. Like accuracy is the key.
Yoni Leitersdorf: that makes sense. are, you a lot of times people talk about what's the level of accuracy that's required in different flows. You know, when reporting financials to the stock market, obviously, accuracy has to be 100%. When, you know, answering just simple questions may be less, but in your case, people's lives depend on this, and you need to make sure that you're accurate. So,
Rittika Jindal: Yeah.
Yoni Leitersdorf: Okay, so let's double click and zoom in a bit more. So you're using AI to accelerate innovation within that process of content generation. And you write lot online about the journey that you've gone through from, you know, a year plus ago ⁓ with OpenC2 you talked about at Gartner to ⁓ how do software teams like today? So maybe if I start with OpenC2, that was a year and a bit ago.
Rittika Jindal: Hey.
Yoni Leitersdorf: what was it? What was that project that you talked about at Gardner?
Rittika Jindal: Yeah, so OpenSETU, setu is a Sanskrit word. ⁓ means bridge. ⁓ OpenSETU is a data agent that helps the professionals in TR ⁓ ⁓ the data, right? So let's say if I am into marketing or HR or sales, ⁓ want to look at the data. ⁓ ⁓ now you would have ⁓ analyst in between, you know, so as a non-technical user, I would ask them the kind of data that I want. ⁓ these data analysts will write the
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: the Python code or the SQL code and then they will fetch the data, create these reports and then send these reports. So we wanted to bridge this gap so that our non-technical users can directly interact with the data. So OpenSETU was a data agent which allows a non-technical user to interact with the data. So you write a question in a natural language, what were my sales last year or how many employees are on leave this month? So you ask a question and converts this natural language question into Python or SQL depending upon the complexity of the question. So it has a list of tools at its access. So it writes the code and then it runs the code on the data and then gets the data the HR or to the marketing ⁓ So it starts to bridge the gap. ⁓ we called it the data analyst agent ⁓ it's like writing the code for you, it's bringing the data ⁓
Yoni Leitersdorf: And it.
Rittika Jindal: to you.
Yoni Leitersdorf: And we talk a bit about accuracy. is the accuracy of OpenC2? What was the accuracy of the tool? ⁓ did you measure it? Can you walk us through that a bit?
Rittika Jindal: Yeah. Absolutely. So it's a tool which is backed by the data. So the tool is not generating the data, it's generating the code for you. So the ⁓ generates the code and it runs the code. ⁓ then the runs on the data that's already there. ⁓ So you run the code on the data, it gets you the accurate data. ⁓ But didn't just stop and relied on how this code was generated. We of course went on and added ⁓ evaluation, We had the ground truth data. So we did a lot of evaluation about the data that it was generating at that time. like now ⁓ that we developed is sort of ⁓ driven and test driven. We make sure that we have the ground truth data. We are evaluating whatever we are building, starting from day one.
Yoni Leitersdorf: Okay, so essentially. you knew what the answers were to some questions. And then you said, OK, we'll run these questions. We'll see what answer OpenC2 gives us. And then we'll compare it to the actual answer that we expected see that they're the same, which makes sense. That's exactly how evaluations work. And we see it in a lot of places. Of course, we also have it in our own platform in Solid. But that's kind of the ⁓ as you're saying. ⁓ start from evaluation so that you can test that you can rely on this thing. ⁓
Rittika Jindal: Yeah.
Yoni Leitersdorf: So that was about a year plus ago. Since then, you posted a lot about how software development looks like And there were a couple of posts that ⁓ put on recently that I want to dig into those. So ⁓ one of them, you talked about how the work of software development has changed completely. The places that people are now putting their effort and emphasis in your own people, your own architects, your own software developers, have shifted their focus they're now using AI to write the code for them. So let's dig into that one first. So tell me a bit about what you wrote there.
Rittika Jindal: Yeah, think honestly it has changed even since the last time I wrote my blog. Like it has changed ⁓ much since and I think I need to write a new blog. But let's about that blog. I think it was about how our software engineers focuses more on ⁓ the planning the designing rather than writing code. So let's say would as a software developer, you would spend ⁓ 30 % of time designing and planning. But now you spend 70 % of your time on designing and planning where you're asking questions like, why am I doing this? For example, let's take a very simple example. You are converting, let's say, we are in content space. We do lot of formatting, a lot of conversions, et cetera. So you're converting from one format to another format. Earlier, you would maybe research or you will look about, how does this format work? But now our software engineers ask questions like, do I really need this format? So that's the design thinking that has changed in this life cycle. But you know, I think I wrote that blog maybe a month or two ago. Right now our focus has shifted, you know, slightly more in towards AI native engineering. So at that time we were focusing on using AI to write code to help us plan, implement, right? But now we look at this AI native engineering where we see the whole SDLC cycle is built using AI. Now, if you look at like the whole SDLC cycle, ⁓ you have planning, right? So you gather requirement, you do the plan, you do the design, then you develop, you test, you debug, you know, then you deploy to production, right? So it's like a full life cycle. So now we have AI at all of these stages, right? So we start with plan, you plan with AI, then you write code with AI, but we also now have built our skills, know, which allows us to test, which allow us to debug, right? So lot of us ⁓ our project, we use cloud code as our AI dev tool. ⁓ what we have done, ⁓ of course have tools or skills that allow us to write code, ⁓ the code, but we now all have ⁓ that helps us test the code. So we have built end-to-end testing using skills that allows us to test. So when I say test, I don't just mean the unit test or the integration test. I mean ⁓ actually the whole workflow ⁓ is built in Cloud Code, right? And then your logs, let's say we're using AWS, so our logs goes into CloudWatch. agents have LangFuse built in, right? So ⁓ we skills that gets us LangFuse logs. So everything sort of comes into the same terminal. We pull in all the logs onto the same terminal and then we debug, ⁓ like there is an error, we debug in the same terminal. So you see, that's how we bring the whole SDLC cycle into our same terminal.
Yoni Leitersdorf: So, okay, so let me see if I understood correctly. So let's say there is a type of content you wanna pull onto the platform. So your team needs to build essentially the pipeline that pulls it in, right? That pulls and adapts it and modifies, et cetera, and loads it into the platform.
Rittika Jindal: the papers. Yes.
Yoni Leitersdorf: Now, activity that, an item like that starts with, OK, we need a plan. We need a plan. What we're going to bring, ⁓ from what sources, how are we going to convert it, how are we going to prepare it for the designated platform, and of course, test it along the way. So there's the plan of that, of what we're going to build. Then there's the building of the code. So ⁓ actually writing code, of course.
Rittika Jindal: you.
Yoni Leitersdorf: Then there's the testing of it and to see is it working. It pulls the content. it, you know, is it making the conversions or the results the way that you expected? And if they're not the way that you're expected, how do you debug that? And your point is in every single phase AI is involved. So let's start maybe from the planning. What, what tools are you using? What's your methodology for using AI within the planning for, for night for a work item, like the one I just described.
Rittika Jindal: Okay, yeah, so if we start from, let's say, just plan, so all the developers ⁓ a plan document, right? So you write, ⁓ with Claude to create a plan document, ⁓ that plan documents get within the team, depending on ⁓ the of that story or impact of the feature that you're working on, right? So let's say if you are just fixing bug, ⁓ a work doesn't, the plan doesn't need to be reviewed by everyone, but if it's like a ⁓ bigger that you're working on, the plan needs to be reviewed. the plan reviewed by the reviewers or by the senior engineers. So now plan just doesn't sit in a software developer's laptop. The plan actually goes into a GitHub, right? So we have a where the plan sits. ⁓ ⁓ take the plan, you ⁓ the plan reviewed, then you go into writing code using that plan, right? So you have written the code. ⁓ you are going to create a PR ⁓ has your code and your plan.
Yoni Leitersdorf: Okay. Okay. Okay, so the original plan goes into GitHub, probably a markdown file, I assume. so it's a markdown file. Senior engineers review it. By the way, when the senior engineers review it, do they actually read through it or do they use their own AI to review the plan that you propose? Like, does that work? ⁓
Rittika Jindal: Yeah, that's a. That's a good question. think initially when we started, sort of struggled with, you know, like, do I start reviewing the plan also with AI? But over the time we have realized we have actually cut down what we write in the plan. So it's easy for me to like just quickly glance over it. So I usually read like, you know, maybe first 10 lines and I know what the thousand lines of code are about. Right. So the plan is too big, you would need AI to ⁓ faster. But ⁓ plan is like really ⁓
Yoni Leitersdorf: Yeah.
Rittika Jindal: It's really about what you are doing. So it becomes really easy to read the plan by a human as well.
Yoni Leitersdorf: Okay. Makes sense. I use Cursor a lot. I have it plan most features in advance and usually I read the top. I don't usually read the actual implementation. mean, in my case, the implementations are because I don't write production codes and my case, the implementation is less my interest, but it's more, it implementing the right logic or going to. So, okay. So they write the plan, plan gets reviewed by humans. Then the code gets written by cloud code based on the plan. And then when there's a pull request open, there's the
Rittika Jindal: Yeah. Yeah. Yeah.
Yoni Leitersdorf: code that Cloud Code generated, and then the plan that was agreed to in advance is also included. OK, so pull request gets open. Now, that you guys do is different. ⁓ now Cloud Code came out with the PR capabilities, then a lot more people are using it. But you started doing this before. And so PR gets open. What happens next?
Rittika Jindal: Yeah. Yes. Yeah. So now it's about reviewing the code, right? So we have two levels of reviewing the code. We have built an on-demand Claude skill that a developer runs when they are creating, or before they create a PR, they run this review skill, which looks at the plan, which looks at the code, which looks at the AI's law, which looks at the test cases that you have written. So it does ⁓ all of that evaluation. this is an on-demand run by a developer, right? ⁓ let's say as a developer, if you miss those, right?
Yoni Leitersdorf: Mm. Mm.
Rittika Jindal: also wanted to automate this. So we also have a Claude bot reviewer that runs when you submit a PR. So when you submit a PR, bot kicks in. It all of these checks about checking the plan, checking the code, checking the architecture alignment, all of that. So ⁓ it that and ⁓ the sits in the PR. So when as a reviewer, ⁓ I the PR, I see the plan, ⁓ I the code, but I also see the review which is done by this bot.
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: automatically. So now I have a lot of things at my ⁓ to review the code. I don't need to read every line of the code now. The reviews are much faster. I just read few lines of the plan, then I read the review done by Claude.
Yoni Leitersdorf: And then I think in your, in your post, you wrote that basically Claude picks up all the, stuff, like the, basic checks that you always have, it picks them up and immediately can push back on, on them. And then that allows the humans to focus on the higher level things in, in that work. by the way, if, Claude finds issues in comments on the pull request, does the developer then go ⁓ and their Claude with the comments that came from the PR ⁓
Rittika Jindal: Yeah. Hmm.
Yoni Leitersdorf: like how does that loop get closed at that point?
Rittika Jindal: Yeah, so when a developer creates a PR, the Cloudbot runs and it gives you the comments, right? Then the developer takes that PR, gives it to their Cloud and say, okay, go fix the comments which are from the reviewer. So it's mainly about how you use different personas of Cloud, right? So you have a developer has a developer persona, but the Cloudbot has a reviewer persona. So you get those comments into the developer terminal and they try to fix, they do a lot of
Yoni Leitersdorf: Mm-hmm. Mm-hmm.
Rittika Jindal: iteration there as well and then you know once they have fixed all the comments then the PR goes to the senior engineer.
Yoni Leitersdorf: Okay, now. So now what happens is PRs, the basic stuff has been flushed out and fixed. And now it's only the things that you need humans for. You wrote a tool that every morning you can see what PRs need your attention, right? I saw you vibe coded a tool for doing that. Why did you need that? Why not just go into GitHub in the morning and say, ⁓ okay, here are the seven PRs that are waiting my attention. Why you need this?
Rittika Jindal: you think there are a lot of things that I found I needed ⁓ at that time. It was about ⁓ do I rank different PRs, what I need to look at first. ⁓ in GitHub, you do see all the PRs. You can sort them by last modified, et cetera. ⁓ can see all of that. But what I cannot see without opening is the number of lines written in that, ⁓ of files changed in that, unless I click on every PR. ⁓ I wanted a dashboard to quickly approve PRs, which were short. ⁓ Like a bug fixed PR, I want to review it, know, like the first thing in the morning. So I built this dashboard that helps me, you know, look at, what are the top PRs that I can review while I'm having like a meeting, you know, in a, yeah, like I can do multitask with these tools. Smaller PRs maybe don't need 100 % of my attention. I can review them while I'm doing something, you know, attending a meeting or listening to something as well.
Yoni Leitersdorf: OK, at a, actually one thing I noticed, the more I use Cursor, the more I multitask. And I'm not necessarily very good at multitasking, but I bought another screen. So now I have my third screen here because I needed a screen for Cursor so that I can watch what it's doing. And ⁓ as on a Zoom, if I'm not the one talking, I can quickly take a look and give it another instruction, another instruction. So. ⁓ Sometimes it feels like my brain burns. I'm like, feel overloaded with all this multitasking, like talking to someone working on cursor, you know, handling things on the other screen over here. It's ⁓ feels feels like a lot. Does that happen to you at all?
Rittika Jindal: Absolutely. So I think there was a time when I was working on like four to five Git work trees. know, like, of course, like three different terminals, but also like in the same terminal, I have two, three different things going on. And I realized that I was mixing my prompt, right? So if I had to write a prompt onto the first terminal, I would rather put it in the second terminal, right? And I think over the time I've started to realize that I cannot work on more than three things and that's my limit. So now I have decided that I will not have more than ⁓ in parallel. ⁓ could be maybe two GitHub repo and then a meeting, but I'm not gonna do more than three things. ⁓ think that's a ⁓ or that's something as an individual ⁓ have to find for yourself, right? Like what's your limit? How ⁓ of multitasking tasks you can do ⁓ in go without burning out and without feeling ⁓ stress ⁓
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: say Tuesday or Wednesday of your week, right? Because you survive till Friday, right? So yeah, I think on every individual ⁓ find that bar, to find that limit.
Yoni Leitersdorf: Yeah, I'm probably on the two or three range too. I wonder if as you know, the next generation, so I have kids they're 14, 12 and 10. And I wonder if when they get into the work world, you know, the corporate life or whatever you want to call it. are they going to have more multitasking abilities than I do, for example, or than we do? Because if you think about it, if you took someone, you know, a generation above us to them, working on zoom, recording on Riverside, sending emails at the same time, what's happening with someone on another thing like to them that that looks like a lot. ⁓ And so and obviously the things that look normal to us today.
Rittika Jindal: Yeah.
Yoni Leitersdorf: looked like a lot to someone who worked in the space of generation before us. So I'm wondering if the generation that follows is going to be able to multitask a lot more than we can today. What do you think? I know it's very theoretical, but what do you think?
Rittika Jindal: really hard to speculate though, but think the next is even smarter than us. ⁓ I think find ways to... maybe not only multitask, but also do things that our previous generation used to do in a much better way. I felt like the generation before us, you said, didn't really do multitask, but they were still completing all the tasks that were required, right? And think the new generation is going to find ways to not burn out, ⁓ but still able to check all the boxes. ⁓ Yeah, I think I have high hopes on the new generation. And I think
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: they have ⁓ sort of clear boundaries ⁓ you like creating that for themselves. know, like if you talk to JSC today, they have a clear understanding of, you know, where they want to stop, right? How do they define the whole balance and...
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: although I don't know where it stands for me right now. I don't I feel like it's a mixture. Like on Sunday sometimes I'm thinking, ⁓ maybe I should build this to help on my Monday. And then Sunday night I'm sort of like, wipe coding, know, that ⁓ dashboard that talked ⁓ to you Now. I think the work that I'm doing on Sunday evening is going to help me, know, for the entire month. Right? So is I don't think that's too bad, right? Because if I spent like maybe two hours on Sunday, but it helps me ⁓ for a I mean, sure, ⁓ I can a relaxing month. I think that's okay, ⁓
Yoni Leitersdorf: Yeah, it's you're paying on a Sunday to be able to work maybe slightly less on a Wednesday on the following Wednesday, et cetera. Yeah, I wonder about that because I feel like at least in role, it feels like we're ⁓ on treadmill that keeps running faster and faster. Like somebody keeps pressing the up button ⁓ and.
Rittika Jindal: Yeah. All the Yeah.
Yoni Leitersdorf: And what you do is you try to do more, do faster by, you know, by building all these tools for yourself. Like there's literally decks, for example, for solid half the board deck is built by Claude, ⁓ because there was recurring things and it's okay, go and, know, pulls it from our CRM and from our Slack and our email and Google drive it on like it does prep work. So it's clear time for me.
Rittika Jindal: Mm.
Yoni Leitersdorf: But then what I did is I filled that time by other things. I don't know if I'm doing more or, I don't know. It's definitely interesting. ⁓ guess speaking about doing more, so now the development is done with AI. So the planning, the coding, the review, everything is done by AI.
Rittika Jindal: Yeah.
Yoni Leitersdorf: Have you measured the change to the results of the team? So are you delivering more work items as a team? Are you delivering them faster? ⁓ you delivering less bugs? How you measure all that?
Rittika Jindal: So the dashboard that I built earlier was not only to help me the PRs, but it was also about measuring ⁓ number of PRs that were approving and merging. So wanted to measure whether our has really increased. Are we merging more PRs? we pushing? Are we really faster? ⁓ So that dashboard tell me that before, I think if I looked at the previous repos, maybe six months, back when we were maybe not AI native engineers. looked at some of the repos where ⁓ data showed me that we were merging maybe ⁓ to six PRs ⁓ a week, right? And that could, the of ⁓ lines the number of code could vary, you know. So I'm not gonna go into ⁓ the of those PRs, but that was the number that I saw. ⁓ But we are more than that every day. data shows me that we are ⁓ 5 to 10 PRs ⁓ single day. ⁓ scale has changed ⁓ think we are ⁓ faster, we are developing faster. like the shows that. ⁓
Yoni Leitersdorf: Mm-hmm. Yeah, we've been experiencing a similar thing. So our sprints at Solid are three weeks long. And I listened today to the sprint demo that we had at the end of the sprint now. And the VP of Engineering said that we merged 170 PRs over the last three weeks.
Rittika Jindal: Hmm. Hmm. Hmm.
Yoni Leitersdorf: So call it 50-ish PRs per week across a team of 20 people. 2 and 1 well actually it's less than 20 people. So let's say 2 and 1 half to 3 PRs per week per person, which is the most we've ever done. That's the highest rate. Now the sizes of these PRs are the same sizes of the PRs we always had even a year ago. The number has increased. The number per person has increased.
Rittika Jindal: Yeah. Mm. Yeah. Yeah.
Yoni Leitersdorf: And I also listened to ⁓ Lenny's podcast and he was talking to the guy who built Cloud Code. And that guy said that the number of PR, basically I don't remember exactly the number he had of PRs that he ships, but we did the calculation and he ships 30 times more PRs the non AI assistant software developer, 30 times more. ⁓
Rittika Jindal: Thank there. Yeah.
Yoni Leitersdorf: So is that where we're going? Just talking about the whole, well, our previous conversation of what we're doing and balancing. So is that what's gonna happen? So the expectation from software developers will be, the standards will be, okay, you need to 10, 20, 30 times of what you would have shipped five years ago. Is that what you see was gonna happen?
Rittika Jindal: think it's already happening, right? That is the expectation we have today. you have all the ⁓ tools at your disposal. Now you are shipping faster, you are writing the code faster, so we are delivering faster. Like it's not something that I'll ⁓ an year from now, I think it's something that we are asking today. Yeah, and the data shows that we are actually doing that. Like you also said that you have 50 PRs a week. ⁓ the current stat that we're talking about. But also feel that.
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: Over the time, it's not just going to be about writing code. It's going to be about how you the whole life cycle around this. So when you look at a product development, it's about ⁓ the requirement. It's about writing the code. It's about testing. It's about evaluating. I think ⁓ focus right now is on the testing and is on the evaluation. So we are focusing a lot on EDD and TDD, where we say that ⁓ write the test cases, you understand the evaluation, you understand the ground truth data, so you understand what you're really going to build and then you make that a part of the development. So now the focus is more on, we are using tools like LangFuse, we are using tools like BrainTrust to ⁓ evaluate, to hook the ground truth data to review the output these agentic workflows are producing for us. So going to become ⁓ a key metric ⁓ for this year.
Yoni Leitersdorf: So. when you're measuring the productivity of the team. So there's the number of PRs, there's the quality, the EDD, the evaluation-driven design, and then the TDD, the test-driven design. So that helps you validate the quality of it. Do you find that AI is creating slop ⁓ a mess in your code that you need to clean up over time? Are we paying something on the other side where the AI is creating more work for us somehow?
Rittika Jindal: ⁓ Yes and no. Like today, yes, we do see a lot of AI slop. We see a lot of dead code, especially when you have a lot of people using AI, right? slop started to vary, right? Let's say I worked on a story and then somebody else is... adding an enhancement to it. I could created a slop. I created a dead code. Now other developers or other engineers Claude is not going to read the code that's not working. It's going to at what's working and then build on top of it. And it's going to create its own slop. But I also feel that with the evolution that's happening in the base models, like we now use Opus 4.6 or Sonnet 4.6 to write a code. And the way I think Anthropic is shipping you know their models and shipping cloud code the enhancements I think all of these will soon be taken care and it will be think of the past as well like today we look at the AI slop today we look at the dead codes you know all the unused imports etc and we have added these into our review skills you know which reviews the code so a lot of this is is done automatically like we don't have a human checking AI slop it's done by AI but I think over the time This will be built in into the cloud code to make sure that you know, it's not generating any slop It's not generating dead code as well
Yoni Leitersdorf: OK, so the dead code in the slop is not a big issue and will be solved on its own. What you care about more is testing the code functionality that it works. And if it works and it passes the evals and the TDD, et cetera, then some slop is acceptable because it's just part of it and you clean it over time. Is that a right way to think about it?
Rittika Jindal: Yeah. Yes, and I think slop was always there. Like even when humans were writing code, we can say, right? Like we didn't write slop free code. Slop has been there. Maybe now the pace at which the slop is written is a little higher, but slop has always been there.
Yoni Leitersdorf: Yeah, true. Yeah. Yeah. Yeah, that's true. I, My code has been, it was always messy. My personal code in my previous startup, the VP of engineering, who's the same guy, the same VP of engineering here at solid. name is Alon. He barred me from writing code. He said, you can't write any more code because your code is a mess. You don't follow guidelines. You don't always have unit tests and stuff like that. So he said, you can't write any code anymore. And I respect that. ⁓ So yeah, was probably a ⁓ human slop So. Okay, I listened to you talk at that talk a year ago. Now, can you imagine what your talk would be a year from now? So let's say March 2027, you know, it was from a, you know, data analyst, automated AI data analyst to now code is being written by cloud code and humans are only involved in the evaluation and checking the plan and all that. A year from now, if I had you here in the podcast again, what are we going to be talking about?
Rittika Jindal: Mmm. Well, it's really hard to imagine, I think, the pace, the things are changing ⁓ week. It's really hard to imagine. if ⁓ I to, think... So, ⁓ you know, try to connect a lot of work we are doing with ⁓ how we doing maybe a decade back or five years back, right? ⁓ So ⁓ we at decade back, there was a called RPA, Probiotic Process Automation, if I'm not wrong.
Yoni Leitersdorf: Mm-hmm. Yeah, robotic process automation.
Rittika Jindal: Right? Right, right. So when we started looking at RPA, it was about automating all the manual things that we were doing. ⁓ We started evaluating whatever we were doing manually and it was a process to automate a lot of these things. when I was working at that time on RPA, ⁓ idea was that anything that you are doing more than five times, it to be automated. You shouldn't be doing it manually. ⁓ So was a ⁓ basic ⁓
Yoni Leitersdorf: Mm-hmm. Mm-hmm.
Rittika Jindal: And now I think that has changed with the AI, with the AI tools. I think it has, the number has reduced from five to three or maybe even two. Right? So anything that you're doing more than three times or more than two times, it should be all done by AI. Right? So that, that number has changed. So I think that maybe is still going to evolve over time. that I'm doing ⁓ than two times a day, I'll put that in an agent. or in an automated AI system that will do. And then I think I've also talked about how this whole SDLC cycle is AI native. ⁓ we are talking about ⁓ just AI engineering, we are talking about ⁓ native engineering, right? So the freshers or the college grads who are joining ⁓ companies, they are I feel like they are more AI native than we are, right? because they don't know how to work without AI. So thinking process starts with AI. I think they're gonna come up with more and more ideas and more and more innovation ⁓ not ⁓ SDLC, the whole of how we think of a new product, how we come up with an idea of building a new product. For example, now, ⁓ I look at, okay, these are the problems I have at my hand I need to build the solutions ⁓ automate these using AI. ⁓ think after year, the problems are also going to vary. The problems are also going to ⁓ change lot and the way we ⁓ build is also going to evolve. ⁓ I know if that answers your question. ⁓
Yoni Leitersdorf: Yeah, it basically you're sort of so what you're saying is let's talk about the fresher the the those fresh out of college. I can maybe equate it to a world where workstations, personal computers, etc. entered the corporate world. And then in parallel, they entered the college and university world, academic world. And at some point, those coming out of college were already well versed in using a computer. And so they could do stuff with it that maybe the people that were working back then didn't know how to do or didn't think about doing because they grew up in a pre-computer world, pre-personal computer world. And then they adapted. those freshers back then could do things or think about things that the current workers couldn't. And so what you're saying is now there's going to be a point where people who come out of college, they did their entire computer science degree, let's say, or whatever degree using AI. They're so used to it when they enter the corporate world. they will put it to use that we can't even imagine. Like we can't think of right now. They will do it because it will be so native to them. So that's interesting. That will make it very interesting, obviously. There's a lot of talk about how those fresh out of college are having a hard time finding a job because a lot of what they can do...
Rittika Jindal: Yeah.
Yoni Leitersdorf: AI can do. So how do you reconcile these two things? On the one hand, the freshers will bring in new ways of doing things that we can't think of, but on the other hand, nobody's hiring them because AI can do the stuff that they can do. So how do you, how do you reconcile that?
Rittika Jindal: Yeah. I think in my personal opinion, the profiles, the roles that we are hiring for are also going to change time, ⁓ upon the problems that we are solving. ⁓ ⁓ year, was checking how, when I hired a developer or an engineer, was checking how are writing code. But today, not looking at how they are writing code. I'm looking at how they are thinking about ⁓ solving this problem. ⁓ So the is about ⁓ what kind roles we are looking
Yoni Leitersdorf: Hmm.
Rittika Jindal: looking at what kind of people we want to hire, I think that's also going to evolve. So let's say today we are hiring software developers. Maybe five years we'll hire software thinkers, you You never know. I feel like, you know, ⁓ ⁓ the kind of work we do is changing. ⁓ the profiles are also going to change and, you know, the way we hire ⁓ grads are also going to evolve. ⁓
Yoni Leitersdorf: Mmm. Yeah, you're right. I mean, we're not going to need developers anymore. We'll need thinkers, planners, dreamers maybe. ⁓
Rittika Jindal: Yeah, absolutely.
Yoni Leitersdorf: Yeah, it's a good point. mean, the profile will also change. I think for software development, you need a certain profile, like someone who's really good at math and their brain works in a very specific way, et cetera. But if you don't need that anymore, then maybe you have other people in the creation of software, you know, with other skills. It'll be really interesting. So I guess your point is...
Rittika Jindal: Yeah.
Yoni Leitersdorf: We don't know. A year from now, you don't know exactly what we're going to be talking about. you can see some trends and take some bets, but it's very hard for you to put the finger on it, which I agree. I can understand that, how complicated that is.
Rittika Jindal: you Yeah.
Yoni Leitersdorf: Okay, so that was a really interesting conversation. So we're nearing the end of the episode. And at the end of the episode, I always ask a few questions. So of them I basically did, you know, where would be a year from now? Usually I say five years, but where would be a year from now is a good one. And I probably know the answer to this, but ⁓ what an AI tool you can't live without?
Rittika Jindal: I do want to say cloud code or cloud.ai. Yeah, think today, yes. ⁓
Yoni Leitersdorf: Yeah. Yeah, I assume that was the answer. What is an AI myth you'd bust in one sense? Is something that people are saying about AI that you think is nonsense? What do think that is?
Rittika Jindal: Hmm, I'll have to think about that. I do want to say that the myth, no not a myth, that might be true. I don't know. So when I talk to a lot of people, think, who are trying to upskill in AI, ⁓ find it hard ⁓ upskill. think that's a myth like upscaling today is quite easy. We have so much content, so many tutorials available at our disposal to learn and to upscale. I think it's just about ⁓ ⁓ path, finding what you're looking for. there is just like even if you look at ⁓ OpenAI's you look at Claude's tutorials, Anthropic has published like an Anthropic Academy where you can learn a lot about ⁓ how we are using tools. So I think upskilling is not as as it may seem. ⁓ Like today, think upskilling is quite easy. ⁓ If have the ⁓ right if you have the time, ⁓ think ⁓ upskilling easy now.
Yoni Leitersdorf: Yeah, I hope everyone has the intention to do it. But yeah, I see your point. A lot of people were talking about how hard it was for them to learn AI, but basically you're saying there's all the tools now. There's no excuse not learning how to use these tools. Amazing. OK, ⁓ is there source? So first of all, for the listeners, if you're not familiar with Ritika's ⁓ medium, so ⁓ RIT.
Rittika Jindal: Thank
Yoni Leitersdorf: Ttikajindal.medium.com. So I really suggest following her there and reading ⁓ content or finding Ritika on LinkedIn and her there. But other than your medium, what is the source that you recommend for people to read? ⁓ Podcast, book? that you recommend people to read. And it could be an AI, and it doesn't have to be. It could be in a completely different space.
Rittika Jindal: I think there are several tools, one is you can just. ⁓ open any tool, any LLM, you can just start asking questions to LLM and sort of prepare about the topic that you are interested learn about. ⁓ also, I think there are already so many tutorials. If you look at ⁓ Engie's ⁓ ⁓ are just so many free courses available ⁓ anybody can learn through. ⁓ ⁓ say ⁓ you want to learn about how ⁓ is working,
Yoni Leitersdorf: Mm-hmm.
Rittika Jindal: you can look at Anthropic Academy. If you wanna learn about how OpenAI is building or how prompt engineering, all of that, I OpenAI also have their own learning documentation. ⁓ I would say if you wanna start from ⁓ maybe start by looking at the documentation that's published by all of these enterprises, like the foundation model enterprises, ⁓ then you can also learn from deeplearning.ai. There are ⁓ like plus courses. And I think Andrew Ng really picks up like, know, so I think last year he created a course which is about writing code with AI, I had not seen before. Like we were all doing it, but nobody was creating a tutorial or a course about it. So I think ⁓ a really interesting ⁓ you know, for ⁓ like a product manager or a product owner ⁓ just see, you know, how developers are writing code with AI. I ⁓ Andrew is creative.
Yoni Leitersdorf: Mm.
Rittika Jindal: creating like courses for all of us. So ⁓
Yoni Leitersdorf: Yeah. I agree. took a couple of the courses on his site. They build it really well. It's really easy to internalize and to understand. So I agree with you. OK. Amazing. Ritika, I'm so happy we got to do this. ⁓ I'm really actually looking forward to doing this again a year from now ⁓ seeing how things changed. Because obviously, things change a lot. And they change fast. They change quickly. But really, thank you very, very much.
Rittika Jindal: You
Yoni Leitersdorf: for being here Riddika and for those listening to the podcast please remember to like, subscribe, thumbs up, whatever may be on the platform that you're listening to this to and we are gonna see you next time. Bye Riddika!
Rittika Jindal: Thank you, bye bye.
