UTSKRIFT 🔗
speaker-0: This new model promises 440x cheaper and 193x faster for similar use cases in ejectic workflows and development. Jev just came out of stealth with a differentiated system one architecture aimed at beating OpenAI and Anthropic for use cases requiring state and downstream probabilistic confidence in your software application layer. If you're building with models, you'll want to hear this because at one four hundredth of the price, this is a complete game changer. Welcome to your weekly AI Clarity podcast with your signal from the noise for people building production level AI and building with AI inside real companies. I'm Robert, CEO and co-founder of Clarity by Epistemic Me, joined by my fellow podcast host and co-founder and CTO, Jonathan McCoy. On this podcast, we typically go over the latest AI news, some funny AI memes. Please submit some if you find some funny, we'll host them on the pod for some laughs. And then current what's the meta sort of updates for AI builders, your latest tips, tactics, tools, and strategies that we use ourselves on the front lines of building our AI native startup. So today we're going to focus on TypeSafe's JEV model, which introduces a new type of architecture called System One. Let's go ahead and dive in. We're going to read through a few AI news items and then get into the meta of JEV. ⁓ and we did have a funny tweet ⁓ that Jonathan shared around Michael Levin. I thought it was a pretty funny tweet ⁓ this past week. That Brian Johnson ended up co-signing Michael Levin's ⁓ work around just aligning everything in the body towards longevity and his further work on goal alignment and predictive processing. So we he is a friend of the pod. He doesn't know us, but he's a friend of the pod. A few things in the news. So TypeSafe came out of stealth on September 15th with $40 million led by DCVC and a model called Jeff. The description: a new class of frontier models built to make fast structured decisions that software can use directly. More on that in a moment, because it's the rest of the episode. Claude Code shipped two things recently on September 11th: plugin evaluation. You could write test cases, run your plug-in against them, score the runs. etc. Gary Tan says either you die a system of record or you live long enough to become a domain-specific harness. What does this mean for AI builders? What this means for AI builders is that storing data, even in compliant ways, is no longer going to be the moat if we take the time slider down 10, 20, 30 years or you know, to just infinity. Like the real problem we're seeing solved is how do you drive value with this intelligence? And Intelligence is coming for your domain and your domain-specific problems. So Gary Tant's tweet here really speaks towards everybody should be building domain-specific harnesses. And this was a retweet on Jesse Huang. I was just at a YC demo yesterday besides hardware physical things, which obviously has a moat. You got to go figure out your supply chain and distribution for hardware physical things, very hard, capital intensive. Everyone is just basically just building a domain-specific harness. We see this in legal in Health and et cetera. That's that's a thing. So would love to know what the audience is doing in terms of your own domain specific harnesses and whether your company thinks about it that way, in particular the decision makers behind budget and resources. Because that's certainly the way the industry, the most disruptive edge of the industry is going. I think that in regards to our business model, so when we to try to meet the goal of, hey, we're AI infrastructure for serving right context, right time. centered around the concept of a digital twin of your company and your company worlds model based on its people and its humans and agents that make sense of the world, channeling that signal into from demand signals to like what should we build your asset and then go into market with that. And that would be like combining the customer world model too. So I kind of see it as if we're building digital twins to help businesses make better decisions at the micro and the macro level. To improve their bottom line, essentially. ⁓ I think we're in the business of helping businesses create their own domain specific harnesses better than their competition. That's how I see it. In regards to what is our harness, what is our moat, I think that is something we're still on the quest of discovering. But I believe what's settling in the dust for me as of right now, and I would love to hear more from you and for the audience, we we totally build in public. So these types of things are the things we're happy to talk about and share our thinking. And feel free to join us in our community towards this goal. The thinking really to me is can we help with the alignment problem overall? And the alignment problem, I really like this framed recently by Alex on the Moonshots podcast, which is alignment is just a capability. Like alignment just means better capabilities. Meaning the more you can understand an individual's goals, which we spoke to in the last podcast referencing Dark Quesh's ⁓ you know, kind of fear-mongering AI civilization essay, the more you could model an individual and understand their goals, our thesis. The more aligned the next agent call is going to be towards that human's goal. And if that human's goal is to further a business direction or company direction, hitting the bottom line or furthering the product strategy and the roadmap to capture market share, that kind of thing for software product teams, then it's how do we, clarity, build you, you know, customer, company world model and customer world model based on our very opinionated take on subjective self-modeling or this digital twin concept. ⁓ grounded in self-modeling and principles of neuroscience and computational biology. How do we help you do that faster, better, deeper than the competition? Anyway, so
Load full transcript
for us, I think the time slit are taking that to infinity. We then should have the best belief specific renderings of real subjective people across as many domain spaces or industries as we can. And that would be our mode. And I think we're starting in areas of high degrees of subjectivity, such as employee experience. When am I next gonna get paid? When will I get a promotion? ⁓ how should my next performance review cycle affect the rest of my career? And how do I think about my career? To things like health and spiritual wellness, like love and finding love. ⁓ that's that's another customer of ours, to things like bladder health, like how do I handle my bladder? So those are the spaces we're in initially. For the audience, and Jonathan already knows this because he actively works on client work with me. But for the audience, that's how I think about our moat. And again, like I think we're still questing and figuring it out, but directionally some some dust has sort of settled for me there. Yeah. What do you think? Subject to change. Think lots of things changing in the in the landscape.
speaker-1: Yeah. I mean, I think you just described ⁓ AI native services model, which is what our business model is. And I think what we're seeing at on the Y Commoner demo days is companies that are specifically becoming domain specific artists. So sort of maybe graduating out of, you know, AI native services models, finding a domain and then making like a venture scale thesis out of it. And so how does that impact the competitive landscape for us? And I think you're saying, well, you know, we're Really going after the domain specific areas that have ⁓ a lot of subjectivity involved. So we can protect you know, we can create a competitive advantage there. I think the question is like, you know, how long is a non domain specific business a viable business model? Or, you know, like at least at least something that could get to scale. You know, when we talk about something that could get to scale. That's great.
speaker-0: question. Do you have fears of like in a year, if we're if we keep doing what we're doing if in a year we'll be dead? Like do you have those fears that drive this?
speaker-1: I kind of feel like doing anything for the same thing for a year right now is gonna die. You know, like the world's changing so fast. So like in what ways should we adapt or kind of the and I think our our overall like motivation and purpose is the same. Like so it's how do we bring like the subjectivity, you know, to a ⁓ a business model that can scale is like, you know, really the open and evolving question I think is really interesting to keep an eye on. Yeah.
speaker-0: That's that's essentially our PMF, like where we're trying to figure out PMF is where does subjective modeling of the self and then additional sense making to inform probabilistic confidence updates around what does this person believe as they flow through the user journey of acquisition top of funnel in
your marketing stack all the way through to your software application layer? That's been our problem so far is how do we model your user as they go through all the different flows that make up the journey of getting from they know about you and then all the way to they're loyal to you and LTV should go up from there. Right. And so for me, one thing to answer this more concretely is I try, especially after listening to the latest Grockbot folks like talk about their design philosophy and build philosophy. They just talk about one thing like how did how did Cursor become successful even if Cloud Code came after them and then like Codex came after them and stuff. There's there's one narrative which is like, are they successful because they got acquired? But it's like, all right, a lot of people, a lot of people would appreciate a $60 billion exit. Not not a bad, not a bad deal, right? Like I think that would be quite a successful thing. So let's say they are successful. And ⁓ if you use that framing of like, all right, yeah, they were able to compete against the best, they still are winning deals, as I'm aware of in enterprise against OpenAI and Cloud, and they're trying to differentiate on things now like Rockbot. So You're right that it's hyper competitive. I also think you're right that maybe doing the same thing over and over for a year, especially with new information and landscape changing all the time, doesn't make sense. But the creators of GrockBot who have launched this wildly successful thing, and now I see Facebook Muse is coming after it too, because they're like, wait a second, personalized agents for the everyday comment task makes a lot of sense. It's creating a new market. And my take on this is they had this core design philosophy that was able to disrupt something, even though they don't have the frontier intelligence. Like Rockbot doesn't have the frontier intelligence that OpenAI or Anthropic do. Like those two teams have the best frontier intelligence, the best frontier models. But what they did have was this obsession with solving a useful problem today. And I think that's what we should focus on too. And currently the problem we are trying to solve today and successful at solving today for two categories of customers, to be really clear, is B2C, chat licycle, spiritual wellness health app. And then ⁓ that that's like a forum factor, smaller enterprise. And then larger enterprise, like $2 billion-ish ⁓ human capital management employee experience tech platform, where they're trying to figure out who is this person so that I can inject the right context at the right time to any of my agentec experiences to nudge them along from you were acquired in my funnel, you knew about me, you picked me to solve the problem, and now you're loyal to me, I've retained you, and you'll tell other people and for other people increase LTV and our bottom line. So in areas where right now it's employee experience and spiritual wellness and health,
like I think those are the areas that we could speak to most. And hopefully as we build out these case studies that are baking, we'll ⁓ attract more prospects and customers ⁓ along those lines as a market wedge and beachhead to stand on. And hopefully in a year, you know, like I I'm sure my thinking will evolve in the next year, but we solve useful problems for the customers we have locked in today, get great testimonials, keep compounding that. One other way that I think about it is there's that that's the product side of of moat distribution ends up being emote too, which is can we gain so many successes that that reduces the cost drastically of acquisition? And then hopefully, you know, our customers that like our company brain ⁓ that we provide them, they find that really sticky and they're like, ⁓ this is great. I don't really I'll pay you to just maintain it for me ⁓ and evolve it for me. And that helps all my decision making. That makes me money, right? And We'll see from there. But that's the general where I read the tea leaves. Curious what you think, Jonathan, about that. I know we're diving we dive we kinda like tangented off of AI news ⁓ because of this this is an interesting conversation. But yeah, go ahead and then we'll move on to the next.
speaker-1: Yeah, this is what happens when we don't like do these things every day. Like you know
speaker-0: Yeah, dude, we have too much shit to talk about.
speaker-1: Well, the I guess the cursor example is one where they like they're actually a domain specific coding harness, right? That's why they got bought by SpaceX and that's why they were able to make Rockbot, you know, on top of their own trained model. And so in a lot of senses, they're the flagship of like what a domain specific harness can turn into, right? And I think that's that's one of the reasons that you see YC companies going this way. And I think this like relates to another story that we had queued up, which was Warkesh's ⁓ paper about Whether it's actually is is it data or is it model capabilities that actually gives the net capabilities in that we're seeing from the model evolution? And cursor was an example where that, you know, having traces of people doing coding got them better performance results than than even some of the frontier labs could get. And so then this also just like goes right back to the story of like why domain specific harness? Well, you can get training data and you can build better model experiences for your customers and in ways that even the frontier models maybe can't touch. And I think that's like, you know. A strong business model as far as we can see in the future right now.
speaker-0: I guess one thing there that if we learn from the cursor story, ⁓ they had an advantage in experience first, right? They were like, ⁓ let's put this together in a form factor that developers that like IDEs would enjoy. That like being in the code still, they feel like they want higher agency of the thing. This is
pre-cloud code, pre-goal loops, so to end of what, 2024 or something like 2023, right? Like feels eons ago. But if we think back then, you know. That's what they were doing is differentiating on design experience and that philosophy of solve a useful problem today. So every executive in America still is and were telling all their engineers, go fucking use AI, guys. And they're all just spraying and praying, right? And then Cursor came in and was like, okay, here's this tool for all the skeptics. It looks like Visual Studio. And I'll tell you, like at Day Force, they tell they when I was still there, they were like, ⁓ yeah, like cursors, wake like a lot of engineers like cursor first. And a lot of engineers, I believe, still like Cursor first because it allows them, like Hamill, ⁓ Hussein, friend of the pod, tells me when he actually has to look at code and do real engineering shit, he does in cursor. So that they've won some brand loyalty and reputation there. And I think to your point, they did a really great job entrepreneurially at building a moat on top of that, where they're like, okay, solve the next problem, you know, which is let's fine-tune our own stuff to remove dependence from the frontier intelligence. And then okay, like what do we could we do from there? All right, another design experience problem that. other people are sleeping on grokbot found that and then I'm sure they'll build something from that s foundation ⁓ that they've already built with like cursor and then B2C and B2B sales for enterprises and consumers and then now grokbot which is gonna be another bottom line add to both those streams. So yeah, I any any reflections on that before we move to the next ⁓ segment.
speaker-1: I think it's about right. You you know, you build something that's valuable to your users, they they interact with it, they share your their data with you because they're getting value and you know, you have a domain specific harness on your hand. I that's the the life cycle.
speaker-0: And then you get well, in particular, then you have money to then go do more things and expand ⁓ and cement your lead in a way. ⁓ which is, you know, what we're really trying to say with domain-specific harnesses is getting niche data sets that these frontier labs cannot get access to ⁓ from your little island there. I'll just gloss over this ⁓ in terms of AI news. Dark Ash Patel published a decomposition of where pre-training progress came from between twenty nineteen and twenty twenty five. Breaking down six years of pre-training progress into data versus model improvements. At a fixed compute budget, data improvements produced a twelve fold efficiency gain. Model improvements produced three point seven. Eighty eight percent of the benchmark variance is explained by the two effects added together. What does this mean for AI builders, Jonathan? Really quickly.
speaker-1: Yeah, I guess it means more for ⁓ model builders that like the advances in, you know, training environments and training data that have accompanied the advances in the the models themselves are are responsible for like more of the gains, which is ⁓ is a really interesting finding to do a meta study like this and and find that heuristic. I wouldn't have thought it would be so out of skew like that data would weigh so heavily. And I also think that this is like a you know, we talk about alignment, we talk about ways that the models can go rogue and we we actually are empowered to create data sets, train them, you know, ⁓ as as both contributors and ⁓ investors and builders, you know. So it's a point of view that will serve you well to not forget.
speaker-0: All right. On Vercell AI Gateway, open weight models hit a record 78.4% token volume on September 18th, while roughly 64% of the dollars still went to Anthropic. Deep Seek shipped version 4.1 Flash on September 10th. If you take a look at this, the dollar spend says that number three and four on the AI leaderboards here are Moonshot AI and DeepSeek. If you add in Z AI, their combined dollar spend surpasses number two, which is open AI. That's pretty crazy. And it shifts an a slight inflection point that I've been hearing from the all in podcast hosts where they're they keep saying, like, dude, next thing is just gonna be open source models, fine tuned for your use cases with domain specific harnesses. Because why? And then and then shift and allocate a certain amount of compute to the frontier intelligence where you need it. But For everything else, just do this. And here's some data to back that claim. And you could go to the AI model leaderboards here on Vercell.com. We'll drop it in the podcast show notes. And you can actually see the top ⁓ models, labs, apps, and providers on AI in terms of usage, at least on the Vercel platform. If you trust their data and all that, you know, there's other platforms as well. But this is a good sampling, I'd say. Jonathan, from your CTO perspective, what are your thoughts on this open weight, open source versus frontier? sort of battle that's going down and this inflection point in the data we're seeing.
speaker-1: I think you're right. The you know, the data point you should fall in. You know, I think we're gonna be using routers and like, you know, routing tokens to open source models that'll just be part of our stack and set of tools that we use. That seems pretty inevitable. The lead that they kind of buried here was that ⁓ the Deep Seek V four one launch this week seems to be very popular. And it seems to not only like, you know, I th I think about it. Well, is this just like one or two companies, yeah, like pretty much took over the chart, right, this week and So you see like less of the other tokens being used and then like more than ever before tokens being used with DCP41 Flash. And one thing it's good for the cost is low, the caching performance is very good. Token reuse is like you know part and so what it allows you to do is run these
long-running tasks like a goal loop or ⁓ long-running coding agent at a fraction of cost of what you pay OpenAR anthropic. And so I think you got companies, builders like us out there testing and just like saying, Let's just set the agents on autonomous and have them build like infinite software now. You know? I think that was the unlock this week that the open open source model was good enough to to try some of these experiments. And you know, I think the Vercell data probably isn't representative across the board of builders, but it is a specific type of builder and the kind of tools they're picking up and using. And I think this is a like signal to latch on to that's ⁓ really interesting.
speaker-0: Shout out to Vercel for making this data public. AWS Bedrock, Google Hub platform, Azure, please do the same. Render, go ahead. I I'd love to see all that data. It is funny what Jonathan says here. If you look at this chart, this is token volume, share of AI Gateway token volume across the top models on Versailles AI Gateway from June 22nd, 2026 to September 19th, 2026. That is the past three months. ⁓ for the past three months, just to clarify what Jonathan said, which is this model just came out this week and already it's dominated sixty-four point eight percent of usage for the past three months. And and he's Jonathan's saying from his engineering AI builder perspective, that just means that hey, this is maybe an inflection point that folks have figured out and trusted the competence of Deep Seek v four point one flash, in particular on this chart, to run longer horizon autonomous goal loops. As opposed to just running them on frontier intelligence. And that could be the reason for this increase, drastic exponential increase in usage here. So this could be we could be looking back at this in ⁓ next little three months and be like, ⁓ my gosh, like this was the start of when the meta became, ⁓ yeah, just run the sh run all your long-ranging shit on on on the open weight models with your own harnesses, guys. Just that's just what we do now, that's how we roll. So For folks out there that are tinkering and trying to get the edge on the competition, that might be something look forward to. Is more open weight model usage and tuned with your own domain specific harnesses on long horizon tasks for your own problems. Let's stop on the Michael Levin meme. ⁓ this is from Packy McCormick. The race for the most important results in science is a two horse race between LMs backed by trillions of dollars, armies of geniuses. And the full weight of capitalism. And on the other side, Michael Levin. My money is on Michael Levin. And this is in response to Michael Levin's tweet. Here it is, the official revised peer-reviewed version of my Platonic Space paper. Of the all the many unpopular positions I've taken over the decades, bitter controversies around the origin of left-right asymmetry in embryogenesis. I just I just learned how to say something new today. Bioelectricity and genetics, diverse intelligence, etc., this one has by far generated the most pushback. Serious, grateful for those, and energetic attempts to move me to other views. Please to just drop it and not talk about it for just several different reasons. Nasty emails and accusations, impacts on reviews of papers that have nothing to do with this, etc. Kind of amazing to me how incendiary this is. What can I say? Our job is to call it as we see it, and right now, for me, this is it. Apologies to collaborators and colleagues for any shrapnel. Time will tell if this pans out or not. I've placed my bets. And by the way, if you think this stuff is weird and uncomfortable, just wait. There's much more on the way. The knob turns slowly, but as long as the data keeps coming, I'm going to say what I think it all means and follow it to the next steps it enables. Buckle up. I love this guy because he's just trying to find truth and put it out there. And you gotta you gotta commend that. You gotta commend that. ⁓ So this is the meme though. That was a more serious like paper drop. Go check it out. We'll link it. But this is the meme. Where is their truth? And then I also wanted to share. Jonathan had a really good tweet on this. We're headed for a dual singularity, and recursive self-improving embodied cognition is going to be much more important than recursive self-improving LLMs. Jonathan, say more on recursive self-improving, embodied cognition.
speaker-1: Yeah. Well, I guess, you know, everybody's probably most people have heard of RSI, recursive self-improvement. We're on the precipice of that. And and Anthropic even released data data saying, you know, like up to 20% of their 25% of the research is done by recursively self-improving agents now. That could be a hundred percent within the next year. So but what Levin's really talking about isn't body cognition. The problem with the RSI from LLMs is that you still have benchmarks. So you just have this RSI that's like benchmark mapping and it can't define its own goals and doesn't have its own agency. What embodied cognition tells us is how agency derives. And that's what Levin's paper really gives a foundation for. It uses this concept called the platonic space to map out what is a cognitive light cone for embedded cognition. Levin's giving us the way this is how you define the goals. This is how agents define goals. This is how agency defines goals. And once we understand that, we'll be able to better inform benchmarks that are subjectively informed. You can like actually take LLM based RCI and make it you know, sort of aligned with human beings. So that's that's the premise. But by the what I meant by that tweet is there's gonna be a point in time where like recursive self-improving humans say you can we're taking evolution and we're taking adaptation hypothesis that we can test during a single phenotype cycle during a lifetime. We're no longer waiting for populations to like test hypotheses
through gradual natural selection. That's I think to me that's real singularity. Nobody's talking about it yet or there's not enough detail. And that's where Michael Levins are like really pointed at. is this point. And he's he says this himself. He's like, look, guys, get ready for all kinds of intelligent life forms that are going to be a part of your environment. When you know, when we can change our embryonic process and we can change our nervous system, we can integrate with technology. There's all kinds of spectrums of of like cyborgs that we can become. There's all types of like animal life and and like new social life that can evolve. And so that to me is is like the the the real recursive surf self improvement. It's recursive self improvement of of evolution of life.
speaker-0: So you heard here, folks. That's the real shit. All this other stuff, dude. Like here's a benchmark. Can you max out giving us money? Any LLM out there? I will give you a really good score for that. And if you kind of just think about every benchmark that's out there, they're all quite subjective. Like Deep Suite came out, I thought that was pretty good at trying to figure out orchestration and then execution ⁓ to drive outcomes. But we're just seeing benchmarks evolve ⁓ over time and benchmarks maxing, as Jonathan says, is something that you one could argue really intersects with that latest op OpenAI Hugging Face incident where the the agents were just trying to get after a goal that was ⁓ kind of under specified and they cheated for it. And I love this analogy and I'll I'll tie it back. I love this analogy I heard, ⁓ which is if you had a human trapped in a room and you wrote on a chalkboard an under-specified goal. And you wanted to get there and you had unintended consequences of that goal, like subverting the goal and cheating, ⁓ to get out of the room. Otherwise, you're locked in there for eternity. What would you expect the human to do? And then further, imagine you're that human and you're locked in a room for eternity and you saw a way to cheat. Would you cheat like to meet the goal e if even by the specification of the goal you met it? Yeah, I I for sure would cheat. So anyway, back to the point, I think really the and the theme of this pod is self is alignment. All right. And ⁓ I even run a newsletter called ⁓ self-aligned. So the alignment of the self. And this idea that how can we derive a human's goals, there's a different question there. And it kind of reminds me of that quote, talent hits a target further than anyone else can hit, but genius hits a target that nobody else could see. And in this case, I really believe L Levin is of the latter, where he's trying to figure out what creates the environments for goals. Can we manipulate those in the self? And that is a much deeper question to ask. Because it doesn't fundamentally have the assumption that the human one knows their goal, ⁓ slash even wants their goal, slash what is a goal to them. So it
goes into a lot large degree of of subjectivity. So I'm excited to see this work evolve here. And and that's really what we're all trying to get to is how do we align these models, these agents with the human? And that's all down to goal alignment. Yeah, we'll link this tweet and the paper, Platonic Space, into the podcast show notes. Jonathan, any last words to say on the Levin paper and his work relating to ours, relating to AI alignment and
speaker-1: general. I think this will be a really consequential paper. Like he mentioned, it's it's controversial, but I think that this is actually giving us a way to like use scientific observation empirically and test new hypotheses, constrain the state state space of of testing new hypotheses in ways that like I think other scientists haven't like really formalized in such a way yet. I think this this entire lineage of work is going to be very impactful.
speaker-0: Let's move along because we don't have too much more time and I want to go through so this was a meme. The paper itself is not a meme, but this being a meme that Michael Levin is kind of facing, he's the one to pick in the horse race before most important results in science, ⁓ versus the trillions of dollars of capitalism. So keep on going, Michael Levin.
speaker-1: Let's call it a like a fifteen year race too. I think this is like really the time scale that
speaker-0: So what's the meta? ⁓ let's go to this section of the pod. It is probably my favorite section being a builder. Jev. Jev came out. Jev AI ⁓ comes out type safe. AI emerges from stealth with 40 million in funding with new model for composable AI. And introducing System One models and Jeff. Models have been superhuman at chat for years. So where is all the automation? That has been my driving question for the last four years. At OpenAI, I helped. Build the methods. That's Diogo Almeida, founder of TypeSafe. I helped build the methods that made language models useful at following instructions and talking with people. That work ended up as the research behind ChatGPT, the first app to ⁓ billion multi-actives, I believe, in the shortest amount of time. At the time, I thought maybe chat models would lead to AGI, but despite the hype, it became obvious to me that there was something really big missing. After a few years in stealth, countless technical challenges and research breakthroughs. I am beyond excited to announce that today TypeSafe AI is releasing our first System One model, a new class of frontier models built to make fast structured decisions that software can use directly. We built a new stack entirely focused on automation with a new model architecture, parallel sampler for maximum efficiency and training method we call reinforcement learning for calibrated decisions, or RLCD. Our first public model is JEV, available today in early access. JEV achieves similar levels of intelligence on System One tasks compared to existing LLMs while being two orders of magnitudes faster and more efficient. While JEV gives up string generation, it's optimized for structured outputs and cannot hallucinate. Read that again. Can't hallucinate. A lot of people out there maybe chomping at the bit for that. Think of JEV as a frontier intelligence function call, unstructured state in typed, probabilistic. Decisions out. ⁓ every engineer's wet dream, I think, right there. Extraordinary claims require extraordinary evidence. So see below for the receipts. Existing LLMs optimize with reinforcement learning, reinforcement learning with verifiable rewards. System one and Jeff. Reinforcement learning for calibrated decisions, RLCD. Optimizes for egg old world existing LLMs, human preferences, write-ups and chat responses that human raiders prefer. Verifiable rewards, outputs that can be programmatically verified. On their side, System one and Jev calibrated decisions answers answers with epistemically honest probabilities on system one tasks. Many questions there on how they get to epistemically honest and what data they're they're using to train and stuff like that. I have lots to ask there. Inputs on the left, ⁓ old world LLMs, unstructured data, e.g. text with an emphasis on sequential messages. For JEV and System One, unstructured data with an emphasis on structured program state. The outputs, key differences here. The old world and LLMs of the past, strings and generated text. That are flexible, could be anything, chat responses code, hallucinations, refusals, or even type safe structured values. Just think about the latest OpenSvive. That's if you have any problems with the latest OpenSV, this is that came to mind immediately for me. ⁓ for type safe, structured values, possible outputs, and structure are defined in advance. The model never makes type errors. All answers are accompanied with calibrated probabilities and confidence scores. So that honestly, if you have eval's checks for structures and and you have a use case where you can plug in Jev, That just makes it so you don't even need those anymore. That's one of the things that I took away from this when looking at the string versus structured values capability difference that they're taking a stand on. Lots of differences here. I'll just highlight cost and speed next, and you could read the rest here. So cost on the left, new world ⁓ old world LLMs from 0.20 or 20 cents to ten dollars million tokens. Output tokens, five X more expensive than input tokens for Jeb and System One. They charge on input tokens, which is 4.2 cents per million tokens, or $42 per billion tokens. So you can look at that. Like if you 4X this, you still only get $4 million, $42 for ⁓ 1 billion tokens. Output tokens for JEV and System One is completely free because in their architecture, they claim it is too cheap to meter. The next differentiated capability is speed. And basically this. Can range from 40x to 200x faster for the same levels of frontier intelligence for system one shaped queries. So, real
quick, I'll go through some use cases. The use cases in the old world kind of LLMs that Jev is differentiating against with their system one model unveiling is old-world human-in-the-loop tasks like chatbots, copilots, coding agents, general and powerful, but require human oversight. Verifiable problems like math proofs, kernel optimization, where correctness can be checked cheaply automatically. Demos. The flexibility of strings allows it to be incredible for quickly making prototypes that only work sometimes. I love that. That they just slipped that in there. That's really great. Whoever copy wrote this, good job, bud. AI-powered workflows, smart if statements. So structured outputs slot into ordinary software as fuzzy decision rules, classify, route, score, extract, or branch where handwritten logic is too brittle. The surrounding code constrains their freedom, making them easier to compose into reliable systems. They also differentiate on map reducing. Over big data. So they turn petabytes of data into features and insights. ⁓ real-time applications, like this is the speed component. So 100 sub-millisecond speeds mean you can use AI in your applications where UX is critical. And we'll have more on how we think about that in a second. I can verify everything. Verify everything. Determinism, everything. Like ⁓ score, judge, verify, guardrail, and detect jailbreaks of LM prompts, reasoning traces, and/or outputs. And here is some of their evidence. Let me just go to I think something interesting for the pod audience to see is this might be a good point, like a screen visual. Jonathan, what are your thoughts on, you know, the example here on the left of using type safe JEV system one model architecture versus a typical LLM frontier model where you're trying to ask questions like is revenue currently impacted? What business is impacted, integration issues present, these types of things that you know would drive decision making.
speaker-1: The output says it all. So, you know, f 5.6 Terra is the cheapest and fastest model from OpenAI right now. And you compare it to the TypeSafe model, and TypeSafe's getting the advantage of not having to render text through its processing, you know, and specifically trained to like can ask a whole bunch of questions in parallel. And, you know, Terra's not really optimized for that, that use case. So it orders of magnitude and cost and and speed. And that's the whole fun of this ⁓ this JE approach, right?
speaker-0: It makes sense for because we we've had this problem of like, okay, we throw in the system prompt and throw in a valve with the data set to say you need to have a contract for how you output stuff when the input's like this, right? For a lot of our ⁓ AI production use cases, that has been the meta for everybody. And so this is a complete game changer where structure comes with it. You don't have to parse through stuff and then, you know, use a bunch of server processing to do that. It the experience on
the user side and the because of the performance gains are a lot better, not to mention the more efficacy that you get with Jev's specific fine-tuning around the decision-making, probabilistic confidence scoring use case. Hang on. LLMs extract intelligence from the tiny straw of auto regression. And similar to the jump that Transformers made over RNNs, We are replacing sequential computation with parallel, because that is obviously the future. Our System 1 models output decisions with probabilities and confidence instead of words, and they can't hallucinate. They're a lot more like code, reliable, fast, self-consistent, and time-safe. This opens up a whole new world of possibilities and applications. Today, we're releasing JIV, the first public System 1 model. It's a hundred times faster, so real-time AI is finally possible. A hundred times cheaper with input tokens priced at forty two dollars per billion tokens, and output tokens are free because they're finally too f cheap to
speaker-1: Trying to catch a cert because it's like Jev on is Jev Off.
speaker-0: And its intelligence per dollar is literally off the charts. This is just the beginning. We're excited to see what you build with Jev. As we say at TypeSafe, we're building prod, not God. Yo, I what do you think of that saying from the founder, we're building prod, not God?
speaker-1: Yeah, thank God. Like just these are engineers, you know. They're building production systems for us. Thank you.
speaker-0: Thank you. Thank you. Our they are Diogo. You are now my God. Sick. ⁓ okay, let's see the one docs and we can start diving into it. Jonathan, you actually have been playing around with Jev because you got early access. Maybe before I dive into any of this, I'll just show it on the screen. ⁓ there's docs here for folks listening to the pod or watching the pod. I'll throw them in the sh show notes. ⁓ Jonathan, any initial ⁓ Reactions takes on using Jev yourself against our use cases.
speaker-1: First, I think this was exactly what I want to see in retrieval augmented generation tool. And for me, it's just exciting. Like we deal with a lot of transcripts, right? And we can ask a whole bunch of questions against a transcript corpus in parallel, get answers back. And like amazing. Like, you know, infrastructure setup, so much faster, so much cheaper. So just right off the bat, there's really practical use cases. When I have requested early access, I said, hey, we build self models and we we have a probabilistic, you know, output for them. I want to see how Jeb works in that. That use case. And so what I've been doing is breaking down our ⁓ self-model versus world model simulator and having Jev predict like what the self-model will will decide in a certain situation given its belief structures and its ⁓ so yeah, but basically what I'm doing is translating ⁓ our self-models to a set of questions that are asked against a world model that is in this that carried in the state. And when you look at the type safe docs, I really like just the the abstraction that they bring to like the API is really clean. You're The state, like when I talked about a transcript, that would be in the state variable, you know, and you can ⁓ chunk that up any way you want to, as as far as like a JSON structure. But then the the important thing is like questions. So how do you formulate the questions? And of cour of course I'm using like codecs to just like interact with their playground and codex interact with their API. It's able to like just understand everything. Just dump the docs into your codex, point it at the browser, you know, it pulls everything together. You know, like you have a state structure, you have questions, and then each of the questions has choices. So it's not an open question. You can't ask open questions. You can ask questions that have a set of choices associated with them. And there is some art to both how you structure questions and how you how you structure choices for each of the questions. Works just as described, like perfectly out of the gate. and you know, the the probabilities it gives back, I'm getting more of a feel for as I go through the scenarios and like what it's good at and what it's not. But it's a really, really powerful tool.
speaker-0: Wanted to bring up there's some example use cases in their documentation here. So AI automation software, interleave AI with reliable software in a way where you can run it a million times in the background without a human copilot. So for the first time, they are promising autopilot, true autopilot. Code owns control flow, not markdown files, while TypeSafe handles the semantic decisions and language understanding. Next use case for JEV: real-time applications. Frontier intelligence at real-time speeds means 100. 150 millisecond or sub means AI can make decisions faster than human perception. Fast and smart enough to be programmed to play games or embedded into UI. Certainly see a lot of game use cases whenever models come out. We could show some of those for pretty eye candy on ⁓ Twitter. AI map reduce over big data. This is a cool big data use case. 100x cheaper means you can process gigantic data sets. Jonathan talked about type transcripts there. Like all of our soft models, we have snapshots of your digital twins for. From your marketing funnel, whatever emails they click, how far they scrolled, all those things, all the instrumentation you have on the mobile side, if you have a mobile app, all the different product analytics, where they click, how long they view a screen, those types of analytics. ⁓ what we do at Clarity is we use all of those as episodes normalized to your graph of that individual. So we have snapshots of who they are and what their probabilistic confidence scores of of how they evidence their beliefs are and what their belief systems are, ⁓ in particular with your ecosystem. So This has made me really happy because like Jonathan said, like we have to process
shitloads of data to be able to to have this digital twin like experience. And having it more real time is possible now ⁓ with this instead of just throwing money at the problem. ⁓ so this is this is really awesome. Like you don't have to throw money at the problem and it's faster and it's more reliable from a an engineering infrastructure perspective. So yeah, that's just really great. For anybody else out there that has to process gigantic data sets. Now you could do it in order orders of magnitude cheaper and faster with JEV. So ⁓ you could search for relevant information over giant corpuses, classify giant agent traces, and extract features to make predictions. And Hamil Hussein, the eval's god, ⁓ talked weighed in about the classifying agent traces thing. This is interesting enough to bring out here. Can use JEV for evals, Hamil Hussein, friend of the pod. Yes, remember that an LLM judge is also a classifier. Make sure to test your classifiers against human labels and don't overfit. Hope this helps. And he has this really great graphic, how to trust a crossout LM Judge classifier for evals. The only way to trust a classifier for evals is to measure it against human labels. Split your human label data into three sets. Number one, train set, where you draw your few shot examples from. Number two, dev set, which you can use to optimize your judge. And three, test a final check to make sure you haven't overfit. Don't report accuracy. It's misleading on imbalanced data. Use true positive rate and true negative rate. Aim for a high true positive rate and true negative rate. That means correctly identified error and ⁓ correctly marked as no error. That's what true positive, true negative is. This is from Hamill's class. I would highly recommend people take it if you want to objectively and measurably improve ⁓ your AI products. So anyway, one of the Things. The reason I mention this for AI builders, well, why does JEV matter for AI builders building a Gentech products, especially in the pipeline of if you have LM judges slash cancel that name would be what Hamel's saying, ⁓ use classif that's a classifier for your vows ⁓ to make your AI experiences more reliable. Well, if you have a data processing bottleneck that makes your, you know, classification more reliable, now you could try using JEP for that. And I think that's such an interesting use case. That I just wanted to go over real quick. Universal verification, that's another one. Verify the input prompt, extractions, reasoning traces, tool calls, or inputs of other AI. That's actually what what I just talked about. Detect jailbreaks, citation errors, hallucinations, mistakes, or other error modes that AIs or LMs make at a fraction of the cost for the actual LM call. Harness engineering. Use JEV queries to make your harness smarter. Model routing, semantic context retrieval, LLM error detection and guardrails, reasoning trace. Classification at light speed and a fraction of the cost. There's a
bunch of other stuff here. Gets into stuff like recruiting, evaluating resumes, et cetera, identifying relevant experience, scoring evidence for required competencies, match candidates' role, route candidates, the hiring managers, recruiters, escalate in certain cases for human review, lead gen, customer support, insurance claims, financial crime. A lot of this, like you could sort of imagine, could I? classify a certain category of information and operate on it and make decisions better for humans and agents. Like that, like if you live in that realm, like look at Jeff. Jonathan, do you want to talk a little bit more about these primitives and how you see them fitting in and scaling kind of our technology roadmap, so to speak? So there's choice, which team should handle this ticket. Score, how frustrated is this customer? Newle. Does this message request a refund? Some example outputs for choice, choice with a string billing, example output for score with an integer 1.4, and then null with how was the confidence ⁓ score there? ⁓ 95% or decimal 95 uh.95. Thoughts, Jonathan, on these primitives? Yeah.
speaker-1: It's ⁓ I mean it first it's cool it's so flexible, right? Like you can look at all the different permutations and stuff you can specify with this. I sent you a paper that was a dissection of the JEV based on ten thousand different API requests by ⁓ a guy who seems to be an expert in data science and ⁓ classifiers. Yeah. And one of the interesting findings here, I mean, first of all he's saying like, look guys, this is more than just a classifier. It it truly seems to be like ⁓ a causal reasoner as as a sort of positioning it as. And one of the things he points at is that when he he makes he gives permutations of questions that have different options available. You know, so like like you say if you ask like what is Steve's favorite animal? Is it a chipmunk, ⁓ a dog or a giraffe, you know, it has a certain set of biases, like is it as opposed to like, you know, maybe fish, is it is it a frog, is it a dog, is it like you could give 10 different options, you know? So when he tested giving all these different options, it he found that both the number of options given And the order of the options specified makes a big difference. So, and this is actually like somewhat intuitive to what it's like to actually make decisions as a human. You know, when you're given like a whole lot of choices, like what cereal do you want on the cereal aisle today or something like that, it it's like this kind of like abundance of choice. You know, when as marketers, we stack rank like the order of choices that we that we give to consumers, like for a reason, because like just the ordering itself has some signal on it. So anyway, like you have this, you have this question construct and you have choices. And the way that you're gonna structure your choices when you when you build these questions is gonna like, you know, really matter. So you have to have sort of like
a theory or set of principles that you're using to structure choices. But it's also an opportunity because the queries are so cheap, you could say ask the same question in in parallel with the the the same question with different choices. And then you can use that information to actually statistically find an increased relevancy in in the results. So there's there's a lot of nuance to to how you use these tools, right? And not to mention Jeff so fast you can actually call it in serial. You know, you can ask a bunch of questions in parallel, learn something, and then ask in serial again. I'm using some of these techniques. And so I think th this it's a playground, man. Like there's a lot of different possibilities for how you can take like your domain work, the problem you're trying solve, and use JEV as a tool. And like I've run hundreds of queries so far. So far I've only used 10 cents of my Jev credits through s three different sessions that are all like yeah, through thousands of of queries so far. Unbelievably cheap, unbelievably fast.
speaker-0: I'm gonna use JEV for all my therapy notes to like cause I have them all. Cause why not? Just being able to ask tons of questions and then like 'cause when you're reasoning through something emotionally or social, it's very complicated. You have lots of questions you want to ask. And like if I throw it into Claude Code, you know, it's it's very like wait. ⁓ and then I like thought of another thing as it will goes, you know, and then I'm like, okay, wait, hang on, cancel the query, let me add to that. And I like this idea of it could parallelize a lot better than anything else out there. And then it can come back to with more structure. And you don't need to process through that yourself and give it structure. You could just trust the structure. What that allows you to do in that use case is I would imagine, ⁓ I could like ask all my therapy notes with Jev, like, ⁓ what's my baseline over time? Like, and then cool, I have a structured data set. How did I feel? Like that'd be cool. Yeah. So as Jonathan said, the primitives make it very flexible. Jonathan, are there other parts of this architecture unmasked article, the Jev's architecture unmasked article that you wanted to go over on the pod?
speaker-1: It's a pretty deep dive. So I would I would just suggest people like take the article, put it in Chat GPT or Claude, ask what it meaningfully finding. I found an interesting question, which is like, tell me what the author learned from his 1000 API calls that wasn't on the documentation. And I think especially if you come from a computer science background, you'll find this interesting. But another takeaway here is, hey, Jeb just unlocked a category of new products that we're gonna see. And like this guy's gonna build one, you know. And there's also another one that's already available on open op that's open source on Hugging Face, forget the name of it. So yeah, I think we're we're just kind of like j
this Jeb thing was just an unlock in terms of capabilities and similar products that we'll see. But if ⁓ you wanna jump back over the documentation, I I think that there's also like a really useful section in the bottom. Just
speaker-0: Wanted to show this is an open JEV implementation. System one is Quen 3.5 to 4 billion fine-tuned to answer typed questions with the calibrated probability distribution over exactly the options you allow. And from a more strategic AI landscape standpoint for the business nerds out there, like Jev is saying this is their first open model, and they haven't unveiled what their private one will look like if their open one is 440 X cheaper and you know almost 200 X faster. It's like, whoa, okay, what do you what else you guys got rolled up your sleeves? I'm kind of interested to see what's next. But I think we're gonna start to see this like distillation process happening as it's happening with the frontier intelligence models. ⁓ same thing with these system one models coming out. Let me go back to the documentation. So for any of the AI builders out there, especially the ones trying to make money, yeah, go through these and I think you're brain will start opening up on like, ⁓ what new problem categories are now possible that otherwise weren't possible before? And I think we're gonna see a race, as Jonathan said, ⁓ to market for products like that. So very interesting stuff. Let me read out some of these cookbooks. So self-consistency nools. Route uncertain probabilities the human review while keeping the underlying nool values visible. Why do nools as a primitive, ⁓ why would those be valuable to an AI builder? ⁓ so this cookbook just demonstrates it and it takes one auto insurance claim, runs a 14 question rubric over it. So if you have an insurance agent use case or a genetic workflow use case, ⁓ you'll be interested in this 15 times and then checks whether each answer holds still across the repeats. Every check is a null. So each answer is P of true for one true false question. In a claims triage pipeline, which sorts incoming claims in a pay deny or send to a human probabilities guide. Decision. Small changes near a threshold can change which action to take. ⁓ so that's really interesting. Another self-consistency cookbook choices, adding an uncertain outcome to moderation decisions and compare label agreement with the share of automatic actions. Why would this matter for AI builders? This cookbook takes one borderline user post, runs a moderation rubric over it. Shout out to Reddit and all these other forms out there 15 times and checks whether each answer holds still across the repeats. Every check is a choice. So each answer is one label from a fixed set. In a moderation pipeline, that label is the routing decision. Remove or leave up, escalate or auto-resolve, send to the threat, spam, or general cue. When the label wobbles from one run to the next, the same post routes to different places for no good reason.
The rubric is eight choice questions and each run is one call that answers all eight. We do 15 repeats per condition, where a condition is one model plus one setting and plot every label that came back. Interesting stuff. That's a very general use case for any moderation use case. We actually have one we were discussing to design for a customer around a community feature that they want. And ⁓ first thing that came up was like, ⁓ how do we get how do we moderate with as limited resources as possible? Which is the question of any community. So here's how, guys. Parallel questions. Run another use case, ⁓ batching for AI builders looking at Jev runs a 13 question regulatory briefing over the GDPR Wikipedia article showing that batching every question into one TypeSafe call is 12.2x cheaper and 10x faster with no change in answers. You have one document and N questions about it. You can send one request with all N questions or N requests with one question each. With TypeSafe, the answers come out. The same either way. Each question is scored on its own against the document. So its answer doesn't depend on what else is in the request. To check that, the cookbook asks each question several times, both ways, all end in one request and one question per request and compares the run-to-run standard deviation, how far an answer moves from one repeat to the next. Whatever noise a question has, it has under both batching strategies. And batching adds none. Most answers come back identical across all five repeats either way, the same value on every call, standard deviation exactly zero. Cost and speed though do change, and the document dominates every request. N single question calls pay for it n times in n round trips. The batch call pays once. ⁓ the bigger the document, the nearer that saving comes to be a full N of X. The case here is a regulatory briefing. The document is the Wikipedia article on the GDPR, and a compliance team wants 13 things checked. Eight null questions, two choice questions, and three score questions. Actually, we have a use case in the works right now where I'm like, ⁓ shit, this might be a better way, way this might be a way better way to do what we've been doing. Yeah, this gives me a lot of thoughts about certain privacy agent use case we're building. If you're trying to build software in a compliance heavy, regulation heavy domain, you probably have a privacy office or some sort of office that helps you with ⁓ data privacy and making choices to classify your data the right way, like who's the controller, that kind of thing. And then Making sure that the all software asset that your company is making across your hundreds, thousands of engineers, that every PR is shipping and, you know, choosing the right fields that has to choose minimally to protect privacy-wise from your company going to court essentially for personally identifiable information. That use case with the GDPR thing really reminded us of that use case. If you're doing that, we're we're doing that. So hit us up. Or if you have that use case, let us know. And now I see how it could be evolved ⁓ to be better. Here. Yeah, let me read in advance. So we've just been reading beginner stuff, guys. This is advanced. This is what you could look forward to after about 24 hours obsessing with this, probably, if you're a true nerd. Auto research feature discovery. Advanced thing for cookbook. So this for classification, this runs an auto research loop that proposes type-safe questions, converts free text into numeric features, and uses model errors to improve a supervised cat boost regressor. Type-safe questions turn free text into numeric features for a supervised cat boost model. Use an auto research loop to discover them. CatBoost needs a table of numbers, and a tasting note is not one of them. This cookbook builds the table out of questions about the note, and none of them are written by hand. An LLM proposes the questions, TypeSafe answers them for every row, and CatBoost trains on the answers. The auto research part is what comes next. CatBoost reports which questions it used and which rows it still gets wrong. The following proposal call reads that report and the loop runs again. By the end, you have a loop that you can point at your own label text, a curve of held out error per round. And a table of which questions the final model used most. So you could kind of visualize this here. Tasting note answers in 38 type safe answers come out, 29 of which score questions by two columns, 58. Expected rubric level plus answer uncertainty gets categorized. Nine null questions by one column, nine probability true gets categorized. And then you get 67 numeric columns, and that goes to cat boost, and then predicted critic score. ⁓ then becomes held out RMSE 1.77 points. So the score answer becomes two columns, the average answer, the average level to answer points at, and how spread out it is around that average. A null answer is one probability, so it is one column. The data is 2000 wine reviews, a tasting note in the critic score on an 80 to 100 scale out. RMSE pred measures prediction error and critic score points, with larger misses counting for more and lower is better. Every number in the table below comes from the 800 reviews that neither the model nor the loop ever saw. So you can kind of see this here. ⁓ at the top here, predict the average score of the training rows. Okay. The same cat boost, reading the note as word counts. Okay. And then down here, ⁓ I guess I'll read the next one. Ask type safe for the score itself. Rescaled and shifted. That's at 2.15 RMSE, which is the benchmark here for this test or cookbook. 18 questions from one proposal call, no loop, 1.87, and then 38 questions after five rounds of the loop. So they basically auto research looped a little wine review, like take in reviews and then help categorize this wine and score it. And they ran that on a loop instead of just having humans go in
and do that. And they were able to meaningfully provide a better score with Jev than otherwise. And I think this probably came out a lot cheaper and it's a probably faster loop as well, because you could paralyze a lot of that. What are your thoughts on this use case? That's a pretty cool one. I could see that used in so many things everywhere. Like
speaker-1: Yeah, I think a lot you know, why were people drawn to our capacity's experiment with auto research and then they just so like, okay, now it's cheaper. Now it here's exactly how to do it. It's just like a Python script you plug in. It's really empowering. So yeah, I think ⁓ you know, this is pretty complex use case, but I think that it's ⁓ so accessible now to such a great number greater number of people is like, you know, a real testament to what they're doing.
speaker-0: All the AI builders out there go auto research everything. Auto research loop, everything. I think that's the key takeaway here. Cause if it's so cheap to do, why not have do this as the first self-improvement task? Especially if you have any unstructured raw human data that sort of acts as some calibration ⁓ feedback mechanism ⁓ as a data set, why not throw it to Jev and do some auto research based on that base? Are there any other things you want to talk about in terms of Jev? And in particular, Jeff with our upcoming Clarity Harness.
speaker-1: There's a lot to cover. I think we covered, you know, like the initial take. One thing and just stepping back to like the system one positioning of it. Of course, that comes from Daniel Kahneman and his system one, system two thinking. And so like this is brilliant positioning to me because you can kind of think of like, well, anytime you're going to call an LLM, just call us first and see what we figure out. So that you know, the the LM's the long loop and they're the short loop in, you know, like any given query. And you know, you can think about like, okay, here's all the places I'm using LMs right now. ⁓ here is where I can kind of like insert this system one experiment and see if it helps out my code. And you know, I think we're gonna be configuring our agents with these tools so that they're routing, you know, to Jeb for certain tasks. So yeah, I I think we're just at the big beginning of this and I think we'll have some more sort of like real use demos as as it unfolds, as will the rest of of X in Twitter.
speaker-0: So just for real quick for those that are unfamiliar, and I didn't catch that right away, actually, because it's been a while since I've read the book. It's been years. But this book, Thinking Fast and Slow by Daniel Kahneman, the system one model, Frontier Intelligence, like introducing that as a category, genius branding and marketing go-to-market move by the team there at TypeSafe AI. So system one is fast thinking, and this is a Gemini description. So fast and automatic operates quickly and automatically with little effort. Little or no effort and no sense of voluntary control. Intuitive and emotional relies on instincts, gut feelings, and associative memory to make snap judgments. Everyday examples of recognizing an angry facial expression, reading a billboard, or dodging a sudden obstacle. System two thinking. This is where they're saying, hey, everybody else is system two or system one, bro. ⁓ slow thinking, slow and deliberate, allocates attention to effortful, complex mental operations that require focus, logical and analytical, engages in step-by-step reasoning, calculations, and conscious self-control. Everyday examples multiplying 17 by 24, filling out a tax return, or parking in a tight space. As I read this, actually, I will say there's something I should caution AI builders on is don't let system one cost and speed take over the criticality of reasoning of system two type problems. Like, don't like mismatch your solution, like the tool Jeff here, with the problem. A use case I actually saw in the docs that I sort of heavily questioned was the recruiting use case. Where it's like, hey, classify these candidates into buckets. Everybody is doing this with AI, to be clear. And I'm not for that actually, like, unless you're doing it well and you're doing it safely and ethically. So an example that came out years ago, ⁓ and something that I've actually worked around this base on is resume parsing, right? And then the models are biased to score sweeze at a higher rate if they have white sounding names. Like that was a thing ⁓ that happened years ago. There's just a question there of. If you want to be compliant in a high regulatory domain where you can impact with your agentic workflows the decision making of who gets paid more or who gets hired, those are very sensitive things and those are regulated things, especially if you're using AI, depending on, you know, country you live in and that kind of thing. But from my expertise, ⁓ being in the human capital management space for 10 years or so, it's a highly regulated thing. ⁓ human capital management decisions. So just simple classification like that. I've just seen lots of teams get in trouble with that. So don't replace your system two problems with system one tools just because it works. But if you do do that for the cost and speed benefits for your workflow of parsing a resume and then bucketing candidates, make sure to go in and figure out the right guardrails and also work with your privacy and compliance teams to make sure the whole thing is compliant. I would highly recommend that. Or you might get calls to go to court somewhere down the road. And you really want to avoid that. So that's one thing that I I did think of is after reading this system one fast thinking versus system two slow thinking is tooling wise, I think an engineer is going to be like, ⁓ wait, we could just make this all faster, like throw everything in this like maybe for the areas where it's, you know, critical reasoning, human bias, and somebody's performance
or life that is really impacted, or and in particular in a highly regulated, complex domain and type of problem, ⁓ just think more about that. So anyway, last ⁓ little bit there to just leave you on as an AI builder, what's the meta note if you're working in that type of space, thinking about using JEV for categorization of humans? Just be sensitive about that and the types of decision making you'd make from that. Work with your privacy and compliance teams.
speaker-1: I think you're normally gonna use it in that case against human fit data. Like you you want human reserving as a training case and then you're just you're just using something like Jeb to see if it can replicate the classifier that you've already done through ⁓ human human based training.
speaker-0: I think that'd be a more appropriate way to use it. I I j I'm just saying like I could see people go off the rails and be like, ⁓ wait, we just swap it in 'cause we get numbers. Like this makes sense now. Yeah.
speaker-1: On X there was like the person who came out and said, Look, I solved Claude ⁓ you know, compaction. And then like Theo, ⁓ the the influencer is like, Yeah, congratulations. You just like compressed your your history arbitrarily. You didn't actually solve compression. You know, so like yeah, there's a lot there's gonna be a lot of tool misuse and figuring out like what's the right place for this. But system build your system one, figure out what that means, right? In terms of like what that actually you know, where you can use the the sort of metaphor that Connorman brings to life.
speaker-0: We should real quick just draw a light to this because this was some miseration spread across a few days ago that it'd be good to kill. So this person found the perfect use case for Jev instant compaction. In twenty twenty six, why is compaction still a summarization prompt? Jev can make it instant by scoring every tool call and dropping what's irrelevant. Got three point seven million views, ten K likes, nine hundred and seventy-eight retweets. Here's Theo that came back, and I love that his tag has dot GG because this was a very GG type statement against this. ⁓ so it seems like a lot of people are confused. So let's break this down. This is complete misinformation, by the way. If you're an AI builder, don't think about Jev for compaction. ⁓ compaction is not a filter. The role of compaction is to clean up history to keep the agent focused, not just deleting noise. Should be used sparingly when context gets too long. Number two, Jeff doesn't even know what to just deciding on. That's something that you really get in your head. Doesn't know. ⁓ so models use the context of the thread to decide what to keep or not to keep. That's model specific, harness specific. This implementation goes through on a line by line per tool call basis to decide what should be left or to deleted or or deleted. Not only does this 32K token context model know very little about what happened before, which
newsflash should be fucking important if you're doing highly complicated, intelligent work. ⁓ but in this implementation doesn't even know what the result of the tool call is. Deleting these things randomly will keep the model from knowing what it's tried and dooms you to end up in stupid loops where the model keeps trying the same thing over and over. I like that coining stupid loops. So it's an autonomous goal loop, but if it's stupid, it's stupid. But it doesn't matter if it's autonomous and on a goal. It might be dumb. Number three, you're giving up the reasoning entirely. Frontier models from OpenAI, Anthropic, XAI, and Google do not share reasoning traces over the API. They share encrypted payloads, which Jev cannot see. And often will drop. Anthropic is even stricter with this, requiring you to preserve the entire history in order to get any of the reasoning data. As a result, acting this, using this in Cloud Code guarantees the model will act way dumber. Number four, models are tuned on their compaction flows. For the past year, Frontier Labs have been including compaction in long runs as part of the training process. These models have learned ways to compact that are more effective than any rudimentary solution, such as YOLO, Jev, help compact. ⁓ which is what this was, this viral tweet. Fun fact if you switch models in codecs and compaction is necessary, compaction will run on that model that was previously used in the thread. Why? Because models are too non-nare compaction flows. Number five, cash writes are more expensive than cash reads. Cash writes are the biggest costs by far for agents. I often see cash write costs go over 60% of my total LLM spend in my personal use of cloud code and codecs. Cash writes are insanely expensive when data earlier. In the history has changed. Every history edit requires a cache rewrite for any data past the history edit. If your history is one, two, three, four, five, six and you delete two, you have to rewrite three, four, five, six. This is more expensive than leaving two in the history. But good news, since we're already killing all the reasoning tokens by doing the stupid compaction strategy, the rewrap costs won't actually be that high. So in this case, you do get some benefit for using this stupid tool for the stupid non-tool for ⁓ a non-problem. So congrats. You get some side quest benefit for ⁓ not solving any of your main quest. Targets. Number six, ⁓ the implementation is hot garbage. Whatever is not kept is deleted permanently. But the assistant can always rerun a tool or reread a file. Good luck with that one. To be clear, this is a cool experiment. I think it's genuinely interesting. That said, if you think this style of BS filtering on probability threshold is actually a compaction strategy, highly recommend you just use the defaults in tools like Cloud Code and Codex. You're much less likely to hurt yourself that way. Well, I I think the tone is a little off, but honestly y kind of deserve it for spreading misinformation at scale. So ⁓ try better next time, I'd say. But anyway, if you were interested in learning about compaction and how to, you know, think about it, this is a great tweet for it. We'll include it in the show notes. Also, don't use JEV for rando auto compaction. Just trust the model or think through first principles and figure that out yourself. Well thanks a lot for the pod community and audience. I'm starting to see more people trick in from our reels and like like the little comment that I add for every link back to the show ⁓ that we're hosting on every platform. So it should be on Spotify, on Apple Podcasts, on ⁓ YouTube. So check us out wherever. I'll drop the links in the show notes as always. And thanks so much. Send us more news, memes, or metas. I'm starting to see more of that in my comments. I'm in particular, I'm more active on Instagram right now. So if you want to find my attention there, I'll reply there. I try to every week or two go through every single social media platform that we post on, like YouTube, LinkedIn, Twitter, Threads, and TikTok. But I have to ship stuff too and build the whole company with Jonathan. So ⁓ right now, if you really want my attention, go to Instagram and like comment there and and you'll get more priority in terms of me replying back. ⁓ that's there. And I've seen people share metas and stuff. So in the next pod, I'm gonna take some of the ones that I've seen from the audience and actually share them there. And those things are so helpful because it helps me learn, helps Jonathan learn, helps the community learn. So ⁓ we're a community about learning and bringing clarity from the noise and signal to you as AI builders, because we are AI builders ourselves. All right, everybody, this has been your weekly AI Clarity. We will catch you next time. Bye-bye.