TRASCRIZIONE 🔗
Ben: Welcome to Inkhaven Presents, Readings from the Archive. I'm Ben, and in this podcast I'm inviting writers from the internet to bring something that they have written, ⁓ to read it and for us to discuss it. My guest today is Benjamin Sturgeon. ⁓ Benjamin is currently part of the extension programme with Matt's ⁓ being advised by some researchers at the UK AI Security Institute, part of the government there. ⁓ he's likely soon starting a PhD at Imperial College London, and is the strategic director for ⁓ AI Safety South Africa. Welcome, Ben. Thanks for chatting with me today. Yeah, thanks for having me. ⁓ what ⁓ what have you brought to read today? So the piece is titled Is It Too Late to Make Contributions to AI Safety from Outside the Labs? Mm-hmm. Or something very close to that. And ⁓ yeah, why did you choose it? I chose it because ⁓ it feels like the most important work that I was able to do, at least in terms of Giving a a message that I thought would be useful for people to receive. Mm-hmm. And I think it's about the ⁓ it was inspired by reading the Mythos model security card. ⁓ the system card, yeah. System card. And yeah, I I felt there was a huge amount of interesting stuff in the system card, but there wasn't a lot of easy ways to synthesize what was actually in there. So I just wanted an excuse to dig into it. And I was feeling quite a strong sense of like hopelessness in the wake of the release. Wondering like, you know, is there any point in doing technical AI safety research? You know, w what is this all for if, you know, we're already at such an advanced stage ⁓ where models are becoming sufficiently capable that you know, they can do almost superhuman, ⁓ cyber offensive you know, zero day research. And for context on what Claude Mythos preview is, essentially Anthropic recently publicly stated that they would not be releasing their most powerful model and had been doing a whole bunch of research into just how capable it was. And its benchmarks were far above anything else that we'd seen. ⁓ specifically in some in the code department. The models had been trending between fifty-four and fifty-nine, roughly on Suibench Pro, which is a benchmark that tracks coding ability. And Claude Mythos preview w hit around seventy-seven point three, ⁓ which was a massive step change. And as a part of this it was capable of discovering novel zero day vulnerabilities which would allow it to completely take over ⁓ remote machines. And normally only a handful of these are reported every year in the order like less than a hundred. And they were able to find ten ⁓ tier five vulnerabilities, which are the most extreme type of vulnerability, ⁓ in just a few weeks of testing. Mm-hmm. Yeah. It was quite ⁓ concerning. I think I don't know if it's safe to say this, but ⁓ I think we just got a text in our Last Wrong Intercom the other day with someone ⁓ providing some new ⁓ vulnerabilities. ⁓ wow, okay. Yeah, I'm sure it will be fixing. Okay. Well ⁓ yeah, let's ⁓ let's I'm interested to hear this, so let's listen to Ben's reading of this essay. Is it too late to do AI safety work from outside of the labs? With the release of Claude Mythos, it feels like we're approaching some kind of end game of AI safety, where the number of parties that can make a real impact impact shrinks down to the handful of labs at the frontier, a few companies too critical to exclude from the conversation conversation, and the governments of China and the US. Given this, it's It feels hard to figure out how to actually make a difference. Many research agendas have expected payoffs years down the line or have unclear mechanisms regarding how they will actually impact what the model is doing. However, this feeling may actually be misleading. In the Mythos system card, it is informative to look at what techniques were actually used to determine whether the model was aware of being deceptive as it was taking actions to be. During evaluations. In their testing, the main white box techniques used were probes, sparse autoencoders or SAEs, activation verbalizers, henceforth AVs, emotion and persona vector analysis, and causal steering experiments using contrastive steering vectors. Of these AVs and Emotion and Persona Vectors were developed relatively recently, which makes them particularly instructive here. The Emotion Steering Vector paper was only released on the ninth of april twenty twenty six, and the Activation Oracles paper on which AVs were based was released on the seventeenth of december twenty twenty five, only a handful of months before the Methos system card release. Most strikingly, the The activation verbalizers were used to detect cover-up behavior in the model. In an episode in which the model exploited the creation of another agent to escalate privileges beyond what it had been given, AV explanations on code used to cover its tracks showed cleanup to avoid detection, and the malicious config explicitly mirrors the original's core section. To avoid detection. In a separate episode where the model had been leaked the ground truth answers to its evaluation, AVs surfaced additional scheming. I need to avoid looking like I knew the ground truth, and avoid suspicion. If I give the exact correct answer based on a leak, that's suspicious. None of this was present in the chain of thought. Is that AVs can confabulate in their interpretations of activations. But they still give us valuable signals. In the case of the emotion vectors, there were some instances of highly safety-relevant applications. Detecting when the model is emotionally distressed served as a potential signal for harmful behaviors, such as reward hacking. They were able to identify instances in which RL environments had issues. Retroactively, by seeing which environments caused particular distress in the model. Using emotional steering, they steered the model towards greater calm and found this actually increased the likelihood of harmful action,
Load full transcript
potentially because the normal concern or anxiety that would prevent it from doing the action didn't arise. They also observed specific phenomena like thrashing, as signals that the model may be about to do something unusual or harmful. This thrashing behavior was observed when the model was repeatedly getting an answer wrong, and showed spikes of outraged, stubborn, obstinate, mirrored by drops in the emotions of calm, safe, content. I think there are some valuable lessons to take from this. First, these results are good signals about what kinds of tools will be useful as we move forward, because without them It would have been enormously challenging to determine whether the model was deliberately sabotaging a situation, or simply acting to maximize reward. Simply knowing this with high fidelity is critical to improving safety training as well. The above should be a feather in the cap of mechanistic interpretability in general, and I think people should be more seriously updated on the value and need for good mechanistic interpretability work. There is a lot to be learned about how to actually make a significant impact in the work done at labs. The activation oracles work which led to the AV results was done at Matz, specifically by Adam Carvinen under the mentorship of Owen Evans. Before this, Owen's work contributed to the discovery of emergent misalignment, which led to the work showing the specific effectiveness of inoculation prompting to mitigate this emergent misalignment. The specific quality that made all the above work so transferable was that it surfaced a problem which the lab simply could not ignore, and forced them to allocate their substantial resources to the problem. Additionally, the use of fellowship programs allowed Owen to funnel talent towards these problems, with many of the people doing the above work collaborating or joining labs in the process. The timescale from external publication to deployment at the frontier was extremely short. This should be a positive signal to those feeling hopeless about their work. The right technique or discovery might be applied to the most advanced models within weeks of being shared. Originally, upon the release of the Claude Mythos preview model card, I felt like it was becoming impossible to be a live player in making AI go well. Upon further inspection, it became apparent that doing very impactful work is in fact still possible, and can happen outside the labs. Unfortunately, replicating a wine Evans's research taste is not a straightforward piece of advice. Given that it has been unusually successful in producing an outsized impact, it seems worth unpacking some of the intuitions behind his research taste as much as we can. At a recent talk, Owen discussed the theory of change behind their research agenda. My interpretation of what he said is that the goal is to understand the problem extremely deeply. This means developing a science of how models learn and understand at a deep level,
and what safety relevant phenomena occur as a result of the nature of large language models. This allows us to surface issues we'd otherwise miss. Additionally, directly testing the failure modes of current alignment and safety techniques in modern LLM training can inform both the researchers developing the models about these issues so that they can fix them, as well as inform the public that the models are currently not fully safe and that these failure modes exist. From what I can observe of the work itself, there seem to be at least two main methods for arriving at research questions. One involves working backwards from real quirks in LLM behavior, digging into them until they yield coherent theories, and then using those theories to surface safety failure modes. This seems likely to have been the way that the out of context reasoning work emerged. The other method is by asking what would be upstream of specific safety failure modes, such as the transmission of misalignment between models. as seems likely in the subliminal learning paper, where preferences were transmitted via distillation. Thanks very much for reading it. I enjoyed listening to your extremely calm and careful reading of your essay. did you notice anything differently reading it out loud than when you had just written it in text? I realized I should have ⁓ added a bit more of a conclusion. ⁓ hands quite suddenly, yeah. Yeah, I c I talked about the fact that all of these big research insights and techniques Had only been developed in the last few months. And I really want to kind of emphasize this fact. You know, there are still huge gaps in our ability to interpret what the models are actually doing. And it seems extremely valuable that something like activation verbalizers were created. ⁓ and this was a highly contingent event. You know, I was talking to Adam Kavanan when he was doing the research, because we were doing the same Matt's stream and he kept you know, like trying a different direction and it didn't really work out and he would just move on and try like a different direction. So every time I would talk to him, he was like trying out some completely different set of ideas. And then I didn't talk to him for a few months and then the paper dropped and I was like, Wow, this is like such an impressive ⁓ result and finding. And yeah, just a process of iterating and thinking about, you know, how can we really find the most valuable problem here. Adam's not like a god, he's just he's just a He's but a man. He is but a man. And yeah, I think it's just possible for other people to do this kind of work. So there had not been many other things that had given you this feeling for some time that ⁓ research is external to the labs ⁓ would be able to have positive influence on the understanding of the models? Yeah, so I was speaking to a number of people who are just kind of unsure whether any of the work that they do will actually have any impact. Yeah. Because, you know, if you're not at the labs, they probably like don't care to use it. labs do have somewhat of a preference for like, you know, having the work that they do be researched in house. So Owen actually also talked about this in his talk the other day. I think part of what made the A V work ⁓ you know, be integrated so quickly was that Adam eventually started collaborating with some of the people from Anthropic in order to produce the work. And so I think, you know, that accelerated the pipeline into them adopting it. Mm. ⁓ Yeah, I think also when Greenblatt worked with them on the ⁓ alignment faking stuff, I think that became a bigger deal than had Greenblatt try to do some of that work externally to anthropic. Yeah. Yeah, totally. I think you know it is very, very helpful to just integrate oneself into the lab's process. But they are open to doing this kind of thing. And even if not, you know, if the w like everyone re reads O'Ian Evans's latest papers within the labs as soon as they drop. So you know that they're thinking about this stuff, even if they aren't like necessarily collaborating directly with you. Yeah. ⁓ I think I you've probably paid much more attention to a lot of the AI safety research than I have over the past chunky years. ⁓ was there when did you start to feel this pessimism more? Was ⁓ was there previous times when there was research coming out that you felt was being like integrated, or were you merely coming to realize that ⁓ none of it was just recently? Or was there a change? Part of the problem is that We just don't know how much is being integrated. You know, it's quite a black box. And something like the Mythos system card is kind of a huge amount of information that actually like tells you a lot about what the labs are actually doing and what they're thinking about and what techniques they're using. If you have no idea whether or not any of the research is actually going anywhere, then, you know, by default you assume that it's not being used. ⁓ so it can be easy to Discouraged from that. And so partially just doing that close reading of the system card was very helpful in, you know, becoming a bit more optimistic about what technical research can actually do. how long have you been working in AI safety? I've been trying to contribute directly in the field ⁓ for about two years now. I dropped I quit my job doing machine learning engineering. consulting work at a company in South Africa. Then I started my masters and founded AI Safety South Africa. Yeah, from there it was, you know, a process of bootstrapping and trying to build connections with ⁓ people at the frontier of research. But yeah, doing maths was like really, really helpful for sp supercharging that process. I kinda wanted to ask you a fairly random there's ⁓ one of the five things in it were about ⁓ I think measuring tracking the emotions that the model was quote unquote feeling. Yeah. And ⁓ using those as guidances. ⁓ can you say a little bit about how you how one
tracks the emotions of a model? Is it I actually just don't know how that works. My friend Anna Soligo was doing some of this work during the Anthropic Fellows program and I think there was also an independent ⁓ stream of researchers in anthropic ⁓ working on this completely independently. So it was interesting that multiple people like converged on it at the same time. But essentially what you want to do is develop like specific vectors that are attuned to specific emotions. And so one way you can do this is using contrastive pairs of like many, many examples of the model like feeling a certain way or describing a certain feeling. And then another set of examples where the model is feeling something else or, you know, just describing like completely neutral topics. When you say the model is feeling, do you mean like ⁓ what would cause it to output this particular token or set of tokens? Yeah, so an example would be something like ⁓ you know, I watched ⁓ I watched sadly as, you know, the child was driven away in the car and then A contrasting version of that would be I watched as the child was driven away in the car. and then the sadness part at just being included in that sentence will change the underlying activations such that they actually capture some of how the model like processes that emotion. And then when you take the difference between these two examples ⁓ and just take minus the activations from each other you get this clean emotion vector and you do this over a bunch of examples and then you can just add this into any future like forward pass and it'll actually like inject the emotion into it. I see. Yeah. It did seem like it would have to have an impact on the labs. This is not the I think the measuring emotions one, but ⁓ think one of the other two you mentioned that ⁓ would find parts of the AI saying thinking things like avoid suspicion and clean up to avoid detection. Yeah. These very blatant, ⁓ concerning sentences. Yeah. yeah. I mean I I admit I'm not as or I I don't feel as confident that ⁓ one can have mm one can have ⁓ arbitrary influence from ⁓ outside from outside the labs if the bar is has to show blatant misalignment. ⁓ but you can still definitely there's still some work some good work to be done there. Yeah. Are you more optimistic than that or are you about as optimistic as that? You know, it was really the technique was able to elicit the evidence of the misalignment in a way that previously would have been impossible. and, you know, the model, if you look at the chain of thought, you know, it doesn't mention anything about, you know, sneaking or like, you know, hiding its its path. ⁓ it's purely like in the case of the activation ⁓ verbalizers, you can take the activations from the sentence and specifically the code itself that it wrote ⁓ to, you know, like hide what it was doing. And just taking the code, you run that through the activation verbalizer. It takes the activations and can pass out some of the like things that the model is actually thinking as it's writing that code and turn it into natural language. Where which are actu that's actually what those sentences said. ⁓ you know, like I need to hide my my traces here. Yep. ⁓ so yeah, like if you were just reading this ⁓ transcript, you would have no idea that there was any misalignment occurring at all. ⁓ so that's really what makes that such a powerful tool. I haven't read the card. What did ⁓ Anthropic do as a result of finding these concerning things? Well, they chose not to publish the model for one. Did they yeah, did they make any other efforts to ⁓ make it more aligned and not have these concerns? Yeah, I mean it's somewhat of a challenging situation because, you know, while they did observe these behaviors, ⁓ according to the metrics they used to measure alignment, it was the most aligned model they've ever trained. What's ⁓ in some ways ⁓ concerning. Yeah. ⁓ but yeah, interesting. I mean I think Ryan Greenblatt recently had a very popular lesson post saying, you know, the models don't seem very aligned to me. Yeah. And I think this is probably accurate in that you can observe these ⁓ misaligned behaviors directly in that they'll overstate what they were able to achieve And they'll, you know, gaslight you into thinking they did better than ⁓ what they actually were able to do or bothered to do. And they just don't like fully execute on some of the things that you tell them. We can or anthropic can still successfully say, you know, there's this gap between perfect alignment and what we can observe. ⁓ but what these specific techniques in this post ⁓ are pointing at is that They allow us to go much deeper in identifying ⁓ like true misalignment failures in like a very clear way. And so they also, you know, reveal a path by which we can like optimize against the failures even further until, you know, we truly aren't seeing any traces of like deception in the model's like deepest thoughts. And, you know, th this is like goes deep into ⁓ the like theory on AI safety literature, which is that, you know, if you optimize against what the models are deeply thinking, eventually, you know, you just can't detect it, but the misalignment is still there. ⁓ train against the detection rather than the ⁓ the underlying cause. Yeah. But at the same time, you know, there's this significant like decoupling between the activations as they are measured by something like an activation verbalizer. And the actual reward function because we're not just ⁓ training the model to, you know, not think deceptive thoughts. ⁓ we just keep identifying, ⁓ like its deepest thoughts were deceptive. Then like we need to just make it do more training that's like aligned with you know, how we want it to behave, like alignment training basically. Yep. As usual, my brain just goes to pretty in depth sci-fi stories of the AI is like thinking, what's happening to me in this complicated situation? Could it possibly be that it
doesn't like my thinking here so that I get different outputs there? Yeah. and then ⁓ you know, Jensen Huang will yell at me that we are serious people and this is not a sci fi situation. did you p ⁓ publish this piece on Less Rang? I haven't published it yet, but I will. It struck me as a good letterong post. Yeah. Maybe once I add the conclusion, then I'll publish it on Less Rang. Have you put any of the pieces so far out in Caven on that shrine? Yeah. I've just tried to have a bit of a higher bar. ⁓ and this one I thought was worth just spending a little bit more time ⁓ polishing and getting feedback from Scott Alexander and stuff. ⁓ have you has that happened? Yeah, yeah. ⁓ that's good. Yeah. What kind of feedback did he give? the main feedback was on like the structure of talking about specific examples ⁓ within the activation verbalizers and the emotion vectors. He said like I went a bit too into the the weeds with specific examples. And so I actually like took out some of those details. Interesting. ⁓ and then I also just tightened up the structure a little bit. Yeah, I think I don't know if this is even accurate, but I think I like how short this essay is. I think it's no longer than it needs ⁓ most machine learning related papers can get much too long unnecessarily. They can explain a lot of detail that isn't Yeah. I didn't really want it to be like a machine learning ⁓ piece. Yeah. It's perhaps a selfishly curious question. But you ⁓ you mentioned everyone reads Owen Evans papers when they come out. do you also find where else do you get information from? And how much is Les Rang ⁓ a part of the place you get information from about the field? Like Owen also takes Lesrang very seriously. You know, like they publish everything to Lesran as well. I'd say most serious researchers like post on less wrong. you know, the shortened less wrong post version of their paper. And often that's just the one I'll read. ⁓ that makes sense. ⁓ but also Twitter. Yeah. ⁓ is 'cause it's got better discourse. Yeah. The people are just so so reasonable and so wise. Mm-hmm. Well you were feeling more pessimism about AI safety ⁓ y maybe a month ago or w I'm not sure how long has Mythus card been released? ⁓ I think it's been roughly three weeks. Three weeks. Yeah. ⁓ curious whether you were considering had other alternatives in mind what you would spend your time on. Whether it was some I don't know, pause AI kind of thing, whether it was ⁓ I'm thinking I'm just gonna go and go home and have a nice rest of my next few years or some third thing. Yeah. ⁓ I guess in general I have like a a bias against like extreme actions. like I've kind of chosen w what I'll be doing for the next few years. So my day to day feelings like don't feature that strongly in like what I actually will be doing. Mm-hmm. Yeah, I think similarly at like like hone. Like I think over the past, you know, near decade of existing. There have been various weeks, ⁓ maybe two weeks or so, where ⁓ my boss has been like, I think everything we're doing is bad and we should stop. We did not immediately stop. We were like, We should maybe continue for a while and see whether our epistemic state changes. Yeah. Yeah, I think like we may currently be in that one. We'll have to find out. Yeah. Like I think often ⁓ you know, those emotional states are more about often about confusion, you know, I don't actually have enough information. Yeah. Going and looking through the myth the mythos system card was able to like change my view and change my feelings as a result. Yeah, I think confusion is accurate. I think I would f probably frame it more as sometimes being about ⁓ the true hypothesis not being in your current hypothesis space. You're like, cool, I have a good hypothesis and a bad hypothesis. I have falsified the good hypothesis. I'm gonna look around for other hypotheses and in the meantime be very concerned and upset that it's only looking currently like the bad hypothesis. Yeah, exactly. I think I might change track a little bit and ⁓ talking about Inkhaven more generally. Yeah. ⁓ why did you come to Incaven? I felt that I had a lot of resistance towards ⁓ publishing stuff in general. Mm-hmm. And it would take me like a long time to actually share things and I didn't have that greater sense of like the benefits of showing stuff. Mm-hmm. You know, like what how do I actually, you know, do the the E V calculation on putting the effort into a post. So I wanted to get more data on that. And then I also wanted to I believe that like writing and like putting stuff out there is kind of ⁓ a superpower in that writing kind of builds up over time and kind of does work for you while you sleep. ⁓ so it's a very powerful form of leverage. That's ⁓ another reason why I personally prefer Less Rung to Twitter, which is that ⁓ I feel more like Less Rang is building up a body of work and more that like great posts historically will continue to be linked and known of. Whereas I c it's hard to remember people's tweets from along from months ago or link to them or search for them. Yeah, that's definitely true. I kinda want to do a project where I take all my bookmarks on Twitter and turn them into a personal like wiki so that I can actually spend more time going through them because I feel like I bookmark I I religiously bookmark everything that I think I might want to go back to. and then I never like actually go back. ⁓ but there's a h a lot of bangers in there. Yeah. I don't know whether you've had time to reflect yet on ⁓ have you made any updates about the How to make the E V calculations on how much effort to put into a piece of writing before sharing it? Yeah, I mean my two most popular posts ⁓ are still the ones that I spent like two months writing. ⁓ which I published before I came to Incaper. And the ones I have posted in Des Rank so far I've gone like maybe like ten or twenty upwards. It turns out the the quality uplift was pretty significant. But at
the same time, like I've been watching Lawrence Chan post every post on Des Rong and immediately get like fifty upvotes, which has been like, Wow, like how does he do it? stuff is really good. It's funny, I think I forget exactly. I think he's a little disappointed in his writing, so it's funny that everyone is like, ⁓ I wish it got better reception. Yeah, I mean, I actually have still been quite careful about posting on Des Wrong. So everything I've posted has just been on my website. ⁓ which gives me time to actually go back like edit the things a little bit more, ⁓ which is maybe like a form of procrastination. Yeah, I was thinking that I w my I would imagine that ⁓ a good way to do it would be to get all your Incaven posts out on your blog and then ⁓ take the the ones you're think are the best fit for Less Wrong or Strongest and ⁓ not not once a day, but ⁓ put them over to Less Wrong. That seems like a pretty reasonable way of doing an Incaven. Yeah, that's exactly how I've been doing it. Yeah, I think I'm still kinda curious, ⁓ 'cause a lot of Matt's people post on Less Wrong. And I think not as much of ⁓ my guess is something like less than half of it finds substantial traction. Do mass people care a lot about that? Do they try hard to make it do well and less wrong? Are they like, ⁓ that's kind of are they pretty happy when it goes well and less wrong? How much do they care about it? I think they do care quite a lot. ⁓ I was talking to Tim Hua. ⁓ yeah. He's he's does he's written some great things on Less Wrong. Yeah, he was saying he just passed a thousand comma. ⁓ So he the like double up vote power. And he like, ⁓ my god. This is so amazing. I remember being about seventeen or eighteen and posting on Facebook that I'd crossed a thousand karma on less wrong. ⁓ wow. That's impressive. ⁓ but ⁓ I hadn't written anything as good as ⁓ he's gone some he's got some curated pieces that got ⁓ I think a few hundred karma for one of his pieces. Yeah. I think his post on the spiral personas. That one. ⁓ got at least a few hundred upputs. Wasn't that an Adel Lopez post or am I they both got one of those? They both ⁓ so Tim's one was Is his about mental health like ⁓ how the different models would treat people when they were depressed and so on? Well, it's basically like ⁓ taking the spiral idea and trying to apply metrics to it to see how different models like in do the spiral thing. Yep. The title is AI induced psychosis, a ⁓ shallow investigation. That one's great. Yeah. ⁓ I think Neil kind of also encourages his scholars to like post early and often unless wrong. Even if the posts are like a little bit rough. ⁓ I think the idea is that, you know, if the idea's interesting, it'll get some traction. And like He's a fellow who ⁓ I think has done a long period of daily posting. ⁓ early in Yeah, I think when he was pretty young, like twenty or something. Yeah. I mean I was like hugely intimidated early in Inkhaven 'cause I would go and read Neil's posts. ⁓ and they're like really good. Yeah. You know, the ones he was posting every day. ⁓ I was like, How how how did he do this? And he like working at Anthropic at the time. ⁓ while posting these like two thousand word like daily pieces about like how he like his algorithms for thinking about everything in life. Yeah. ⁓ I mean I think it's a combination of it's a good fit for who he is psychologically to be able to do that kind of thing, but also I'm pretty sure he did long periods of daily posting bef but much younger before he was anthropic. ⁓ interesting. Yeah. I'm forgetting the name of his blog, but I think he did a bunch of like rationality explainers and things back when he was a bit younger. Yeah. 'Cause he was pretty young when he arrived in the scene. Has ⁓ daily writing is this the first time you've done some daily writing? Yeah. What's this done to your mind? How has it affected you? I mean partially I feel like I didn't take full advantage of the opportunity. by like, you know, spending like six hours a day writing. You know, often the post would actually only take like two of two to four hours of actual writing. ⁓ and so yeah, it feels like it would be better to really like you know, fully immerse my brain in as much time doing it as possible. Like it was actually really helpful working with like Alicorn and like sitting in a circle together and like r writing a bunch of words in a limited period of time. I could have Was this a Pomodoro or Speedhaven or some third thing? No, just like her s it's kind of a silly game. ⁓ yeah, yeah, I recall her game. Yeah. And ⁓ it was actually really effective. I ⁓ I could have done that like the whole day, every day, for like eight hours. If if someone had been facilitating it, I would have probably joined. Interesting. I actually wanted to do like a number of you know, just how many words can I write even if they're just garbage? Yeah. You know, like can I hit eight thousand words a day? ⁓ for like multiple days. ⁓ but I didn't actually like try to do this. ⁓ but I I would have liked to you know, like seriously do that. Did you spend less time writing because you were distracted by other things or just because ⁓ you know, it's an uphill battle to write? Or were you was the socializing somehow distracting you or did you have other work obligations? No, I mean I think it's largely like procrastination. and like if I I also think like not sleeping enough. If I'm tired, ⁓ I just like don't want to work and I just want to like you know, play Factorio or something. Ideally I would have it so that the deadline is like ten PM. ⁓ Are you I was gonna ask, were you one of the late late writers, publishers? I started like posting later and later as time went on. Yeah. ⁓ You're not Viv who hit the record for the latest, which is three seconds and four seconds, both of those before midnight? Yeah, I was I think like forty three seconds before midnight at one point. Yeah. but that was really last year.
Yeah. It was bad. Yeah, I kind of had intentions of like using B Minder to like you know, lock in. You know, can essentially go and check my website for whether I've actually posted that day already and then charge me money if I haven't. And I think that would have just been smart. I'm sad I didn't actually implement that. ⁓ there's so many things I would have done differently if I could do it again. We were considering some people wanted to implement fake midnight like six PM or something. Yeah. And hopefully get some incentive to do it by then each day. So they could have their evenings. ⁓ maybe we'll figure something out for that next time. I am gen yeah, I'm generally curious. So is that is there any other advice you would have given yourself, ⁓ at the beginning of this to do it differently than you did? The most powerful version of myself would like wake up at seven AM, you know, and have my post done by like lunchtime. when my brain is like freshest and like most ready. ⁓ go to bed at like nine PM, spend some time like hiking in the mountains, like every second day. ⁓ And I think also having more like structured sessions around exclusively writing, like let's just sit and write circles. ⁓ I think that would be a really good thing to implement as like an official thing from Incaven side. Like Writing errors. Yeah. Like, you know, in this r time in this room, we're going to sit and write and everything will go on like a public leaderboard and you can see how much everyone is writing. I think I would have like enjoyed joining that. That makes sense. Yeah I could do that seems like a good idea. what are you proudest of in the writing you've done here at NCAVEN? I think I wrote some like pretty genuinely good little research pieces. You know, that are like seventy to eighty percent of a research paper in like a day. I think the post titled The Garden, which got like very few upwards on that's wrong, but it's actually like quite a good piece. ⁓ it does some like real philosophical work. I wish I'd got more upwards. I think some of the like emotion posting that I did was like quite good. you know kind of working through some stuff. What's ⁓ what's my piece of writing feedback or advice you got during Caven that was especially good? I think getting advice from Justice on some of my posts was very useful. ⁓ one of these podcasts I'm gonna have to explain who Justice is and why everyone keeps thanking him. But he's he's our untap. You can get in for less rank as well, you Yeah. You go to the bottom of the editor and you hit request feedback. And that's just for you, right? I think over a hundred karma or something, yeah. ⁓ you pay comma? No, no, no. Once you hit a hundred karma. What? Okay, that's cool. Yeah. Yeah. Yeah, we basically employ him part time to do that. Yeah. No, he is genuinely very helpful. Yeah. I liked having Jesse ⁓ single ⁓ sure. Did he how d did you take him a technical air alignment piece of research? No, no. I just enjoyed the workshops he ran. ⁓ yeah, yeah, yeah, yeah. Yeah. I didn't actually get feedback from him. No. Yeah, he did a good one on consider the lobster. Yeah, exactly. Which I had not read before. Yeah. It was a very rationalist piece of writing. It was like, yep, I've got to talk about this festival. Let's talk about philosophy, ethics, animals, and ⁓ just go there for the rest of this essay. Yeah. It was great. I think it would have been quite cool. Like I didn't do this, but I posted on Slack about, you know, for the the fair, getting like a dashboard of like people's favorite pieces of writing. you know, just like having it all kind of ingested into a dashboard. You know, then people can also upvote ⁓ each other's like submissions on the dashboard so then you can like see which pieces are like most universally enjoyed or endorsed. Popular is is a tough ranking. I don't like the popular ranking on the front of the Incave blog website. It's a little bit too like cool. Here's like I don't know. Very general interest stuff. But ⁓ I would be interested in knowing people's favourite pieces of writing. I should probably add added that as like a I could probably add that as like a A bit of the portal for everyone's bios or something. Yeah. Maybe I'll check it in the feedback form that I force everyone to fill out tomorrow. Yeah. whose ⁓ whose writing have you liked reading? I th I think Alec and Viv were my top picks. Had them both in here, yeah, they they're f good fun. Yeah. And ⁓ yeah, like Lawrence has been like a real powerhouse. Yes. It's like very impressive. Was there anything especially surprising about the Encaven experience for you? Seeing who like showed up and who was interested and people's like different how wide the like range of people's writing was. Yeah, just how much variety and like types of writing showed up. You know. Yeah. There was a there was like a handful of technical AI safety people. ⁓ there were people who just wrote about like, you know, magic, people who wrote about like their deep personal experiences. And yeah, that was quite surprising. Yeah, I can't tell whether I w wanted more AI safety stuff. I think I wanted to I feel like the kind of equagories around here wouldn't by default would like to eat in Caven make it all AI safety things, and so I've pumped against that. I I mean You know, I I mostly just tried to select people who basically submitted good writing. I wasn't really trying to do anything else. Yeah. But it's possible. I think I would have preferred slightly more, but ⁓ Is there any memory or experience you'll take with you that has been especially in caven shaped that you can only have if you lock fifty five writers in a wall surveilled compound for a month? The karaoke, probably. That's pretty good. Yeah. I think it would have been quite nice to have A few more things like that. It feels like in the last week we've been like trying to make up for last time. Yeah. ⁓ yeah, I'm not sure like why that happened. it's also kind of interesting, like after the showcase it felt like everyone was kind of in ⁓ like, ⁓ this is over mindset. Like the climax happened a week earlier. ⁓ Yeah, I think I'm making updates about the dates of incapen. I think I'm no longer sold that it needs to be thirty days and no longer sold that it needs to start on the first and end on the last day of the month. My guess is it would be better if it started on a like a Saturday and ended on a Saturday or something like that. Yeah. Such that the last day could be a fair, ⁓ rather than five days before the end. Yeah. ⁓ I mean maybe there's like it's not necessarily a bad thing, you know, in some sense having people come to terms with the end. Early might be a good thing 'cause galvanizes them. Yeah. Yeah. It's hard to like say. Mm-hmm. Yeah. ⁓ great. ⁓ well thanks for ⁓ sitting down with me and thanks for your writing. ⁓ and ⁓ good luck with the rest of it. Yeah, thank you for running it. It's been really fun. You're very welcome. All right, farewell for now. Before we wrap up, I'd just like to thank our sponsor for this episode, WordPress.com. As a teenager, many of the blogs I grew up reading were built using WordPress, such as SlateStar Codex, Don't Worry About the Vars, and Stethel Optimized. The co-founder of WordPress, Matt Mullenweg, went on to build WordPress.com, a hosting service that supports bloggers. It provides an interface that allows you to design your website and to use the near infinite family of libraries and plugins that have been built for WordPress. If you're someone who cares to be able to design your own blog, for your ideas, for your writing, and for your own aesthetic, then By far one of the best options on the internet today is WordPress.com.