TUNGKOL SA EPISODE NA ITO
In this episode, James Bowler, Head of Research Partnerships at AE Studio, is joined by Erick Martinez, Alignment Researcher at AE Studio, to explore why pre-training is a largely neglected frontier in alignment research, and what it actually takes to do alignment work there.
Erick breaks down what pre-training is and why it matters: it accounts for roughly 99% of the compute that goes into a frontier model, producing the base LLM from which everything downstream (instruction tuning, RLHF, capability fine-tuning) inherits its character. The key implication for alignment is that the span of possible behaviors and personas a model can exhibit, including misaligned ones, is largely determined during pre-training. Post-training interventions can elicit these latent behaviors but cannot fully excise them. The emergent misalignment finding, where fine-tuning a model to write insecure code caused broadly adversarial behavior the model was never trained to produce, is a concrete example of a problem whose roots lie in the base model, not the fine-tuning step.
Erick draws on AE Studio's own gradient routing research, developed in collaboration with Anthropic and awarded a spotlight at ICML 2026, to illustrate just how technically demanding pre-training alignment work is. Gradient routing lets researchers control which model parameters get updated in response to which data labels, for instance isolating the parameters that encode dual-use biology knowledge so they can be localized or removed without degrading general capability. But making this work at scale requires operating well below the PyTorch abstraction layer: managing gradient graphs across distributed hardware, ensuring micro-batch accumulation stays mathematically equivalent to full-batch updates, and catching silent gradient leakage that can quietly corrupt an expensive training run before anyone notices.
The conversation closes on an under-appreciated asymmetry in alignment incentives: post-training papers are cheap and fast to produce because the base model is already baked, while pre-training experiments require significant compute, specialized infrastructure experience, and tolerance for expensive mistakes. That cost is also an opportunity. The field is sparse enough that well-resourced researchers willing to work at this layer can find genuinely novel ground.
In this episode:
*Why pre-training sets the "span of possible personas" a model can exhibit, including misaligned ones
*How the emergent misalignment result points to pre-training as the root cause of certain alignment failures
*What gradient routing is and how it lets researchers isolate dual-use knowledge during pre-training
*The Chinchilla scaling law and what a realistic pre-training data corpus looks like (50M to 5B parameter runs, up to ~200 GB of tokens)
*Why distributing training across GPUs creates subtle gradient accumulation bugs that can silently degrade alignment interventions
*Why alignment incentives currently push researchers toward post-training and what makes the pre-training layer a fertile, underexplored area
Learn more: https://www.ae.studio/alignment
AE Studio is hiring: https://www.ae.studio/join-us
Subscribe to our newsletter: https://aestudio.beehiiv.com/
James Bowler LinkedIn: https://www.linkedin.com/in/james-bowler-84b02a100/
Erick Martinez LinkedIn: https://www.linkedin.com/in/erick-f-martinez/
Contact us: alignment@ae.studio
