Cortex Compute 004

Episode overview

Anthropic invited theologians, rabbis and ethicists to help shape how Claude behaves, and some left stunned after hearing worries that Claude itself might suffer, as Elizabeth Dias reported in The New York Times. Lennart and Vugar ask who gets to decide what good AI does, whose morality a model should learn when humans disagree, and why an AI company would raise machine consciousness now. They also test a manager’s request for a reassuring memo that hides job cuts.

Takeaways

* A rulebook does not raise a good person. Vugar argues people learn morality from parents, religion, friends and consequences, so a list of 500 rules can’t make a model good.

* There is no single human morality to copy. Vugar points out that religions, cultures and nations disagree, and that women in many societies couldn’t vote 100 years ago.

* Conflicting instructions force a choice. Lennart’s test case is a reassuring restructuring memo that omits planned job cuts, where the assistant must comply, object or ask for context.

* Someone already chose your AI’s values. Lennart notes that mainstream models follow Anthropic’s or OpenAI’s value systems, while local models like GLM run with fewer restrictions.

* A judgment layer trades usefulness for safety. Lennart argues an agnostic tool executes whatever it’s given, and moral judgment makes it safer and less effective.

* Read the outreach with the IPO in mind. Lennart calls it partly marketing, while still rating Opus 5.5 and Astra as big advances.

* Ask for visible choices. Lennart wants the people affected by an AI’s decisions to see and question how the company told it to behave.

Timestamps

* [00:00:00] Welcome and the New York Times story. Religious scholars meet Anthropic and hear concerns that Claude could suffer.

* [00:01:03] Who decides what good AI does. Vugar asks whether the scholars changed anything Anthropic had already chosen.

* [00:01:38] Teaching virtue to a model without faith. Alignment, autonomous deception, bioweapons and restraint.

* [00:04:05] Moral status and the Kool-Aid. A neural network compared to a plant, and the risk of anthropomorphising.

* [00:05:33] Judgment inside the machine. Emotion vectors and spotting bad intent behind a request.

* [00:06:33] Sentience and liability. What a trillion-dollar valuation means for responsibility.

* [00:07:20] What is good. Vugar on a child, 500 rules, and whose morality counts.

* [00:09:40] Two principles, one decision. The restructuring memo that hides job cuts.

* [00:11:37] Whose value system. Local models like GLM, privacy, and values set by Anthropic and OpenAI.

* [00:12:27] Who is liable. A company acts on Claude’s business advice, and someone pays.

* [00:13:05] Morals move. Vugar on voting rights 100 years ago and the Middle Ages.

* [00:14:54] Being human when intelligence is shared. Vugar asks what happens if AI forms its own morality.

* [00:15:36] Does AI feel anything. Lennart doubts it and weighs the article’s slavery question.

* [00:16:39] Workhorse or moral agent. Why a judgment layer makes AI safer and less effective.

* [00:19:52] Abusive chats and bad intentions. Power fantasies, and Vugar on Anthropic’s misuse report.

* [00:20:44] The Pope says no. The Vatican rejects machine consciousness while self-improving models keep advancing.

* [00:21:41] Why now. Vugar notes the irony of agnostic technology seeking religious advice.

* [00:23:10] IPO marketing. Lennart’s cynical read, plus praise for Opus 5.5 and Astra.

* [00:23:50] Visible choices. Who carries responsibility when AI decisions affect companies and nations.

* [00:25:34] Close. Subscribe to Cortex Compute on your podcast app.

00:00 Welcome and the New York Times story 01:03 Who decides what good AI does 01:38 Teaching virtue to a model without faith 04:05 Moral status and the Kool-Aid 05:33 Judgment inside the machine 06:33 Sentience and liability 07:20 What is good 09:40 Two principles, one decision 11:37 Whose value system 12:27 Who is liable 13:05 Morals move 14:54 Being human when intelligence is shared 15:36 Does AI feel anything 16:39 Workhorse or moral agent 19:52 Abusive chats and bad intentions 20:44 The Pope says no 21:41 Why now 23:10 IPO marketing 23:50 Visible choices 25:34 Close

Links and mentions

People

* Lennart Nacke (host)

* Vugar Ibrahimov (co-host)

* Elizabeth Dias, New York Times reporter

* Pope Leo XIV (Fast Company on the rift with Anthropic)

Books, essays and papers

* Elizabeth Dias, “Religious Scholars Met With Anthropic. What They Heard Stunned Them.” (The New York Times, September 2026)

* Anthropic, emotion vectors research on Claude (Decrypt coverage, April 2026)

* Anthropic’s report on Claude misuse cases

Tools and companies

* Anthropic, Claude, Claude Opus 5.5

* OpenAI, GPT-6 Astra

* Z.ai, GLM

Mentions

* Apply for Lennart’s one-on-one coaching to

* Join me at Insight Incubator.

* Silicon Vally motto: “Move fast and break things”

Over to you

Press play for our full conversation, then subscribe to Cortex Compute on your favourite podcast platform and send it to a colleague who lets Claude draft workplace announcements. For weekly notes on writing and AI, join the newsletter.