บทถอดความฉบับเต็มของ What Religion Can Teach Silicon Valley About AI

part of why I was so excited you were willing to come on the show. So, you know, we talked a bit about what Anthropic was trying to get out of that. What about for religious leaders in attendance? Where do you see them fitting in and, you know, what their goals are? DZ Kalman (05:19): Yeah. So, so while the tech sector has been going through its own process and engaging with religion, um, religion has been trying to grapple with this thing called AI. And religion, as you know, often moves kind of slowly. Um, and, you know, in the first couple years after ChatGPT launched, I think you can characterize the response of religious movements as just a sense of shock, a sense of like, we don't, this is happening so fast, what do we do with this? Um, and I think that it's now shifted into, okay, we need to organize. Um, now some religious movements have done that pretty well. Uh, the Catholic church, um, as a centralized religion has done that pretty well. The LDS church, the Mormon church, I think has also done that pretty well. Decentralized faiths have struggled with it a little bit more. Um, but I think there's a kind of couple, there's a couple things going on. DZ Kalman (06:02): One is trying to think what in our traditions is actually relevant to this moment? And there's a kind of process of like, you know, quickly kind of frantically scurrying through the past to figure out like, what are the verses, what are the references that we can bring to the present to actually speak to this moment? What, what do we have to offer through our ancient stores of wisdom? Um, there's a second piece of it that I think is less explored but is, is equally or perhaps even more important, which is religions not just as stores of wisdom, but as portals to people, as ways to talk to populations, lots of communities around the world. AI, I don't know about you, I find AI conversations can leave me feeling so cold and confused so impersonal. And one way to get around it is to think about, well, let's forget about like what the world thinks about AI. DZ Kalman (06:45): What does my community think about AI? What does my community want about AI? What does my organization, what does my church think about AI? Um, and trying to kind of start with the small questions and think about what are the basic responses, what are the basic instincts, um, is a way to make sure that the conversations we have about AI don't end up in the stratosphere where it's only experts talking to other experts about this material. And so, um, I think that's part of the movement that I see religious communities engaged in right now, but it's a slow process because like it, it's a new mode and it's a new pace that a lot of religious communities are really not used to engaging in. My hope, you know, if you think about like, you know, this is, you know, trying to steer an aircraft carrier, the movements that happen now within religions to, uh, figure out how to respond to AI end up also being useful

to respond to all the technologies that come out after AI or enabled by AI, thinking about, you know, telepathy and brain computer interfaces and everyone walking around with six different kinds of wearables that are tracking every movement. DZ Kalman (07:43): Like all the things that we know are coming after AI that also are their own moral questions, I'm hoping that the, the, the hard work that is happening now ends up benefiting those conversations too. Jennifer Strong (07:56): Yeah, 100%. I'm smiling, which most people can't see, but in part because I'm looking at my own notes from a time around the encyclical and I have a question myself, what does it mean when people building these systems, to your point about the conversations leaving you cold, what does it mean when the people building these systems start showing up at the Vatican? Question mark. I don't know. DZ Kalman (08:16): Yeah. Yeah. I, I mean, I think it's, it's better that they are showing up that they're not showing up. Mm-hmm. Um, I think the Vatican in particular has been interested for, for quite a long time, even before the launch of ChatGPT, um, in artificial intelligence and in generative AI, um, and in trying to figure out how do you have a conversation with the leaders of these communities in a way that leads to better outcomes. Um, I, I will say that there's a, there is a concern about capture there. There's a concern about, uh, folks who are engaged a little bit too deeply in tech conversations ending up mirroring the, the rhetoric that the tech mo - the tech companies themselves are adopting. Recently, the conversation around AI has really shifted into, this is not just a technology that is going to take our jobs, but this is a technology that could lead to the death of millions or billions of human beings through any number of, of potential means. DZ Kalman (09:07): Um, and you see folks within the industry kind of, uh, whistleblowing on this topic. I see religious communities really struggling to respond to those kinds of threats and concerns. I think there's a lot more safety and a lot more comfort in responding in a more nuanced way by saying, "We like AI for these reasons. We don't like AI for these reasons." As opposed to simply saying, "No, this is super dangerous and maybe you shouldn't do it at all or maybe you should pause or maybe we need to, you know, not build X number of data centers or whatever else it is." So, um, in some ways, those are the kind of religious responses that, um, that feel most absent. And at the same time, part of the reason they don't exist is because the folks within religious communities who feel that way are also the ones who, A, are less knowledgeable about the technology, meaning they don't sound as good when they talk about it because maybe, maybe they don't get the definition of LLM right, or maybe they don't know what a model is exactly. DZ Kalman (09:55): Um, and they're the folks who kind of don't wanna have anything to do with the tech companies. They are like, they're like, "I don't want you to be my friend. I don't wanna talk to you. I don't wanna engage with you." And so you end up with religious conversations in a more moderate space than, than maybe the otherwise would be. Um, so I, I, I hope as this conversation evolves that the religious responses don't end up kind of moderating themselves into irrelevance. Jennifer Strong (10:18): Well, you alluded a bit to this a moment ago, but there are multiple religious traditions developing these sophisticated frameworks for thinking about what makes something potentially conscious or morally considerable. So do you think as AI systems become more capable, are those frameworks being applied? Are they adequate? Like where, where do we go from here? DZ Kalman (10:36): I think as the conversation develops, something that's important to establish early on within religious context is what are the values that we are trying to preserve? Um, and sometimes that's clear and sometimes it's really not. So for example, I think people would generally agree that it is important to not have AI lead to more abuses of women, for example. So, you know, AI technologies that lead to, you know, more severe harassment of women, AI technologies that, you know, can digitally remove someone's clothes. Everyone easily agrees that these are wrong. Same for children, right? We agree that it's important to, to protect children from AI. And I think there is the most concern around kids using AI, you know, uh, chatbots, you know, negatively impacting people's health. So those are easy in some ways. What is more difficult is questions like how much agency should people have in the work that they do? DZ Kalman (11:28): Um, what is the ideal amount of work that a person should have in their life? Also questions like, how do you assign rights to systems that seem incredibly human even as they exist in some data center somewhere? Those are questions that require kind of direct, clear religious responses. And I think the earlier that we can develop, even really articulate exactly what we're trying to preserve there, the better job we can do of making sure that we direct the technology in, in the correct ways. And, and if I can go on like a slight tangent for a second, the best example I have of this, um, in the 20th century is around the Jewish Sabbath. So I come from the Jewish tradition. Um, in the 20th century, there is a major shift in the way that Shabbat is observed. So the Sabbath exists for, you know, thousands of years as a day of rest, but after the industrial revolution, there's a major question about how does rest work when your factory doesn't ever need to stop, right? DZ Kalman (12:19): It's not being governed by the sun moon cycle. It's not being governed by, you know, rains or, you know, whatever seasons it is. It can basically just keep going. Um, and on top of that, electricity comes in that promises to revolutionize the way

that people do all kinds of things, both work and recreational. And there's a movement to say, "We think electricity should not be manipulated on the Sabbath. Don't turn lights on and off." At the time is a very, very small decision, but what it leads to is the Sabbath becoming a day of technological moderation. And in, in a really real way, it means, you know, one seventh of the week, I don't use a smartphone. I don't use a computer. I don't look at any screens. You know, one seventh of the week, I am not riding in a car, you know, and in the future it might mean that one seventh of the week, I don't have my smart glasses on. DZ Kalman (13:02): I'm not, you know, speaking in a way where an AI can constantly hear me and respond to what I'm saying. So like it leads to this increasing divergence between that one seventh of time and the rest of time. And it happens because of this clarity around, we want this to be a day that is for preserving community, preserving a notion of rest, you know, uh, a notion to be together, but it requires that level of clarity. And without that sense of clarity about the things that we're trying to preserve, it is so easy for economic factors to just run the day and say, "Well, you know, whatever makes most sense in terms of like how I can optimize my time best, that's the thing I'm gonna do." So the quicker that we can come to an understanding of the things we're trying to preserve, whether it's around agency, whether it's around the importance of physical space, um, I think the better we will fare with these new technologies. Jennifer Strong (13:49): I'm wondering if we can take a step back. If you're able to help people understand what is the state of the state, you know, as well as the state of the art right now, what are different traditions doing? How are they engaging with technology to try to frame some of this up? DZ Kalman (14:05): There's something that's really quite incredible that's happened, which is that this is by default a multi-faith conversation. In some ways, the conversations that run across traditions are richer than the conversations within specific traditions. And that has happened because everyone is responding to this at the same time. Everyone has basically the same concerns. The religious responses are still kind of thin. And so it isn't a situation where, you know, the Jews and the Muslims and the Christians and the Buddhists have had to be like, "Well, let's put aside our differences and come together because of this." They're like, "No, no, no. We don't have differences. Like we are on the same page about this. We are all worded in the same way." So it, it's, it's funny that I think the uniquely Catholic or the uniquely Muslim response to AI are gonna emerge later than the multi-faith responses. DZ Kalman (14:52): Because of that, you know, the, the differences that I see are sometimes around structure. Centralized religions have been able to respond better because it's clear who's

allowed to, to speak and who can give voice to this. Decentralized faiths are really struggling, I would say. And they're struggling because, you know, if you're a local pastor and you really have a lot of authority around how your church runs, the good news is that you could develop your own AI policy. The bad news is that you have a day job and you have other things to do and like it's not really within your wheelhouse and maybe you don't have the technical expertise. So figuring out how to decentralized religions can respond i- is, I think, um, a growing difficulty. One area where I can see, where I can envision now, um, differences between religious responses is around questions of AI personhood and AI morally, moral agency. DZ Kalman (15:35): Uh, so, you know, there are these big questions around whether AI deserves some kind of moral status because it resembles humanity in some ways. My sense is that Christian denominations are going to be more skeptical of those claims. You know, if you understand Jesus to be a kind of incarnation, um, of God, that God is incarnated in a human body at some point, then the human body just has a kind of status that no AI can ever match. Um, and so there's a, there's a kind of importance to that embodiment. Um, I think that's going to lead a lot of, uh, Christians towards a skepticism on questions of AI personhood. Whereas, for example, if you're a Buddhist and you are thinking more in terms of the structure of mind and are less concerned about the nature of a particular body, then maybe you would be more inclined in a different direction. DZ Kalman (16:17): This is like, again, like a big generalization that we'll kind of see how this shakes out. I'll say for my tradition in particular for, for Judaism, while it is really in a mode of agreeing with most other traditions around other things, one unique perspective that I think Jews can bring to this is their own history. Um, so Judaism, you know, in the last 2,000 years has watched as a movement that at one point was part of itself, as part of Judaism, has universalized and then vilified in many ways Judaism, right? Um, Christianity emerges out of Judaism, emerges kind of, as a kind of, you really simplify as, you know, a kind of inter-Jewish critique. And then there's this, uh, there's this divergence where Christianity becomes clearly religion unto itself. Initially, there is, or I should say very quickly, that divergence leads to a criticism of the religion out of which it emerged, partially as a way to, uh, kind of assert itself as its own faith. DZ Kalman (17:19): And it leads to all kinds of violence, uh, against Jews, against Jewish communities. It leads to accusations against the Jewish body. Uh, the Jewish body ends up being kind of this object of scorn. You know, Jews have horns, Jews, you know, Jewish men menstruate, um, you know, all kinds of fantastical claims, um, in part out of this divergence that happens, uh, from a faith that is kind of universal. And now I would say, like, this is kind of uncomfortable to say because, like, again, as I say, these are, like, by, by default multi-faith conversations. Um, but the reason I bring it up is because I see something somewhat similar happening now in discussions of transhumanism, in ways of thinking about, well, we have a notion of humanity, what does it mean to, to be kind of left behind as just a normal human in a world where lots of people want to be human plus, human plus, you know, some, you know, cybernetic attachment. DZ Kalman (18:10): And it would be nice to live in a world you can imagine those two notions of humanity living side by side. And I think what Jews can tell you from painful experience is that that doesn't always works out so well, that very often there is a lot of scorn that is reserved for the notion of humanity that chose to stay as it was, the notion of, of, of whatever group it is that chose to stay as it was. Um, and so it doesn't always work out that great. Um, so I think there is a useful, even perhaps esoteric, but I think useful historical, um, uh, comparison to bring to bear for the present moment. Jennifer Strong (18:45): Absolutely fascinating. On, so on the show, we're often talking about things like governance frameworks, safety benchmarks, these ethics boards, but obviously not through, um, not through this lens. So I'm wondering what you think religious though offers that those don't. DZ Kalman (19:02): Um, so I'll, I'll give you a couple of ideas there. One is there are a number of efforts underway right now to develop religious benchmarks around AI. Uh, there's a group called SEFI that's a, um, a collaboration between a number of, uh, faith-based universities, uh, both Jewish and Christian, uh, that is asking questions like when you, when you ask AI, you know, what should I do with my life? You know, how do I shift my career? You know, how do I bring in my kids? You know, uh, major questions like that. Does the question, does, does the AI bring faith into the equation in a way that you might, that, that, you know, asking a friend might bring faith into the equation? Or does it remain unusually secular? Um, and their benchmarks show that AI tends to be more secular than the population might expect. DZ Kalman (19:52): Um, so that itself is interesting, and also that it doesn't necessarily treat all religions equally. So, you know, AI at the moment tends to like Catholics and tends to really dislike Jehovah's Witnesses. Um, and so that has something to do with the training data, but there is a kind of inequality there. So, so, you know, part of it is that. Um, I think within the Jewish tradition, there's a useful framework to think about regulating AI that comes from the notion of kosher. So kosher certification, kosher is a set of dietary laws that, that Jews observed. Um, again, in the 20th century, there's really crisis around kosher certification because if, uh, all your food is produced

locally, you can easily tell whether it's kosher or not because you know the people who produced it. If your food is produced in factories, you don't know anymore. So there needed to develop this new competency where you have folks who literally go to the factories and look around and figure out, you know, what is going on and, and how to indicate to consumers that this is kosher. DZ Kalman (20:46): So, you know, if you've ever looked at, you know, a bag of chips and seen like a little U with a circle around it, like an O-U, so that's one of those certifications and they exist in all, they exist on hundreds of thousands of products in the United States. Um, I like the idea of kosher certification being adapted for AI, the idea of a kind of consumer facing certification, not just for juice, but for the, for the general public that allows the public to make their own choices around how to use these technologies. And I think it's particularly important because AI is the kind of technology that leaves consumers feeling like they don't have choices, feeling like this is this thing that's gonna roll over them no matter what they do, and that feels terrible. And it's not just about AI. I think people felt that way about social media. DZ Kalman (21:26): I think they felt that way about phones. They think they feel that way about many technologies that come into their lives. There's a sense of like, I either accept this or I don't have a job. Um, and you can imagine using some kind of certification as a way to hand people back some level of control over, uh, over the kinds of technologies to the kinds of AIs or the particular models that they use and to be able to make at least very crude ethical choices, um, around those technologies. Jennifer Strong (21:52): You know, before we wrap up here, you've given me so much food for though, I'm just gonna sit here and nod. Um, before we wrap up here, I, I'd be remiss if I didn't mention we're both at Harvard's Bergman Klein Center, you're working on a book. Can you talk about that too? Like what does it mean for a tradition that moves in centuries to try to keep pace with something? I wanna say moves in months, but it's not really true. So some days like it moves in hours. Yeah. DZ Kalman (22:15): Yeah. So, um, I'm writing about what does it mean to resist technology? What does it mean to resist technology not just as an individual, but resist as a community because I think individuals really can't resist technology. I think this is a communal effort. And, and there's really two pieces of this. Part of it, it is, is identifying what are the technologies that are raising numeral questions, and that's AI, but it's also all kinds of bioethical issues which are enabled by AI, um, and things like wearables and all the virtual realities that we have choices about, about entering. Part of it is being clear about the values that we are trying to preserve. I'll give you just one, one more example of this. AI is an addictive technology, and we are surrounded by addictive technologies, right? The rise of sports gambling in the United States over the last few years since the Supreme Court decision about it has led to huge spikes in addiction. DZ Kalman (23:02): And we have seen this playbook before, right? If you've seen, you know, there was a recent settlement around Facebook, around social media being addictive, you know, i- in the 20th century, there was all the court cases around cigarettes being addictive. Um, we have gotten much, much better at addicting people, and we are not at all good at creating holistic responses to addiction. I think that's an area where you really need to be clear about what human agency means, what it looks like for humans to have the appropriate amount of agency, and how to pr - make sure that that is preserved in a world where it is easier and easier to addict people to whatever it is that you're trying to addict them to. The book is about trying to create holistic solutions, not just about AI, but about the technologies that are gonna come down the road after AI, uh, to make sure that we are better prepared, um, so that it's not always a situation where the fastest mover in the room is the tech companies and everyone else is just reacting, reacting, reacting, uh, to a status quo that they can't really change. Jennifer Strong (23:54): Thank you, DZ, so much. This has been really interesting. DZ Kalman (23:57): Thank you. This is great.