Christoph: Hello and welcome to episode four of the Define and Conquer podcast. I'm Christoph Bamberg from the University of Salzburg.
Olga: Olga Perski from Stockholm University.
Christoph: And this is a podcast, which is part of the special interest group, conceptualization, formalization and measurement by the European health psychology society. We are recording this podcast on the 27th of March, 2026. And our guest today is Dr. Paulina Schenk. senior research fellow at the university college, London. And hi Paulina. I would like you to introduce yourself briefly now.
Paulina: Hi, thanks so much for having me and I'm really excited to be here. It's such a great podcast. So really looking forward to chatting with you guys. But yeah, my name is Paudina Schenck and as you said, I'm based at University College London. yeah, I mainly work on ontologies that are used for behavioral sciences.
Christoph: Right. I think then we can already ask the first obvious question. What ontologies?
Paulina: ontologies ⁓ basically ⁓ of classification systems that include basically categories of things, which we often refer to ⁓ classes ⁓ relationships these classes. just to give you maybe an example, you can have really classes about anything that you're interested in in terms of scope. And then you would have a label ⁓ definition and a unique computer readable ID in the ontologies we talk about at least. So the ones that are more related to computer science. There's of course also ontologies sort of within philosophy, which is something slightly different, ⁓ which can talk about in a bit as well. But sorry to get back to the example. ⁓ you ⁓ an example we often use is just a pizza example where, for instance, you could have a class of different types of pizzas and then you would have relationships between them where you could have, for instance, have subtypes of pizzas such as a mushroom pizza or a margarita pizza. I guess a nice thing with anontologies is that you don't necessarily just have these simple hierarchical relationships of a and a subclass, which would be like a pizza and a mushroom pizza. But you can also have more complex relationships so that you, for instance, say a pizza has topping mushrooms. And sort of that helps then form define what that type of pizza is. Now within sciences or health psychology this is obviously a bit more complex but can for interventions that's a good example where you can specify that you have interventions that have content and types of delivery for instance.
Olga: Great. So, I mean, just thinking about sort of the problem that ontologies are trying to address. So kind of why do we need them in the first place?
Paulina: So I think it depends on which ontology you're talking about, because there are different types of ontologies. think if people are kind of interested in the European Health Psychology Society, they'll often be familiar with the behavior change intervention ontology, because we present about that quite a lot. But there are different types of ontologies. I think that's quite important to appreciate as well. So a huge advantage of them is that they have this computer readable attributes. engineers are often really interested in them as well to sometimes integrate data, but sometimes also just use logic rules. So kind of in complex machine learning context that is quite interesting as well. But if we go back to kind of behavioral sciences or health psychology, so one of the key issues, I guess, that behavior change intervention ontology is trying to solve is sort of import also the jingle jangle fallacy that we have and that we're producing a lot of evidence that we can't actually integrate because we use language very differently. But also another aspect is really this data integration aspect of we're producing evidence at a rate that is much higher than humans can kind of keep up with. So there's that aspect as well of can we use ontologies to integrate our data better and then produce usable insights.
Christoph: Right. one of our previous episodes, I think it was episode one and two actually, ⁓ already briefly talked about the Jingle-Jangle Fallacy. Could you just briefly recap it and how ontologies ⁓ solve it?
Paulina: Sure, so the Jingle-Jangle fell asleep basically. You guys covered that quite extensively already, but refers to that we'll use labels for basically concepts with different definitions or vice versa, that ⁓ you'll have concepts with the same label that have different definitions. And the same issue also exists for measurement. So for ontologies, I think the clear solution in the first instance is more on the language aspect. So kind of moving towards having a more shared language and making the differences between concepts more formal so that if you, for instance, have a different definition for something that becomes a different class and that you don't use the same label twice if you can avoid it, at least within the same ontology, so that you capture differences much more precisely. So even sometimes having different classes for belief about capability and self-efficacy if the definition is different, so if they're capturing different things. but then you don't use self-efficacy twice as a label.
Olga: So something you already referred to, so you were saying that, well, we talk about ontologies, ⁓ there ⁓ ontologies the philosophical domain, but ⁓ you're to it here is a little bit different. why do you call it an ontology and ⁓ sort of how, in any way, does it relate to the philosophical term?
Paulina: some ways it does relate. ⁓ I guess the term ontology and philosophy refers to the study of what is and the we're talking about are tools or representational systems to capture or represent ⁓ what ⁓ basically what kind of things we assume exist and how they relate to one another. So that's the parallel. That being said, I do think it's sometimes quite ironic we use a term that causes that much confusion because I feel like when people first hear about ontologies, they always assume that it's about the philosophy rather than actually about the tools. So I do think that's a fun one. And I do wish they were called something different, to be quite honest.
Olga: So it sounds like it's almost so within ontology as kind of a knowledge representation tool, ⁓ but ⁓ it still kind of connected to what you think exists in that domain?
Christoph: because it's...
Paulina: It is connected. It's a way of formalizing what exists. But I think with ontology you often refer to more ontological standpoint and so forth. So I think that's what causes the confusion maybe where people don't think about it as much as a practical tool, which is often what we developed. But yeah, they were absolutely parallels. I completely understand how people have come to name it that, but at the same time, sometimes I think it just creates a bit of confusion when people first learn about them.
Christoph: to add another term to it, taxonomy. So when you describe ontologies, just ⁓ isn't it just a taxonomy? Could you ⁓ it in more detail? ⁓ taxonomy, right, is like, are animals, there are mammals, there are cats and dogs, right? So kind of just a hierarchical relation. So how does an ontology go beyond that?
Paulina: Yeah. Mmm. Of course. Yeah. Yeah. So, yeah, ontologies, basically, refer to them, I guess, as dictionaries or taxonomies on steroids. in taxonomies, you would basically mostly just have hierarchical relationships. So you would say, for instance, yes, ⁓ a cat is an animal. a dog is an animal and we'll have taxonomies as a backbone of our ontology. So of the relationships within the BCIO ⁓ simple hierarchical relationships such as a cat is an animal. But you have these more complex relationships. So if we go back to the pizza example, the mushroom is a topping of pizza is not something you would capture in a taxonomy. So you have these more complex relationships ⁓ allow you to relate things to one another in a more complex way. And then you can make logical reasonings about those. So again, we go back to interventions, kind of being able to say that content connects to intervention delivery in this way so that it has an intervention delivery or something like that.
Christoph: And with cats and dogs, so in a taxonomy, I can only say they are both mammals, but with an ontology, I could say dogs hunt cats or something like this. So I can specify the relation between them. Taxonomy on steroids. Okay.
Paulina: ⁓ yes. ⁓ a bit more or I mean that would be a ⁓ I would so with that relationships I'm being probably quite picky but because it's not necessarily established that all cats sorry all dogs will hunt cats I don't know if I'd specify that but you could specify cats I don't know have whiskers if that actually holds for all cats for instance so that you help kind of I guess you could in some ways relate to dogs and cats, but I can't think of anything off the top of my head. Yeah, that works. There you go. So you have commonalities based like that. So they have part leg or forelegs. And then, yeah.
Olga: both have four legs.
Christoph: But I think there's an important point in there that ⁓ ontologies, do they claim universality so that the relations always hold?
Paulina: Hmm. I think that's a bit of a difficult one where not always, but you would assume that they hold most of the time. are still ⁓ cat if you lose one of your legs or if you're born without one. So, but you could specify that cats generally have legs. So, but I completely appreciate your point of it's not 100 % clear sometimes whether it is universal or not. But I think there are, you are allowed to have exceptions, generally you would expect that to hold, the relationship to hold if that makes sense. it depends on what ontology we're talking about again, because some ontologies are just use case driven and there it will be different. And there you can start talking maybe about more causal relationships. But if you're talking about more general ontologies that are meant to be used more widely, such as the BCIO, so the Behaviour Change Intervention Ontology, then you would want to have relationships that hold most of the time at least.
Christoph: Okay, maybe before we go into the details about ontologies, maybe we could delineate it from another term, namely dictionaries, which you already mentioned as well. So because you said, yeah, a cat has four legs and so on, which is kind of a description, you would also find in the dictionary, but then an ontology goes beyond this description by also relating the dictionary entries, so to say, to other entries. Is that it? Okay.
Olga: So.
Paulina: Mm-hmm. Yes. Or I guess, again, unfortunately, it depends on your ontology. But on a, especially with them, I'm going to draw mostly on the BCIO because that's the one I'm most familiar with. But I'll try to kind of have a bit of a nuanced approach when I talk about ontologies and give examples of others where I can. But within the BCIO, for instance, you have on the higher level classes, so the broadest classes, you have clear relationships. But the more detailed classes, we actually have less semantic relationships between them because it becomes so difficult to assert relationships. So just to give you an example, we don't have as many relationships specified between the different mechanisms of actions or potential mechanisms of actions, which are basically processes through which interventions ⁓ work change behavior, because they can function very differently across different contexts and so forth and for different behaviors, of course.
Olga: So you already mentioned some of the benefits of using ontologies is in synthesis or with research growing ⁓ at a pace much faster than probably we can work through systematic reviews on different So do you want to us a little bit more about that and maybe give some examples of how ontologies are used in evidence synthesis.
Paulina: Yep, that would be great. Sounds good. So I guess that's kind of... I think an application that's quite interesting because we do evidence synthesis and a lot of different teams work on developing these annotation schemas or coding schemas that end up being very different. So it's actually very difficult to integrate data. So we've been working also with teams that work with large data sets or with ⁓ ongoing systematic reviews and then end up kind of with data that is completely incompatible so that they can actually compare evidence. So the idea is that if you use really any framework, but also just an ontology, if it's well developed, to develop those annotation schemas or again coding schemas, then you'll have a more systematic way in which you're integrating your data. So I think that's really an interesting one where there is a lot of potential. Then of course, if you have a data set that is ⁓ readable, the aggregation also becomes easier if you want to, for instance, look at metadata from several systematic reviews or something like that. So I think that's really example where the potential is great, but we do still have to do a lot of work on it. I think for some annotations, it's maybe slightly simpler. So for instance, if you're looking at behavior change techniques and the question of presence and absence. But even there you need detailed coding guidance and people can interpret the categories differently. So it should be approached with caution. And I think we should be really transparent about how we're coding things. ideally you use, for instance, the ontology, but then you provide feedback as well on classes to say this one doesn't work or I had to make these assumptions. And then we make that explicit in the framework. but for other things like outcomes, so behavioral outcomes. I think that's still something we need to work on much more to see how the different categories can actually help maybe standardize how we capture knowledge from different papers about behaviors, for instance.
Olga: So just a question on that. because kind of at face value, it would seem to me that the sort of coding instructions and sort of annotations, that that would also relate to the definitions of the different labels. So if the definitions are quite long and specific, shouldn't that then help with the coding instructions or kind of how do they relate?
Paulina: What's? Mm. That's a really excellent question. Thank you. So we have definitions that are actually relatively brief. So we develop definitions based on specific principles, which say that they should ideally be one sentence and so forth, so that they capture kind of the essential attributes, but they're also concise and understandable. And that, I think, can be criticized because concepts are very complex. So we do have additional entries that can be added to an ontology where you can have sort of comments on different classes so that people can elaborate on what the definition means in more detail, how it can be measured and so forth. But we haven't been able to develop that that much yet because we're developed, we're working with a lot of different classes. So I think also maybe going back to the previous episodes where you talked more about the importance of conceptualization, I think that's definitely an element where we need to work on it more. So if people send us back their coding guidance or tell us where the definitions are or are missing things, we update the ontology. That's maybe something I should have said earlier, but I think really one of the best things about ontologies is that they need to be maintained and they should be updated both based on evidence and feedback. I think that's kind of where the potential lies. But ⁓ yes, so Again, thank you for the question.
Christoph: Maybe it would be nice to actually talk about how you come up with an ontology, but ⁓ before that, you mentioned two terms that I think are probably quite specific, which is annotation and coding. They probably have a specific meaning with ontologies, right? Maybe you could explain that.
Paulina: Okay. Yeah, it's actually quite similar to how you'd approach it in sort of qualitative coding, I guess, if you're using some kind of framework. So it's basically using a class from the ontology ⁓ that to a text ⁓ a paper, for instance, or it can also be within a data set where you say, OK, this text is this. class. So that's how you basically annotate. it's linked, the text basically becomes linked to the class, if that makes any sense. Of course. So an example would be if I'm annotating using the Behaviour Change Techniques ontology, which is about sort of the content of
Christoph: Could you give an example?
Paulina: of an intervention, so basically what we do there. So let's say I find a text that where practitioners were talking with patients and they asked them to set goals for their physical activity. So that text I would think is a goal setting ⁓ change technique. So I would use my class to basically annotate that text. So that means that the behavior change technique was used within that intervention. And then of course you could have additional annotations, but that really depends on annotation schema, I guess. For instance, are you recording how often that behavior change technique was delivered and so forth? Does that kind of make sense?
Christoph: Yeah. Yeah. Thank you. And coding was the other term.
Paulina: Coding used interchangeably. I use coding to serve as a simpler word instead of annotation, but maybe not.
Christoph: ⁓ okay. So yeah, maybe we can talk about how you come up with an ontology because ⁓ almost sounds like you're creating a new ⁓ with new words and new grammatical structures and everything. So it seems quite a big endeavor. So maybe you could ⁓ tell us in principle you come up with this and also specifically for the behavior change ontology, right?
Paulina: Sounds good. ⁓ So there isn't hard or fast rules on how to develop an ontology. Generally what you would be doing is first specifying your scope. So what you want your ontology to cover because these are systems for a specific domain. So you have to specify what that domain is. Then will tend to look at what already exists in literature. So what's out there. And this can kind of looking at existing classification systems, existing terms that are used out there, for instance, by looking at papers. Then ⁓ ideally you will some kind of stakeholder consultation ⁓ or ⁓ you'll of people whether what you've produced makes any sense. And ⁓ you will kind of, yeah, what you do next? ⁓ Then have some kind of use test. So depending on what you want your ontology to do, you'll apply the classes ⁓ and ⁓ if, well, working at all. And that can basically mean, do my definitions make sense ⁓ when I try apply them in practice? Do the relationships hold and so forth. and then you will tend to have an evaluation of your ontology. again, applying those either to use case or having sort of competency questions where you say, is my ontology kind of coherent? Are the relationships logical and so forth? So ⁓ those, ⁓ guess, the key steps. Really the nitty gritty that you'll be doing is deciding what classes you're including, labeling them, defining them, and adding any additional information you think would be helpful to and their use case.
Christoph: Maybe coming back to the pizza example, path dependent is this? I could, maybe want to have an ontology of food and then I start differentiating classes based on their geometrical features or ⁓ based on macronutrients, right? Like the first few steps already kind of define how it propagates them. So would you say it's a very path dependent process or ⁓ with this you kind of.
Paulina: Hmm. Yeah.
Christoph: arrive at ⁓ best ⁓ of say option in the end.
Paulina: It's difficult to say. I'm also quite biased, obviously, because I'm in this. So I'm hoping that we do it to the best of our ability. think there are path dependencies. You're starting with a domain that you assume makes sense in the first place. And then ⁓ you classes from as systematically as you can. But obviously, the methods you choose ⁓ depend, of determine. which classes or what content you're including. And then in terms of categorization, yeah, that's an interesting one. I really struggle with that. As part of my PhD, was developing the mechanism of action ontology, again, kind of captures processes through which work to change behavior. And ⁓ went back and forth quite a lot on the structure of that. So it took a really long time to arrive at what we produced. I will also say that we use within the BCIO a specific upper level structure. So it's called the basic formal ontology. And this is based basically a structure that is not domain specific. So it's supposed to be used by different types ⁓ domains ⁓ within which is so broad. So it's supposed to work for everyone. And it basically helps you define what something is, I guess going back to why it's called ontology. But ⁓ the way it kind of distinguishes between things is it for instance says that you have processes and you have continuance and processes are things that unfold over time. And we have to make executive judgments almost of what something is. ⁓ The thing is you can have multiple, multiple versions, but you can have, you can be actually quite precise. So you can have belief as a disposition or mental disposition. And then you can have something like thinking that, for instance, involves a belief or something like that. So you're not losing the process version of it, for instance, but you are kind of making an executive judgment, I guess, on how the label is being used within your ontology. And that does shape language. So it's I hope I haven't gone too technical there. a bit.
Olga: No, I think that that makes sense. But I ⁓ in process, you kind of draw on what the best available sources of knowledge you have to make these decisions. But it sounds like if you also involve stakeholders in the process, maybe ⁓ of path that you go down also depends on who you involve in that process and what disciplines they come from or
Paulina: Yes. Yes. Yeah. No absolutely.
Olga: of scientists than maybe practitioners or people with lived experience.
Paulina: Yeah, that absolutely all of those matter quite a lot. So instance, mean, the people we reach out to even if we're trying to be very broad and recruitment will often come more from the ⁓ US or from the UK, for instance, because, ⁓ well, center works ⁓ or more within behavioral science circles within those scopes. So our And we do make an effort to make sure that we're at least recruiting a number of participants beyond those circles. But yes, that. that will definitely be reflected in how we categorize things. I guess the advantage is, though, again, going back to the flexible framework thing, is going forward, we are trying to be quite flexible in how we take up feedback so that we also do make structural changes or add high-level classes that will meet needs ⁓ users talk about. So I guess that's... that kind of tries to address, I guess, that critique or that issue. But again, I do appreciate you frame a framework or an ontology a certain way, you are kind of constructing it to a specific path almost.
Olga: Can I also ask a question, well, linked to something that you said a bit earlier, sort of also about the more general versus sort of more specific sort of ontologies applied to use cases and sort of you mentioned another term there, which was causal or sort of a causal model. So do these sort of more
Paulina: Okay,
Olga: general ontologies, do they make causal claims or how do they relate to other representations of causality, like causal graph models or I will mention a term that we will discuss in more detail in a future episode, so acyclic graphs for example, which I suppose are causal graph models.
Paulina: So these broad anthologies tend to try to avoid them to a certain extent unless something again holds almost all the time and is kind of agreed on by consensus. case based ontologies ⁓ Maya Brown ⁓ developed the copper ontology for physical activity recommendations can use rule based sort of or to help kind of make more causal inferences or at least sort of suggest potential ⁓ causal relationships I ⁓ in terms of of saying okay this barrier is likely to lead or ⁓ you should be addressing this barrier with this recommendation where it's not directly maybe causal, but there is a clear association between how you think things would work together. And this is more about detailed classes so that you would have specific recommendations that is linked to a specific barrier. And I guess depending on how you design your system. could make some causal inferences basically based on the data that you're working with. So if it's again very much linked to a use case and you're working with data, I think you can make those assumptions. When it comes to these more broad ontologies, again, we tend to only do them on a very high level. For instance, that an intervention works through a mechanism of action where the mechanism of action isn't specified to produce ⁓ outcome So again, you would kind of avoid doing it on a lower level. I do think you can use the classes and some of the relationships to build maybe some of these models where you draw on it almost as your concepts or your categories that you're using, but there's a lot of work that would have to be done additionally, obviously, and it would be informed by your data. And then you would be able to talk about causality and the direction of relationships and so forth. you
Olga: So just a follow up there because it still sounds like, even for the broader ontology, which is saying that ⁓ a particular intervention leads a specific mechanism of action and then an outcome in terms of behaviour change. So isn't that making causal, well, assumptions about the relationships? But kind of, well, very broadly, I suppose. Yeah.
Paulina: Mm-hmm, of course. Mm. Yeah. Yeah. It is. Very broadly, yes. So a lot of ontologies won't. We are trying to have those relationships on a high level. So it's something that we're working on still, and I don't think we do it perfectly. But yes, on a very high level, you are saying that. But that is obviously caveated with that it will depend on the context. ⁓ how it's delivered and so forth. And I think if you have that many caveats, it doesn't make sense on a high level. But when it comes to kind of the granular classes, you can't really assert that relationship so that I would never feel comfortable within this ontology to kind of say that this behavior change technique usually works through this mode of delivery well. the medium of delivery. So I would really be cautious not to make those assumptions within an ontology that's that broad and is trying to capture that much. I guess that's what I'm kind of trying to get at. ⁓
Christoph: So there's no inherent reason to not have causality. It's just very hard to make it work.
Paulina: guess so. Yeah, I think for most topics that are that broad ⁓ this complex, so for instance, behavior change interventions, I think you'd to be really cautious when you're making those relationships. ⁓
Olga: And another follow-on question, because when you were talking through the steps of how to develop an ontology, you also mentioned that want to evaluate it at different steps, ⁓ you mentioned a few different ways in which you could evaluate it. So checking in with stakeholders, does it make sense? But then, can ⁓
Paulina: Hmm, of course. Yes.
Olga: essentially fit an ontology to data or it not, well, it's not the statistical model, I suppose, but sort of how could you it against data?
Paulina: So. You definitely can. So I feel like I lack a bit of knowledge on how a lot of people do it. So what we did was because the ontologies or an important use case of the ontology is again, going back to annotations, annotations. That's why we tested inter-rater reliability between two coders who were familiar with the ontology first who applied it to ⁓ papers to that specific ⁓ ontology. then for two researchers who were unfamiliar with the ontology who again tried to use it for 50 papers. ⁓ that was how we approached it and I guess that that is data-based. normally I think if an ontology has some kind of goal or a thing that it should perform and you can compare it against data to show whether it is doing that successfully or not, that's how would approach it.
Christoph: Maybe trying to think pragmatically about this or how I could use it in my research. Let's say I want to run a meta analysis and there's a well-developed ontology, like the behavior change intervention ontology out there. And there are a lot of articles that are annotated, right? Then I could kind of search for the right labels, classes that I want to relate and then find the articles and see the text. that links that is annotated is that kind of the end goal or.
Paulina: Maybe to a certain extent. think if you are doing an evaluation on whether the BCIO actually sort of increases searchability, then yeah, think, I understand that correctly?
Christoph: let's say so behavior change. Maybe I want to see how ⁓ an intervention like forming if then sentences, how this ⁓ affects ⁓ behavior. could I use this ontology to find relevant texts, relevant research and ⁓ the relevant within the research since it's annotated?
Paulina: Mm-hmm. Okay, yeah.
Christoph: Or is this on another level?
Paulina: I think that's a bit different maybe just because I would say that the ontology wasn't necessarily developed specifically to do that. if you kind of ⁓ feedback and say, you know, I need to do this, this is my goal and you want to test it against that, you could certainly, I think that would be a good use case. Or I guess that's a bit more of a small scale evaluation that I'm thinking of there, where you're looking at a very specific topic that the ontology wasn't specifically developed for. So I think it might be a very harsh evaluation the framework. But certainly, if you develop a specific part of the ontology for that, then yes, I think that would make sense as an evaluation. I guess another example would be, so I to... I don't know how Maya will feel about me constantly mentioning her ontology, but going back to the copper ontology. So for instance, for the recommendations that Maya has linked to kind of the barriers for physical activity, if you would want to evaluate that in the real world, you could do that by seeing whether, you know, giving those recommendations actually address those barriers within ⁓ when you run an intervention, that could for instance be something and then whether the behavior actually increases. So that's maybe a more practical way of evaluating it.
Christoph: Okay, so with those more encompassing ontologies, the goal is not to then zoom in on ⁓ specific relation. It's more to look at it ⁓ a whole.
Paulina: Yes, but I guess the relationships, whether they hold, it does make sense in terms of is it logically coherent the ontology of like, do I actually is it compatible with the literature in that? So it just would be a small scale evaluation of it or a part of it.
Christoph: So, but then it kind of feels like the end, like the goal of doing an ontology is to have the ontology in itself, right? And not to it ⁓ a tool to answer specific research questions. So I guess I'm asking ⁓ how it be pragmatically used? ⁓
Paulina: So what's... Yeah, Yes. ⁓ I think that's an excellent question. Thank you. So I think the challenge that we have is that some of the classes we have are quite broad. So developed the ontology based on looking more at smoking literature, for instance. ⁓ So the we developed are also better adapted to capturing that information.
Olga: Okay.
Paulina: But in practice, do have a lot of classes that are lot more generalist, but that's where we really go into much more detail. So that's, think, the interesting one on now looking at different behavioral domains or applying it in practice, what is actually missing. So, for instance, if you're looking at a specific... So for instance, has ⁓ Robert has worked with ⁓ a looking at driving behaviors and they're trying to annotate outcomes of driving behaviors. And what they're finding is that they need a lot more classes to help capture that information because the BCIO, it does have 2,500 classes, but doesn't have kind of that level of detail required for those outcomes. So they need to develop new classes that need to then be added to the BCIO. So ideally, if we think about an ontology as a community tool, which it really is meant to be, so that's what it is. within biomedicine, looking for instance at the gene ontology, people will start at some point developing their own classes or making suggestions for the BCIO so that you don't just apply it as it is and assume that it's kind of a prescriptive tool, but you'll actually start saying these categories are missing, please add them in. that kind of thing, or you develop your own ontology that is hopefully compatible with the classes that are within the BCIO or other ontologies within behavioral sciences.
Christoph: But so sticking with the example of driving behavior, ⁓ if I have my ontology on driving behavior, then what can I learn from it about driving behavior?
Paulina: Mm hmm. Yeah. Well, ontology is a tool. ⁓ you will ⁓ how you organize different types of ⁓ driving You'll give people as well a tool to capture or describe driving behaviors more consistently. You will say things like, ⁓ you should be measuring all of these different aspects. about driving behavior, but it is really a tool and then how you use it really depends on your research questions. So an ontology, I often see them just as supporting frameworks that then have to kind of be used flexibly within a specific research question.
Olga: And then so a little bit more then about how to use them as tools and sort of ⁓ a class isn't available, you add a new one to an existing ontology or do you start developing your own? And to me that links a little bit also to some of the more philosophical questions ⁓ about, I suppose it is the ontological questions of what exists, then also ⁓ of our epistemological stance ⁓ ⁓ we take quite a realist stance that these things kind of exist independently of ⁓ humans some sort of interpretation or maybe the other end of the spectrum that we're fully constructivist and sort of ⁓ thinking these are very dependent on the person who is defining the class, example. because I suppose making the decision as well, whether it fits within existing one or whether you should create your own might depend on whether you think that ⁓ the of the class provided is sort of commensurate with your own one or you have, yeah. Do you have any thoughts on those bigger questions?
Paulina: Yeah, can, I can certainly try. So I think in terms of perspective, ontologies don't necessarily have kind of, I guess, one... epistemological stand, will really depend. If we go back to the basic formal ontology that I talked about, which is this upper level, which stems from for the work that was done primarily in biomedicine, although it extends now beyond that. So we're reusing it as well for the change intervention ontology. So that does have realist assumptions that are being made. And ⁓ an advantage of using this upper level and also sort of reusing classes generally is really this interoperability concept that you can both reuse classes so you don't have to go through the additional work of creating classes in the first place and you can also integrate data using the same structures basically where you can draw on different ontologies or data that has been annotated with different ontologies as well so that's kind of the advantage of aligning with some of these upper levels or even aligning with the BCIO. When I do say though that they have sort of a realist stance, I think in practice, that's not really what it looks like, because all of these ontologies are being updated over time. And I think sort of for Biomedical ontology is, yes, ⁓ are more stable classes, of course, because you have concepts like cell division, which can proven or observed very easily. But as we're going into more behavioral sciences and so forth, when we're actually talking about having these frameworks and updating them and asking people for feedback, I think we're really acknowledging that a lot of these constructs that we have or the classes that we have constructed. they aren't ⁓ think realism is a strong word to use for the BCIO and this is more my opinion of how it works in practice but I would say it's much more of a practical realist. ⁓ standpoint that we have, because we do very much acknowledge that different people will conceptualize things differently and that as the developers, we're coming in with a specific perspective and we do want to update them based on what other people think. So we do have kind of more mechanisms in place now that will allow us to do that. So we have something called an issue log on GitHub where anyone can really provide feedback and then we review that feedback and and respond to the people who raise the issue and ideally we address it. those mechanisms are in place and sorry this is a very long answer to your question of should you actually align or develop your own ontology. But I think in terms of that it really depends on how people operate and practice I think as well. But A key issue, think, in developing your own ontology is really that technical expertise sometimes and having the resources to do it, which is a really unfortunate thing because ideally you do want people to develop ontologies within their specific topic, topics that they're experts in basically. But ⁓
Christoph: So does your answer slightly mirror something you said earlier? So of the higher order concepts and there, you might be more safer to assume a realist stance, but then the further down you go, the more into detail you go, the more ⁓ it of might be ⁓ hard make this claim. And earlier you said ⁓ that annotations for the more detailed ⁓ classes also fewer and ⁓ less the ontology kind of becomes more hard to describe or there's less information the more detailed you get. Is that it?
Paulina: Yes, yeah, I think that's good way of describing it.
Christoph: So it's more fuzzy at the borders basically.
Paulina: Yes, I think it is definitely more fuzzy at the borders. I guess also listening to your previous episode where Anik was talking about concepts and how context dependent they are, instance, how self-efficacy can look very different across different contexts and for different behaviors. I think there's definitely scope. to elaborate on things. And I don't think one framework should necessarily become so broad that it tries to capture everything. I think sometimes, you know, reusing definitions that make sense to you as an upper level. So let's say the self-efficacy class, you're like, okay, this is a useful, broad concept that aligns with how I think about it. then having ⁓ your own subclasses for the different types of self-efficacy that you think exists within your field providing feedback to us as developers that concept ⁓ doesn't for that use case so that we kind of have to rethink, do we to make updates? I think there's kind of a nice midway where there are some opportunities there to be still a useful framework, but not be prescriptive and ⁓ tell that this is what this concept is.
Christoph: Now got a little bit more philosophical, but maybe we could actually go the other way and ⁓ more pragmatically about it some more. Because I ⁓ found quite interesting that in the UK where you're based, it seemed to me like there's already some integration these ⁓ ontologies into or at least efforts to do that. Could you describe that a little bit more?
Paulina: Yeah. Sure, so I think the main uptake would be in academia for in terms of especially evidence synthesis. I think most people are quite interested in. using them for their systematic abuse, instance, where they'll use part or some more ambitiously try to use multiple different ⁓ lower ontologies. But in the show notes I included Carolina Silva's ⁓ review used for instance only the or some classes that were specific to delivery from the BCIO. So I think that's a good example of kind of not necessarily engaging with the entire framework, which is extremely complex and would take a lot of time, and only using parts of it that are really relevant to your research question, for instance. There are also more pragmatic ways of approaching it, where, for instance, Ella House, who used to also work for the Human Behaviour Change Project that started kind of the behaviour change intervention ontology, that, sorry, who just used the upper level to structure how she's extracting or organising evidence. ⁓ I think there are different, especially in evidence synthesis, there are interesting moves there. Robert West collaborates with lot of different teams across the UK to particularly try to improve capability on people being able to use ontologies to support evidence synthesis as well. So those are a couple of examples. In terms of practice, so I ontologies can be kind of supportive tools in practice. So ⁓ again, probably biased by the behavior change intervention ontology. So I think with other frameworks, if you kind of link them to it, can ⁓ of use for intervention development, for instance, but I don't think it's a very straightforward way for them work together. Maya's ontology is a much better example of the potential of how ontologies could be used in practice to inform sort of intervention development. But again, I think the core limitation there of translating it into practice is really the technical expertise. do you think that practitioners will always need some support, either in terms of technical expertise or from people who are more familiar with ontologies to kind of translate that into workflow? And some ways we can do that is by providing more guidance and more user-friendly interfaces as well with the ontology. So those are a couple of options as well.
Christoph: But so there's the intention to link it to a practice, but it's not done yet so much.
Paulina: Actually, sorry, I did forget about this, but Robert West, again, working on something called the Cactus tool, which draws on both the behavior change intervention ontology and the behavior change wheel. The behavior change wheel is this practical framework that is meant help with intervention development. So drawing on those two frameworks, he basically tries to have a simpler way of identifying behavior change techniques to target specific mechanisms of actions, ⁓ instance. So I do think ⁓ is work to create tools that are ⁓ ontologies, but do give kind of an easier interface.
Christoph: maybe looking more ahead, you mentioned that ⁓ there's also the idea or the intention to make it machine readable, right? And now with AI, would you say that these intentions to make it practically useful could be helped by AI or do you see any way that this might up the ontology development or move it in a different direction?
Paulina: So the ontologies that we produce and that we often find useful are already computer readable. So that's maybe something I should have pointed out more in the development process thing. So that's also a key step of turning them into a computer readable format. in terms of ⁓ them, so there's definitely scope in terms of using them for machine learning applications, both in terms of kind of. automating extraction more as well as making novel predictions ⁓ decision support systems. So those are, I think, the key elements where there's already work ongoing to use ontologies more. terms of the ontology development, I know as much about integrating AI the development process itself. in the past, we've seen, for instance, when we use these, well, when the BCIO was used for, for instance, ⁓ extraction and prediction systems, sometimes results were better when classes were separated into smaller things so that instead of health practitioners, you had and nurse. And when you separated those out, the performance was better, for instance. So I think in that case, it can almost be another feedback for developers to say, OK, it works better when I more granular classes for these types of things. So in that way, I think it can feed back into development.
Christoph: Yeah, interesting. Maybe the last and maybe most difficult question. What can I take away from this talk ontologies as a researcher and also just as person in general, even though it's a very theoretical topic, right? ⁓ Maybe ⁓ there's something be taken away.
Paulina: think just about them and working within this field, think ⁓ of the key things I'm quite interested in about is kind of how we can take more ownership of the frameworks that we use. As we all know, we're not the best at having flexible or updated frameworks, or even our theories are quite static. So I think really for all of us as a community, really taking ownership of the frameworks, providing feedback, providing critical feedback, but also as the developers of these frameworks, actually listening to that feedback and being responsive and actually trying to help them. become better is a core part of the work we should be doing. So maybe that kind of works then.
Olga: That sounds like an excellent So that takes us to the end of this podcast. Thank so much Paulina for sharing your wisdom this area. And Christophe, Christophe do you have any final things to share?
Christoph: I mean, we will add the things, the information you mentioned in the show notes, right. And some further readings. and I think I will probably think some more about pizza ontologies now. And ⁓ we have another episode on ontology. So I feel like there's a lot to discuss there. Right. Okay.
Paulina: Thank you guys so much for having me. I've really enjoyed it. I hope I didn't make it too technical, although I feel like it ended up being quite technical.
Christoph: That's what the podcast is there for. Right. All right. Thank you. Bye bye.
Paulina: True. Thank you, bye.
Olga: Olga Perski from Stockholm University.
Christoph: And this is a podcast, which is part of the special interest group, conceptualization, formalization and measurement by the European health psychology society. We are recording this podcast on the 27th of March, 2026. And our guest today is Dr. Paulina Schenk. senior research fellow at the university college, London. And hi Paulina. I would like you to introduce yourself briefly now.
Paulina: Hi, thanks so much for having me and I'm really excited to be here. It's such a great podcast. So really looking forward to chatting with you guys. But yeah, my name is Paudina Schenck and as you said, I'm based at University College London. yeah, I mainly work on ontologies that are used for behavioral sciences.
Christoph: Right. I think then we can already ask the first obvious question. What ontologies?
Paulina: ontologies ⁓ basically ⁓ of classification systems that include basically categories of things, which we often refer to ⁓ classes ⁓ relationships these classes. just to give you maybe an example, you can have really classes about anything that you're interested in in terms of scope. And then you would have a label ⁓ definition and a unique computer readable ID in the ontologies we talk about at least. So the ones that are more related to computer science. There's of course also ontologies sort of within philosophy, which is something slightly different, ⁓ which can talk about in a bit as well. But sorry to get back to the example. ⁓ you ⁓ an example we often use is just a pizza example where, for instance, you could have a class of different types of pizzas and then you would have relationships between them where you could have, for instance, have subtypes of pizzas such as a mushroom pizza or a margarita pizza. I guess a nice thing with anontologies is that you don't necessarily just have these simple hierarchical relationships of a and a subclass, which would be like a pizza and a mushroom pizza. But you can also have more complex relationships so that you, for instance, say a pizza has topping mushrooms. And sort of that helps then form define what that type of pizza is. Now within sciences or health psychology this is obviously a bit more complex but can for interventions that's a good example where you can specify that you have interventions that have content and types of delivery for instance.
Olga: Great. So, I mean, just thinking about sort of the problem that ontologies are trying to address. So kind of why do we need them in the first place?
Paulina: So I think it depends on which ontology you're talking about, because there are different types of ontologies. think if people are kind of interested in the European Health Psychology Society, they'll often be familiar with the behavior change intervention ontology, because we present about that quite a lot. But there are different types of ontologies. I think that's quite important to appreciate as well. So a huge advantage of them is that they have this computer readable attributes. engineers are often really interested in them as well to sometimes integrate data, but sometimes also just use logic rules. So kind of in complex machine learning context that is quite interesting as well. But if we go back to kind of behavioral sciences or health psychology, so one of the key issues, I guess, that behavior change intervention ontology is trying to solve is sort of import also the jingle jangle fallacy that we have and that we're producing a lot of evidence that we can't actually integrate because we use language very differently. But also another aspect is really this data integration aspect of we're producing evidence at a rate that is much higher than humans can kind of keep up with. So there's that aspect as well of can we use ontologies to integrate our data better and then produce usable insights.
Christoph: Right. one of our previous episodes, I think it was episode one and two actually, ⁓ already briefly talked about the Jingle-Jangle Fallacy. Could you just briefly recap it and how ontologies ⁓ solve it?
Paulina: Sure, so the Jingle-Jangle fell asleep basically. You guys covered that quite extensively already, but refers to that we'll use labels for basically concepts with different definitions or vice versa, that ⁓ you'll have concepts with the same label that have different definitions. And the same issue also exists for measurement. So for ontologies, I think the clear solution in the first instance is more on the language aspect. So kind of moving towards having a more shared language and making the differences between concepts more formal so that if you, for instance, have a different definition for something that becomes a different class and that you don't use the same label twice if you can avoid it, at least within the same ontology, so that you capture differences much more precisely. So even sometimes having different classes for belief about capability and self-efficacy if the definition is different, so if they're capturing different things. but then you don't use self-efficacy twice as a label.
Olga: So something you already referred to, so you were saying that, well, we talk about ontologies, ⁓ there ⁓ ontologies the philosophical domain, but ⁓ you're to it here is a little bit different. why do you call it an ontology and ⁓ sort of how, in any way, does it relate to the philosophical term?
Paulina: some ways it does relate. ⁓ I guess the term ontology and philosophy refers to the study of what is and the we're talking about are tools or representational systems to capture or represent ⁓ what ⁓ basically what kind of things we assume exist and how they relate to one another. So that's the parallel. That being said, I do think it's sometimes quite ironic we use a term that causes that much confusion because I feel like when people first hear about ontologies, they always assume that it's about the philosophy rather than actually about the tools. So I do think that's a fun one. And I do wish they were called something different, to be quite honest.
Olga: So it sounds like it's almost so within ontology as kind of a knowledge representation tool, ⁓ but ⁓ it still kind of connected to what you think exists in that domain?
Christoph: because it's...
Paulina: It is connected. It's a way of formalizing what exists. But I think with ontology you often refer to more ontological standpoint and so forth. So I think that's what causes the confusion maybe where people don't think about it as much as a practical tool, which is often what we developed. But yeah, they were absolutely parallels. I completely understand how people have come to name it that, but at the same time, sometimes I think it just creates a bit of confusion when people first learn about them.
Christoph: to add another term to it, taxonomy. So when you describe ontologies, just ⁓ isn't it just a taxonomy? Could you ⁓ it in more detail? ⁓ taxonomy, right, is like, are animals, there are mammals, there are cats and dogs, right? So kind of just a hierarchical relation. So how does an ontology go beyond that?
Paulina: Yeah. Mmm. Of course. Yeah. Yeah. So, yeah, ontologies, basically, refer to them, I guess, as dictionaries or taxonomies on steroids. in taxonomies, you would basically mostly just have hierarchical relationships. So you would say, for instance, yes, ⁓ a cat is an animal. a dog is an animal and we'll have taxonomies as a backbone of our ontology. So of the relationships within the BCIO ⁓ simple hierarchical relationships such as a cat is an animal. But you have these more complex relationships. So if we go back to the pizza example, the mushroom is a topping of pizza is not something you would capture in a taxonomy. So you have these more complex relationships ⁓ allow you to relate things to one another in a more complex way. And then you can make logical reasonings about those. So again, we go back to interventions, kind of being able to say that content connects to intervention delivery in this way so that it has an intervention delivery or something like that.
Christoph: And with cats and dogs, so in a taxonomy, I can only say they are both mammals, but with an ontology, I could say dogs hunt cats or something like this. So I can specify the relation between them. Taxonomy on steroids. Okay.
Paulina: ⁓ yes. ⁓ a bit more or I mean that would be a ⁓ I would so with that relationships I'm being probably quite picky but because it's not necessarily established that all cats sorry all dogs will hunt cats I don't know if I'd specify that but you could specify cats I don't know have whiskers if that actually holds for all cats for instance so that you help kind of I guess you could in some ways relate to dogs and cats, but I can't think of anything off the top of my head. Yeah, that works. There you go. So you have commonalities based like that. So they have part leg or forelegs. And then, yeah.
Olga: both have four legs.
Christoph: But I think there's an important point in there that ⁓ ontologies, do they claim universality so that the relations always hold?
Paulina: Hmm. I think that's a bit of a difficult one where not always, but you would assume that they hold most of the time. are still ⁓ cat if you lose one of your legs or if you're born without one. So, but you could specify that cats generally have legs. So, but I completely appreciate your point of it's not 100 % clear sometimes whether it is universal or not. But I think there are, you are allowed to have exceptions, generally you would expect that to hold, the relationship to hold if that makes sense. it depends on what ontology we're talking about again, because some ontologies are just use case driven and there it will be different. And there you can start talking maybe about more causal relationships. But if you're talking about more general ontologies that are meant to be used more widely, such as the BCIO, so the Behaviour Change Intervention Ontology, then you would want to have relationships that hold most of the time at least.
Christoph: Okay, maybe before we go into the details about ontologies, maybe we could delineate it from another term, namely dictionaries, which you already mentioned as well. So because you said, yeah, a cat has four legs and so on, which is kind of a description, you would also find in the dictionary, but then an ontology goes beyond this description by also relating the dictionary entries, so to say, to other entries. Is that it? Okay.
Olga: So.
Paulina: Mm-hmm. Yes. Or I guess, again, unfortunately, it depends on your ontology. But on a, especially with them, I'm going to draw mostly on the BCIO because that's the one I'm most familiar with. But I'll try to kind of have a bit of a nuanced approach when I talk about ontologies and give examples of others where I can. But within the BCIO, for instance, you have on the higher level classes, so the broadest classes, you have clear relationships. But the more detailed classes, we actually have less semantic relationships between them because it becomes so difficult to assert relationships. So just to give you an example, we don't have as many relationships specified between the different mechanisms of actions or potential mechanisms of actions, which are basically processes through which interventions ⁓ work change behavior, because they can function very differently across different contexts and so forth and for different behaviors, of course.
Olga: So you already mentioned some of the benefits of using ontologies is in synthesis or with research growing ⁓ at a pace much faster than probably we can work through systematic reviews on different So do you want to us a little bit more about that and maybe give some examples of how ontologies are used in evidence synthesis.
Paulina: Yep, that would be great. Sounds good. So I guess that's kind of... I think an application that's quite interesting because we do evidence synthesis and a lot of different teams work on developing these annotation schemas or coding schemas that end up being very different. So it's actually very difficult to integrate data. So we've been working also with teams that work with large data sets or with ⁓ ongoing systematic reviews and then end up kind of with data that is completely incompatible so that they can actually compare evidence. So the idea is that if you use really any framework, but also just an ontology, if it's well developed, to develop those annotation schemas or again coding schemas, then you'll have a more systematic way in which you're integrating your data. So I think that's really an interesting one where there is a lot of potential. Then of course, if you have a data set that is ⁓ readable, the aggregation also becomes easier if you want to, for instance, look at metadata from several systematic reviews or something like that. So I think that's really example where the potential is great, but we do still have to do a lot of work on it. I think for some annotations, it's maybe slightly simpler. So for instance, if you're looking at behavior change techniques and the question of presence and absence. But even there you need detailed coding guidance and people can interpret the categories differently. So it should be approached with caution. And I think we should be really transparent about how we're coding things. ideally you use, for instance, the ontology, but then you provide feedback as well on classes to say this one doesn't work or I had to make these assumptions. And then we make that explicit in the framework. but for other things like outcomes, so behavioral outcomes. I think that's still something we need to work on much more to see how the different categories can actually help maybe standardize how we capture knowledge from different papers about behaviors, for instance.
Olga: So just a question on that. because kind of at face value, it would seem to me that the sort of coding instructions and sort of annotations, that that would also relate to the definitions of the different labels. So if the definitions are quite long and specific, shouldn't that then help with the coding instructions or kind of how do they relate?
Paulina: What's? Mm. That's a really excellent question. Thank you. So we have definitions that are actually relatively brief. So we develop definitions based on specific principles, which say that they should ideally be one sentence and so forth, so that they capture kind of the essential attributes, but they're also concise and understandable. And that, I think, can be criticized because concepts are very complex. So we do have additional entries that can be added to an ontology where you can have sort of comments on different classes so that people can elaborate on what the definition means in more detail, how it can be measured and so forth. But we haven't been able to develop that that much yet because we're developed, we're working with a lot of different classes. So I think also maybe going back to the previous episodes where you talked more about the importance of conceptualization, I think that's definitely an element where we need to work on it more. So if people send us back their coding guidance or tell us where the definitions are or are missing things, we update the ontology. That's maybe something I should have said earlier, but I think really one of the best things about ontologies is that they need to be maintained and they should be updated both based on evidence and feedback. I think that's kind of where the potential lies. But ⁓ yes, so Again, thank you for the question.
Christoph: Maybe it would be nice to actually talk about how you come up with an ontology, but ⁓ before that, you mentioned two terms that I think are probably quite specific, which is annotation and coding. They probably have a specific meaning with ontologies, right? Maybe you could explain that.
Paulina: Okay. Yeah, it's actually quite similar to how you'd approach it in sort of qualitative coding, I guess, if you're using some kind of framework. So it's basically using a class from the ontology ⁓ that to a text ⁓ a paper, for instance, or it can also be within a data set where you say, OK, this text is this. class. So that's how you basically annotate. it's linked, the text basically becomes linked to the class, if that makes any sense. Of course. So an example would be if I'm annotating using the Behaviour Change Techniques ontology, which is about sort of the content of
Christoph: Could you give an example?
Paulina: of an intervention, so basically what we do there. So let's say I find a text that where practitioners were talking with patients and they asked them to set goals for their physical activity. So that text I would think is a goal setting ⁓ change technique. So I would use my class to basically annotate that text. So that means that the behavior change technique was used within that intervention. And then of course you could have additional annotations, but that really depends on annotation schema, I guess. For instance, are you recording how often that behavior change technique was delivered and so forth? Does that kind of make sense?
Christoph: Yeah. Yeah. Thank you. And coding was the other term.
Paulina: Coding used interchangeably. I use coding to serve as a simpler word instead of annotation, but maybe not.
Christoph: ⁓ okay. So yeah, maybe we can talk about how you come up with an ontology because ⁓ almost sounds like you're creating a new ⁓ with new words and new grammatical structures and everything. So it seems quite a big endeavor. So maybe you could ⁓ tell us in principle you come up with this and also specifically for the behavior change ontology, right?
Paulina: Sounds good. ⁓ So there isn't hard or fast rules on how to develop an ontology. Generally what you would be doing is first specifying your scope. So what you want your ontology to cover because these are systems for a specific domain. So you have to specify what that domain is. Then will tend to look at what already exists in literature. So what's out there. And this can kind of looking at existing classification systems, existing terms that are used out there, for instance, by looking at papers. Then ⁓ ideally you will some kind of stakeholder consultation ⁓ or ⁓ you'll of people whether what you've produced makes any sense. And ⁓ you will kind of, yeah, what you do next? ⁓ Then have some kind of use test. So depending on what you want your ontology to do, you'll apply the classes ⁓ and ⁓ if, well, working at all. And that can basically mean, do my definitions make sense ⁓ when I try apply them in practice? Do the relationships hold and so forth. and then you will tend to have an evaluation of your ontology. again, applying those either to use case or having sort of competency questions where you say, is my ontology kind of coherent? Are the relationships logical and so forth? So ⁓ those, ⁓ guess, the key steps. Really the nitty gritty that you'll be doing is deciding what classes you're including, labeling them, defining them, and adding any additional information you think would be helpful to and their use case.
Christoph: Maybe coming back to the pizza example, path dependent is this? I could, maybe want to have an ontology of food and then I start differentiating classes based on their geometrical features or ⁓ based on macronutrients, right? Like the first few steps already kind of define how it propagates them. So would you say it's a very path dependent process or ⁓ with this you kind of.
Paulina: Hmm. Yeah.
Christoph: arrive at ⁓ best ⁓ of say option in the end.
Paulina: It's difficult to say. I'm also quite biased, obviously, because I'm in this. So I'm hoping that we do it to the best of our ability. think there are path dependencies. You're starting with a domain that you assume makes sense in the first place. And then ⁓ you classes from as systematically as you can. But obviously, the methods you choose ⁓ depend, of determine. which classes or what content you're including. And then in terms of categorization, yeah, that's an interesting one. I really struggle with that. As part of my PhD, was developing the mechanism of action ontology, again, kind of captures processes through which work to change behavior. And ⁓ went back and forth quite a lot on the structure of that. So it took a really long time to arrive at what we produced. I will also say that we use within the BCIO a specific upper level structure. So it's called the basic formal ontology. And this is based basically a structure that is not domain specific. So it's supposed to be used by different types ⁓ domains ⁓ within which is so broad. So it's supposed to work for everyone. And it basically helps you define what something is, I guess going back to why it's called ontology. But ⁓ the way it kind of distinguishes between things is it for instance says that you have processes and you have continuance and processes are things that unfold over time. And we have to make executive judgments almost of what something is. ⁓ The thing is you can have multiple, multiple versions, but you can have, you can be actually quite precise. So you can have belief as a disposition or mental disposition. And then you can have something like thinking that, for instance, involves a belief or something like that. So you're not losing the process version of it, for instance, but you are kind of making an executive judgment, I guess, on how the label is being used within your ontology. And that does shape language. So it's I hope I haven't gone too technical there. a bit.
Olga: No, I think that that makes sense. But I ⁓ in process, you kind of draw on what the best available sources of knowledge you have to make these decisions. But it sounds like if you also involve stakeholders in the process, maybe ⁓ of path that you go down also depends on who you involve in that process and what disciplines they come from or
Paulina: Yes. Yes. Yeah. No absolutely.
Olga: of scientists than maybe practitioners or people with lived experience.
Paulina: Yeah, that absolutely all of those matter quite a lot. So instance, mean, the people we reach out to even if we're trying to be very broad and recruitment will often come more from the ⁓ US or from the UK, for instance, because, ⁓ well, center works ⁓ or more within behavioral science circles within those scopes. So our And we do make an effort to make sure that we're at least recruiting a number of participants beyond those circles. But yes, that. that will definitely be reflected in how we categorize things. I guess the advantage is, though, again, going back to the flexible framework thing, is going forward, we are trying to be quite flexible in how we take up feedback so that we also do make structural changes or add high-level classes that will meet needs ⁓ users talk about. So I guess that's... that kind of tries to address, I guess, that critique or that issue. But again, I do appreciate you frame a framework or an ontology a certain way, you are kind of constructing it to a specific path almost.
Olga: Can I also ask a question, well, linked to something that you said a bit earlier, sort of also about the more general versus sort of more specific sort of ontologies applied to use cases and sort of you mentioned another term there, which was causal or sort of a causal model. So do these sort of more
Paulina: Okay,
Olga: general ontologies, do they make causal claims or how do they relate to other representations of causality, like causal graph models or I will mention a term that we will discuss in more detail in a future episode, so acyclic graphs for example, which I suppose are causal graph models.
Paulina: So these broad anthologies tend to try to avoid them to a certain extent unless something again holds almost all the time and is kind of agreed on by consensus. case based ontologies ⁓ Maya Brown ⁓ developed the copper ontology for physical activity recommendations can use rule based sort of or to help kind of make more causal inferences or at least sort of suggest potential ⁓ causal relationships I ⁓ in terms of of saying okay this barrier is likely to lead or ⁓ you should be addressing this barrier with this recommendation where it's not directly maybe causal, but there is a clear association between how you think things would work together. And this is more about detailed classes so that you would have specific recommendations that is linked to a specific barrier. And I guess depending on how you design your system. could make some causal inferences basically based on the data that you're working with. So if it's again very much linked to a use case and you're working with data, I think you can make those assumptions. When it comes to these more broad ontologies, again, we tend to only do them on a very high level. For instance, that an intervention works through a mechanism of action where the mechanism of action isn't specified to produce ⁓ outcome So again, you would kind of avoid doing it on a lower level. I do think you can use the classes and some of the relationships to build maybe some of these models where you draw on it almost as your concepts or your categories that you're using, but there's a lot of work that would have to be done additionally, obviously, and it would be informed by your data. And then you would be able to talk about causality and the direction of relationships and so forth. you
Olga: So just a follow up there because it still sounds like, even for the broader ontology, which is saying that ⁓ a particular intervention leads a specific mechanism of action and then an outcome in terms of behaviour change. So isn't that making causal, well, assumptions about the relationships? But kind of, well, very broadly, I suppose. Yeah.
Paulina: Mm-hmm, of course. Mm. Yeah. Yeah. It is. Very broadly, yes. So a lot of ontologies won't. We are trying to have those relationships on a high level. So it's something that we're working on still, and I don't think we do it perfectly. But yes, on a very high level, you are saying that. But that is obviously caveated with that it will depend on the context. ⁓ how it's delivered and so forth. And I think if you have that many caveats, it doesn't make sense on a high level. But when it comes to kind of the granular classes, you can't really assert that relationship so that I would never feel comfortable within this ontology to kind of say that this behavior change technique usually works through this mode of delivery well. the medium of delivery. So I would really be cautious not to make those assumptions within an ontology that's that broad and is trying to capture that much. I guess that's what I'm kind of trying to get at. ⁓
Christoph: So there's no inherent reason to not have causality. It's just very hard to make it work.
Paulina: guess so. Yeah, I think for most topics that are that broad ⁓ this complex, so for instance, behavior change interventions, I think you'd to be really cautious when you're making those relationships. ⁓
Olga: And another follow-on question, because when you were talking through the steps of how to develop an ontology, you also mentioned that want to evaluate it at different steps, ⁓ you mentioned a few different ways in which you could evaluate it. So checking in with stakeholders, does it make sense? But then, can ⁓
Paulina: Hmm, of course. Yes.
Olga: essentially fit an ontology to data or it not, well, it's not the statistical model, I suppose, but sort of how could you it against data?
Paulina: So. You definitely can. So I feel like I lack a bit of knowledge on how a lot of people do it. So what we did was because the ontologies or an important use case of the ontology is again, going back to annotations, annotations. That's why we tested inter-rater reliability between two coders who were familiar with the ontology first who applied it to ⁓ papers to that specific ⁓ ontology. then for two researchers who were unfamiliar with the ontology who again tried to use it for 50 papers. ⁓ that was how we approached it and I guess that that is data-based. normally I think if an ontology has some kind of goal or a thing that it should perform and you can compare it against data to show whether it is doing that successfully or not, that's how would approach it.
Christoph: Maybe trying to think pragmatically about this or how I could use it in my research. Let's say I want to run a meta analysis and there's a well-developed ontology, like the behavior change intervention ontology out there. And there are a lot of articles that are annotated, right? Then I could kind of search for the right labels, classes that I want to relate and then find the articles and see the text. that links that is annotated is that kind of the end goal or.
Paulina: Maybe to a certain extent. think if you are doing an evaluation on whether the BCIO actually sort of increases searchability, then yeah, think, I understand that correctly?
Christoph: let's say so behavior change. Maybe I want to see how ⁓ an intervention like forming if then sentences, how this ⁓ affects ⁓ behavior. could I use this ontology to find relevant texts, relevant research and ⁓ the relevant within the research since it's annotated?
Paulina: Mm-hmm. Okay, yeah.
Christoph: Or is this on another level?
Paulina: I think that's a bit different maybe just because I would say that the ontology wasn't necessarily developed specifically to do that. if you kind of ⁓ feedback and say, you know, I need to do this, this is my goal and you want to test it against that, you could certainly, I think that would be a good use case. Or I guess that's a bit more of a small scale evaluation that I'm thinking of there, where you're looking at a very specific topic that the ontology wasn't specifically developed for. So I think it might be a very harsh evaluation the framework. But certainly, if you develop a specific part of the ontology for that, then yes, I think that would make sense as an evaluation. I guess another example would be, so I to... I don't know how Maya will feel about me constantly mentioning her ontology, but going back to the copper ontology. So for instance, for the recommendations that Maya has linked to kind of the barriers for physical activity, if you would want to evaluate that in the real world, you could do that by seeing whether, you know, giving those recommendations actually address those barriers within ⁓ when you run an intervention, that could for instance be something and then whether the behavior actually increases. So that's maybe a more practical way of evaluating it.
Christoph: Okay, so with those more encompassing ontologies, the goal is not to then zoom in on ⁓ specific relation. It's more to look at it ⁓ a whole.
Paulina: Yes, but I guess the relationships, whether they hold, it does make sense in terms of is it logically coherent the ontology of like, do I actually is it compatible with the literature in that? So it just would be a small scale evaluation of it or a part of it.
Christoph: So, but then it kind of feels like the end, like the goal of doing an ontology is to have the ontology in itself, right? And not to it ⁓ a tool to answer specific research questions. So I guess I'm asking ⁓ how it be pragmatically used? ⁓
Paulina: So what's... Yeah, Yes. ⁓ I think that's an excellent question. Thank you. So I think the challenge that we have is that some of the classes we have are quite broad. So developed the ontology based on looking more at smoking literature, for instance. ⁓ So the we developed are also better adapted to capturing that information.
Olga: Okay.
Paulina: But in practice, do have a lot of classes that are lot more generalist, but that's where we really go into much more detail. So that's, think, the interesting one on now looking at different behavioral domains or applying it in practice, what is actually missing. So, for instance, if you're looking at a specific... So for instance, has ⁓ Robert has worked with ⁓ a looking at driving behaviors and they're trying to annotate outcomes of driving behaviors. And what they're finding is that they need a lot more classes to help capture that information because the BCIO, it does have 2,500 classes, but doesn't have kind of that level of detail required for those outcomes. So they need to develop new classes that need to then be added to the BCIO. So ideally, if we think about an ontology as a community tool, which it really is meant to be, so that's what it is. within biomedicine, looking for instance at the gene ontology, people will start at some point developing their own classes or making suggestions for the BCIO so that you don't just apply it as it is and assume that it's kind of a prescriptive tool, but you'll actually start saying these categories are missing, please add them in. that kind of thing, or you develop your own ontology that is hopefully compatible with the classes that are within the BCIO or other ontologies within behavioral sciences.
Christoph: But so sticking with the example of driving behavior, ⁓ if I have my ontology on driving behavior, then what can I learn from it about driving behavior?
Paulina: Mm hmm. Yeah. Well, ontology is a tool. ⁓ you will ⁓ how you organize different types of ⁓ driving You'll give people as well a tool to capture or describe driving behaviors more consistently. You will say things like, ⁓ you should be measuring all of these different aspects. about driving behavior, but it is really a tool and then how you use it really depends on your research questions. So an ontology, I often see them just as supporting frameworks that then have to kind of be used flexibly within a specific research question.
Olga: And then so a little bit more then about how to use them as tools and sort of ⁓ a class isn't available, you add a new one to an existing ontology or do you start developing your own? And to me that links a little bit also to some of the more philosophical questions ⁓ about, I suppose it is the ontological questions of what exists, then also ⁓ of our epistemological stance ⁓ ⁓ we take quite a realist stance that these things kind of exist independently of ⁓ humans some sort of interpretation or maybe the other end of the spectrum that we're fully constructivist and sort of ⁓ thinking these are very dependent on the person who is defining the class, example. because I suppose making the decision as well, whether it fits within existing one or whether you should create your own might depend on whether you think that ⁓ the of the class provided is sort of commensurate with your own one or you have, yeah. Do you have any thoughts on those bigger questions?
Paulina: Yeah, can, I can certainly try. So I think in terms of perspective, ontologies don't necessarily have kind of, I guess, one... epistemological stand, will really depend. If we go back to the basic formal ontology that I talked about, which is this upper level, which stems from for the work that was done primarily in biomedicine, although it extends now beyond that. So we're reusing it as well for the change intervention ontology. So that does have realist assumptions that are being made. And ⁓ an advantage of using this upper level and also sort of reusing classes generally is really this interoperability concept that you can both reuse classes so you don't have to go through the additional work of creating classes in the first place and you can also integrate data using the same structures basically where you can draw on different ontologies or data that has been annotated with different ontologies as well so that's kind of the advantage of aligning with some of these upper levels or even aligning with the BCIO. When I do say though that they have sort of a realist stance, I think in practice, that's not really what it looks like, because all of these ontologies are being updated over time. And I think sort of for Biomedical ontology is, yes, ⁓ are more stable classes, of course, because you have concepts like cell division, which can proven or observed very easily. But as we're going into more behavioral sciences and so forth, when we're actually talking about having these frameworks and updating them and asking people for feedback, I think we're really acknowledging that a lot of these constructs that we have or the classes that we have constructed. they aren't ⁓ think realism is a strong word to use for the BCIO and this is more my opinion of how it works in practice but I would say it's much more of a practical realist. ⁓ standpoint that we have, because we do very much acknowledge that different people will conceptualize things differently and that as the developers, we're coming in with a specific perspective and we do want to update them based on what other people think. So we do have kind of more mechanisms in place now that will allow us to do that. So we have something called an issue log on GitHub where anyone can really provide feedback and then we review that feedback and and respond to the people who raise the issue and ideally we address it. those mechanisms are in place and sorry this is a very long answer to your question of should you actually align or develop your own ontology. But I think in terms of that it really depends on how people operate and practice I think as well. But A key issue, think, in developing your own ontology is really that technical expertise sometimes and having the resources to do it, which is a really unfortunate thing because ideally you do want people to develop ontologies within their specific topic, topics that they're experts in basically. But ⁓
Christoph: So does your answer slightly mirror something you said earlier? So of the higher order concepts and there, you might be more safer to assume a realist stance, but then the further down you go, the more into detail you go, the more ⁓ it of might be ⁓ hard make this claim. And earlier you said ⁓ that annotations for the more detailed ⁓ classes also fewer and ⁓ less the ontology kind of becomes more hard to describe or there's less information the more detailed you get. Is that it?
Paulina: Yes, yeah, I think that's good way of describing it.
Christoph: So it's more fuzzy at the borders basically.
Paulina: Yes, I think it is definitely more fuzzy at the borders. I guess also listening to your previous episode where Anik was talking about concepts and how context dependent they are, instance, how self-efficacy can look very different across different contexts and for different behaviors. I think there's definitely scope. to elaborate on things. And I don't think one framework should necessarily become so broad that it tries to capture everything. I think sometimes, you know, reusing definitions that make sense to you as an upper level. So let's say the self-efficacy class, you're like, okay, this is a useful, broad concept that aligns with how I think about it. then having ⁓ your own subclasses for the different types of self-efficacy that you think exists within your field providing feedback to us as developers that concept ⁓ doesn't for that use case so that we kind of have to rethink, do we to make updates? I think there's kind of a nice midway where there are some opportunities there to be still a useful framework, but not be prescriptive and ⁓ tell that this is what this concept is.
Christoph: Now got a little bit more philosophical, but maybe we could actually go the other way and ⁓ more pragmatically about it some more. Because I ⁓ found quite interesting that in the UK where you're based, it seemed to me like there's already some integration these ⁓ ontologies into or at least efforts to do that. Could you describe that a little bit more?
Paulina: Yeah. Sure, so I think the main uptake would be in academia for in terms of especially evidence synthesis. I think most people are quite interested in. using them for their systematic abuse, instance, where they'll use part or some more ambitiously try to use multiple different ⁓ lower ontologies. But in the show notes I included Carolina Silva's ⁓ review used for instance only the or some classes that were specific to delivery from the BCIO. So I think that's a good example of kind of not necessarily engaging with the entire framework, which is extremely complex and would take a lot of time, and only using parts of it that are really relevant to your research question, for instance. There are also more pragmatic ways of approaching it, where, for instance, Ella House, who used to also work for the Human Behaviour Change Project that started kind of the behaviour change intervention ontology, that, sorry, who just used the upper level to structure how she's extracting or organising evidence. ⁓ I think there are different, especially in evidence synthesis, there are interesting moves there. Robert West collaborates with lot of different teams across the UK to particularly try to improve capability on people being able to use ontologies to support evidence synthesis as well. So those are a couple of examples. In terms of practice, so I ontologies can be kind of supportive tools in practice. So ⁓ again, probably biased by the behavior change intervention ontology. So I think with other frameworks, if you kind of link them to it, can ⁓ of use for intervention development, for instance, but I don't think it's a very straightforward way for them work together. Maya's ontology is a much better example of the potential of how ontologies could be used in practice to inform sort of intervention development. But again, I think the core limitation there of translating it into practice is really the technical expertise. do you think that practitioners will always need some support, either in terms of technical expertise or from people who are more familiar with ontologies to kind of translate that into workflow? And some ways we can do that is by providing more guidance and more user-friendly interfaces as well with the ontology. So those are a couple of options as well.
Christoph: But so there's the intention to link it to a practice, but it's not done yet so much.
Paulina: Actually, sorry, I did forget about this, but Robert West, again, working on something called the Cactus tool, which draws on both the behavior change intervention ontology and the behavior change wheel. The behavior change wheel is this practical framework that is meant help with intervention development. So drawing on those two frameworks, he basically tries to have a simpler way of identifying behavior change techniques to target specific mechanisms of actions, ⁓ instance. So I do think ⁓ is work to create tools that are ⁓ ontologies, but do give kind of an easier interface.
Christoph: maybe looking more ahead, you mentioned that ⁓ there's also the idea or the intention to make it machine readable, right? And now with AI, would you say that these intentions to make it practically useful could be helped by AI or do you see any way that this might up the ontology development or move it in a different direction?
Paulina: So the ontologies that we produce and that we often find useful are already computer readable. So that's maybe something I should have pointed out more in the development process thing. So that's also a key step of turning them into a computer readable format. in terms of ⁓ them, so there's definitely scope in terms of using them for machine learning applications, both in terms of kind of. automating extraction more as well as making novel predictions ⁓ decision support systems. So those are, I think, the key elements where there's already work ongoing to use ontologies more. terms of the ontology development, I know as much about integrating AI the development process itself. in the past, we've seen, for instance, when we use these, well, when the BCIO was used for, for instance, ⁓ extraction and prediction systems, sometimes results were better when classes were separated into smaller things so that instead of health practitioners, you had and nurse. And when you separated those out, the performance was better, for instance. So I think in that case, it can almost be another feedback for developers to say, OK, it works better when I more granular classes for these types of things. So in that way, I think it can feed back into development.
Christoph: Yeah, interesting. Maybe the last and maybe most difficult question. What can I take away from this talk ontologies as a researcher and also just as person in general, even though it's a very theoretical topic, right? ⁓ Maybe ⁓ there's something be taken away.
Paulina: think just about them and working within this field, think ⁓ of the key things I'm quite interested in about is kind of how we can take more ownership of the frameworks that we use. As we all know, we're not the best at having flexible or updated frameworks, or even our theories are quite static. So I think really for all of us as a community, really taking ownership of the frameworks, providing feedback, providing critical feedback, but also as the developers of these frameworks, actually listening to that feedback and being responsive and actually trying to help them. become better is a core part of the work we should be doing. So maybe that kind of works then.
Olga: That sounds like an excellent So that takes us to the end of this podcast. Thank so much Paulina for sharing your wisdom this area. And Christophe, Christophe do you have any final things to share?
Christoph: I mean, we will add the things, the information you mentioned in the show notes, right. And some further readings. and I think I will probably think some more about pizza ontologies now. And ⁓ we have another episode on ontology. So I feel like there's a lot to discuss there. Right. Okay.
Paulina: Thank you guys so much for having me. I've really enjoyed it. I hope I didn't make it too technical, although I feel like it ended up being quite technical.
Christoph: That's what the podcast is there for. Right. All right. Thank you. Bye bye.
Paulina: True. Thank you, bye.
