speaker-0: welcome to this special conversation for the inaugural IEEE Ethics Week 2026. My name is Dolly Cheetah, chair of the research ethics subcommittee. As artificial intelligence becomes embedded in research, a fundamental question is emerging. If AI is increasingly doing the research, where does human responsibility begin and end? we often hear that the solution is to keep a human in the loop. But is a human truly exercising scientific judgment and accountability or simply approving an increasingly automated process? And could humans in the loop itself become a form of ethics washing? To explore these questions, I'm delighted to welcome Dr. Mark Esposito. Mark is a professor of strategy and technology policy at Northeastern University and a globally recognized scholar, educator, and practitioner. At the forefront of technology, economics, and public policy, specializing in the fourth industrial revolution and AI. He holds appointments at Howard University, Berkman Klein Center, Center for International Development, and Institute of Quantitative Social Science, and is a senior associate at the University of Cambridge and adjunct professor at Georgetown. He serves as a chief economist at Micro One, a Silicon Valley AI lab. Mark Given your work across AI governance, economics, and technology policy and the relationship between humans and increasingly intelligent systems, I can think of very few people better positioned for this conversation. So, Mark, thank you for joining us today. Nice to have this conversation, Dalid. Thank you. So let's start with ethics. I'm an engineer entering this conversation from the perspective of research and technology and engineering. So let's begin very simply. When we speak about ethics in relation to AI, what are we exactly talking about? So so as as a background, I as you know me, right, I come from economics. So I I faced the ethics problem for the first time about twelve, thirteen years ago. I I wrote a book on kind of the moral dilemmas that the financial industry was facing back then. and it was the first time where I was kind of more exposed to the philosophical compounds around ethics. So I'm not talking out of of an ethicist or an ethics scholar, more about an application of ethics and and the line of work that I do, which you rightly kind of label as governance or technology. So look, the I think there are two dimensions that we always want to look at. One is whether we're aligning any form of either internal value or external value, whatever is the kind of the the the degree of conformity you're having with internal external factors. to policy that are to some degree regulating technology. And so you could argue the internal could be an organizational mission or vision or or or core core set up a manifesto on how companies want to achieve a specific kind of goal. You could argue the same could happen in communities. You could argue it happens in society. So it's kind of the organizing principle of what determined a society functioning in general. And that that could be as micro as macro as you want this to be. And so a lot of the ethics, conversation around AI, always goes around the alignment between the the value system and the manifestation of the very same through technology. And we tend to think that is an extension of the saying, because you could argue technology is instrumental. And so if you have a a technology that's simply facilitating Or accentuating a specific process is simply more of the same. You're not necessarily estranging yourself from the kind of the normative ethics sandbox the organization have. but this has changed when we started to navigate technologies that were fundamentally cognitive. And so they were representing thinking rather than physical labor. And so today the conundrum that we are all navigating is whether or not the technology that we're using with some degree of autonomy that is part of where the agency comes into the conversation is reflective or not of the societal societal norms that are basically inferring the the the inputs to this technology to function in society. And so I think this is the most common set of conversation that you have in in the habitat of governance of technology. And I would say this is probably the one that is most functional. The one that is, I think, of a higher order is the moral cole, right? are we as a society to some degree downsizing humans at the expenses of technology? Are we using technology to directly indirectly discriminate? Am I are we creating massive structure of disequalization across social fabrics? And I think this is a different story. Where in a society where thinking, cognitive information, truth, they are becoming difficult to corroborate, then you do have more fundamental questions that are basically rising. And so the moral question sometimes gets equally attached to the ethics question. But we should think of ethics and morality as two very different venue of reflections. They're not necessarily mutually exclusive, they're just different. lenses of analysis. And I think in the context of research, up to a certain point, you could argue is ethical. but when research is designed for the improvement of well-being or the improvement of society at large, it does become moral as well. So there is there is definitely a space where the interchangeability of ethics versus morality cannot necessarily be ignored. I had my next question was about around legal, legally and technically permissible versus ethically permissible, right? So we're talking about the morality of things. So can you comment a little bit about that? are they the same? Well, it depends, right? There are some systems that have used the let's say the societal norms to shape their legal systems. And so you could argue that this particularly true, for example, in countries where you don't have a separation between religion and state. Right. You could argue that country where, let's say, an ex-religion is the official religion and therefore they're not secular. You could argue that the religious foundation, they were translated into the legal system. And so in the specific case, there is a clear space of transferability between one system to the other. In other systems, like for example, in the United States, but mostly in Western Europe or in the Western world, right? you could argue there is a separation between state and and let's say church, right? Or or or spirituality. And therefore, sometimes our legal system have not been based on religious principles, with the exception to the point of about almost 100 years ago, there was an article that Basically, was was a book that came out from a philosopher called Benedetto Croce, who's basically saying, no matter who you are and no matter what you think, it's difficult to decouple you from the Christian footprint of morality that has inspired society. So the Western democracies, directly and indirectly, have been permeating Christian values in their legal system. So you could argue that if you go a little bit deeper into that. You will probably still find some inklings of the transferability that doesn't happen in secular countries, also in secular countries. But let's say for the sake of the arguments that we are able to distill it enough to their separate systems, a lot of our legal systems are based on principles that are basically applied across society, either the principle of minimizing the harm to the others, the principle of of conformity of a social society of a civic society that has to coexist together. Is either whether there are specific behavior that tend to be condemned and and consider inadmissible for a society well-being. So there's a number of different tenets, right, that are are usually integrated in constitution and things of this nature. Now, here is where the catch comes. A lot of our legal systems, they are largely driven by evidence base, in which we tend to see that some form of of evidence that is either processual or precedent-based or or historical has defined the maturity. That's one of the reasons why sometimes in cases where we have no precedent, we do cite when a court has has decided and for what ruling, right? Because you kind of define the progression of the ruler law in relationship with how the law is applied to differing degrees of contingencies. Now When you apply the same to technology that is by design experimental, largely hypothetical and largely, I would say, unscientific, like technology like AI, then our legal systems they struggle to normalize, vehicle to regulate the input of technology in the real world. And so you might have a legal process that is unethical. Like you could have a legal process that is equally immoral. Simply not because there is an intent behind that, but simply because there is a disconnect between the foundational legal system and where they come from and what they use to regulate, most likely conservatively, versus technology they are growing at a pace we have never seen before. And therefore they're shaping the contour of how a landscape of regulatory application could possibly look like. This is why. The narrative on legal sandboxes is a much more carefully designed tool rather than the low as it is. You've written a little bit about actually quite a bit about AI ethics washing. And I want to bring your previous work directly into this conversation. Most people are familiar in engineering about greenwashing. Ethics washing is perhaps less widely understood. So What distinguishes genuine ethical governance from ethics washing? So, you know, the the core idea around humans in the loop, as you were mentioned before, which used to be, by the way, a military jargon, right? then later on was transferable to technology, is that you are delegating the ultimate decision to a person. Now, the epistemic problem with that. Is that what if the person does not have the knowledge to make the determination, but does have the agency? Yeah. It's like if, for example, you're delegating to me a medical decision. I am not a medical doctor, but I will carry the accountability for this. It's very likely that even if I am a human in the loop, that's not the best use of a human in the loop. And now this is a very there's very, I think, extreme case. But you could argue a a system where a person is assigned oversight, but the person does not have the expertise to express judgment. That is ethical washing, because even if you have edified, right, a human in the loop process, which definitely will check or tick the box, you haven't necessarily improved the decision process itself. You haven't necessarily improved the human judgment. Neither you have made a better use of people. I think you have used people almost like a bottleneck that is integrating and concentrating the accountability without having the benefit, right, of the decision based on knowledge. And so this is why in most of the human in the loops narrative, the question about do we have epistemic knowledge about that is more important than just having a human in the loop. Sometimes If you have an entirely automated process, you can say it's limited because there are biases, there are questions about how the algorithm was basically working, But somehow you're not creating the ambiguity of having delegated a poor decision to a person that doesn't have the knowledge of that. So this is with the most common would say application of ethics washing, in which is really about the knowledge itself. but there's another one, Dolly. It's when the oversight. Does not have the perimeter of autonomy that will critically assess a situation in favor of the greater good, but is largely conditioned by vested interests. For example, a financial auditor that is incentivized for certain kinds of decisions, you could argue that even if I am providing agency to this person. The incentive model of the principal agent dilemma will naturally bias decision towards incentive structure. And so you could argue, are we really giving people full autonomy in decision, or are we conditioning their decisions based on what are basically predetermined interests? And I think this happens a lot in a lot of industry. And so I think there's a there's a problem with the contestability of the decision that is not coming from the technical knowledge or the expertise itself. It's coming from the in this case, the moral problem or dilemma on how much are we expressing an opinion based on knowledge and how much we express an opinion based on vested interests. Right. And in the the context is very important because we are talking about the incentivization in research is much more in the form of publications and everything else. So that would be leading us to one of the further questions later on in the conversation. Can you also give us an example where an organization might appear to have responsible AI safeguards while those safeguards actually provide very little meaningful protection? So I think in in terms of industries, right, you could argue that any industry that is using human in the loop, but the industry has a very clear commercial mandate. are you providing human in the loop because you are really helping a decision to be made? Or are we providing human in the loop because you're simply accelerating a decision that is validated by human in the loop infrastructure, but fundamentally it's a commercial decision that is basically happening. So I think that any industry so rather than a specific case on a specific company, right? It's more about detecting and recognizing where this kind of trap or catch twenty-two are basically visible, right? Every time that you're perceiving that a decision is not driven by the authentic need for an assessment to coming from a point of critique, but it's coming from basically an assessment that is reinforcing a specific goal that has to be achieved. And so rather than having this an automated bot in which people could argue it's is estranging from a human interaction, which I think is really the the issues here. You're using people to establish trust But the process itself is not based on assessment, it's based on reinforcing something that's been already predetermined. And so that level of preemptiveness in which you're preempting the process to be assessing the different variables in favor of what is basically the best possible outcome. It doesn't follow best outcome by optimization, but follows outcome by basically interest. So it's more about recognizing where the system are basically being set up so that that they are constantly trying to use the humans in the loop. to establish a trust token that is basically used for a decision that carries on a certain level of, you know, say partial interest or or at least provisional interest. So talking about the decisions themselves, there is an AI generated decision that the human is ultimately having some oversight on, and not being merely present, presence of the human is one thing, but also having meaningful decision making is One other thing. what if the decision fails catastrophically? So who should be accountable in that process? Well, at the moment, right, clearly it's the human in the loop because we we delegate the agency also to delegate the accountability, right? So it's somehow the agent who is basically exercising the very final decision that in theory is in charge because we're assuming that humans should be in charge rather than machines. And It's a fair argument to make, but it goes back to what we were discussing before. What if I am not empowering the person for a number of different conditions, right? Either indigenous or exogenous forces, right? See that the internal or external forces, never to be able to be in a position of expressing exercise in the fair decision, but always exercising an unfair or inequitable decision, eventually in case of disasters or or bad choices, right? You're gonna have this person. And look, if you wanted to see this in in real applications. one of the reasons why we have the technologies for driverless cars, but most of us still drive or being driven, right? It's not because we have a technology limitation, but we have an accountability limitation. Because in the contest of a car, I would need to carry my own limitation myself, right? You could argue, will the insurance insure me? Right. That's the question. in system where, for example, you have in public transportation, with exception of subways or metro systems that that tend to be automated driverless because it it is kind of an engineering design. It's not necessarily intentional in the real traffic, right? Because there is simply is a straight line that follows through the city. So you can plan this with precision, or to some degree trains in some cases. With the exception of that, it's also very, very difficult, right, to really fundamentally assess how do I assign a canability with the exceptional having people in the loop. So I think there's a space where one think is what a technology can do in in terms of the degree of delegation of a decision making to an let's say an automated agent and how much of that accountability is eventually shared. And where are who are the possible stakeholder? Look, you could argue you are the you the operator is the accountable one, fine. You could argue the software company's accountable one. You could argue the hardware company's accountable one. And you remember in Phoenix, there was this case of a of a driverless car that ran over a pedestrian and became illegal conundrum, right? Because it was a Toyota that was operating through Uber. And so the question is: who is responsible? Is it Toyota as a manufacturer? Is it Uber as the Kind of the company who's operating the system. but Uber did not develop the algorithm itself. Uber probably subcontracted somebody to build the algorithm. So the moment that we don't have any clear way of sharing the cascade of accountability across the different stakeholders, and we are only delegating that responsibility to the very final mile, right? Which usually has to be in this case humans. I think we are equally preventing technology to distribute itself. Because we're simply creating, as I mentioned before, bottlenecks in which what we have preserved is the human in the loop as a bottleneck, rather than the human in the loop to be the ultimate decision maker of any of a, let's say, a perfective decision that could come through technology. So are there any cases of human versus AI in the court right now? There must be. There are several. I think you go into chatbot, they have given wrong information, you definitely will have more in in in, I'm sure in hardwares or in warehouses, right? We have a lot of automation, there must be a lot of cases like this. you know, if you follow the news about the robotics in China, very likely there's in I don't know whether they are eventually landing into a a a room of law, right? A quarter law. But clearly there are controversy between the use of technology and the impact that they have I think across this this degree of of uncertainty that we have in in assigning responsibility. and I think you will continue to see this because robotics, also the United States, is now gaining traction fundamentally because it's a combination of increasing the manufacturing to become advanced so that you can produce at a scale that we couldn't do before. it is a a bit of a conversation related to the fact that when you're investing into hardware, you have a safer expectation on possible returns than when you're investing in software because hardware does not fluctuate as much as software. So it is a bit of a of a shortcut for investors to try to curtail some of the losses, right? By having physical assets that basically either appreciate or depreciate. And so you could argue that robotics is on the rise also in the United States. But with robotics in on the rise in the United States, question like the one you asked me will continue to occur. And clearly Until we're gonna have governance or technology shaped around the legal system, we'll continue to have issues of this kind. So I didn't mention it explicitly before, but I'll mention it. We should not build governance out of our current legal systems. Otherwise, we're gonna see almost like this impossible storm, right, to navigate across a system that are not designed to be compatible or interoperable. Okay, thank you for that insight. let's talk about AI becoming the researcher, a hypothetical, basically. if you want to really talk about scientific research, This is very pertinent to the IEEE researchers, engineers, and students. at what point does AI move from being a research tool to making meaningful intellectual contributions and should that contribution be disclosed. Yeah. You know, there's there's a philosopher called Chano Floridi at DA ATL and he has been writing extensively about this. And you know, I think contrary to the mainstream where sometimes the mainstream feels very vocal about one direction or the other. There's nothing necessarily wrong with the fact that we're using AI now to conduct research per se, if the premise of research is still coming from a genuine inquiry standpoint that is observed in the real world by people. But whether you're kind of looking at phenomena that are either in need of a why or in need of a how or what, right? Whether we are asking the question whether two variables can be measured, and therefore we can determine a statistical model out of this. So, whatever is the initiation, if it's coming from a genuine point of inquiry, and there is architectural oversight in which it's literally like an architect that is delegating to workers the construction of a building, people will not remember. The name of the workers, but they will remember the name of the architect who will become famous because of that, right? But the architect will never move a brick. And so the scaffolding, which is the intellectual capacity that is basically expressed in research, should be in the in the analogy, the architect that is delegating to workers, the engineering part of that, right? The the construction part. Now, if we don't do that, so if at one point in time the design itself is generated Through a digital cognitive process. You could argue that it might be factually effective for the purpose of building leading the construction of a building, but it does create a challenge with what is authentic, what is genuine, and what is basically original. And the issue is not that, you know, per se it is a problem, but we're coming from. Legacies or a legacy system where ownership of idea only exclusively pertain to humans. So here is where I think the opportunity comes. We can see this quite it's twofold. You can see this as something that is almost like eviscerating right that sensor normal on what determines the original thinking, novel thinking. And you can say when AI interchangeably Create a process that is generative and it's not coming from an act of ingenuity. Right. That creates a degree of ambiguity that society cannot tolerate because we are unable to navigate that space. And you can argue, sure, it's a fair question that could become ethical. Somebody could argue with the mindset of tomorrow, we might not be able to see the same challenge we have right now as we see now, right? I would imagine remember when there were cases where people were thinking, personal computer, that will never take off. This is like, you know, a fantasy, right? I'm sure there were people never believing that we could have music in in clouds, right? Or in in streaming. So there's always been resistance to technology, right? Because how can we understand something that is happening and largely changing our sensor normal with the mindset of, today or yesterday. Right. And so you could argue that maybe the moment that we are unburdening ourselves from a sense of pertinence around originality, the research can be generated artificially as well. Now, what I don't know, though, is whether the effectiveness of research that is generated by existing data that will become generative towards a research model that eventually conducts its own data, either authentic or synthetic data, whatever. What I don't know is whether in terms of effectiveness, right, that will equally be effective. You could argue that in some specific fields and domains, generative research might become more effective than humans simply because the purpose might have been not driven by how you can publish or whether you're placed in your tenure career or whether you're placed in your editor, but mostly on the specific outcome that has to be stipulated. That could be one possibility. I'm not necessarily excluding the fact that our limitation on appreciation of generative research today are generally coming on philosophical contrast. It's more about am I able to even assess it with a mindset of now, considering that it's fundamentally changed in the way. And look, if you think that this is very, very far fetched, I'm sure that back in the days, the concept of universal care or the concept of everybody should go to school and it should be paid for was equally like a radical idea. It's just that I think we sometimes don't really know whether we are in the presence of something that is fundamentally undermining society and his collagen, or whether we are in the presence of an entropy that we can't necessarily stop and it will eventually move society forward. So I don't know the answer to that. This is why I would make a very bad ethicist, right, in this case. But I would say this is where I think research should equally start contemplating not solely on what is possible with technology. But are we clear on the contribution that we're making through research or are we simply complacent just because we never had anything challenging it in the first place? But there were historic moments. how long was the technology like the transition or adaptation to this new scenario or coming up with frameworks was if you can look back into history? They were much longer because if you think about to number two reflection on this. Number one. Most of our technologies of the past was mechanical technology. And mechanical technology, because it's integrated in the physical world, does not go as fast as things that are fundamentally aerial, right? Like software. And so you could, for example, imagine radio or TV in the house of Americans back in the days. And you could ask yourself the question, how long will it take before every single house in America will have a TV? Right? So look, it'll take, I don't know, 30 years, 40 years. Actually, I know that radio took 38 years to reach half of the houses in America, right? And so I don't know how long it took for the second half, but you can argue this technology was taking time. I'm sure that we could argue that technologies that started to move from coal to oil and gas, right, as a way of powering our economy didn't happen overnight, right? So I think there's a difference in terms of the penetration rate that you're having with technology that tends to be fundamentally mech mechanistic, right? And technology that instead are mimicking thinking. And so I think that velocity of decision making that is currently occurring in every single day of our life, it is fundamentally a shock to our ability for us to understand, aggregate, collect the data, measure it. generate empirical findings and eventually determine based on scientific principles, whether or not certain things are or are not. Right. You could argue that science has been great in understanding what would be better for efficiency, what would be better for idling of engines, what would be better for construction of material and what would last longer, considering the impact, for example, of the external agents. Science has helped us to determine, you know, how do we start having clean water to drink, even in in part of the world where you don't have it. So they're all fundamental physics and and mechanics driven. But we are dealing now with an entirely different environment where I I don't know. I mean, I cannot imagine that we're gonna be able to process it, understand it, abrogate it, redact it, measure it and publish it by the same time that it happens within a normal cycle. And I think this is where We have to start also realizing about the inevitability of what we can't control. because we have never been exposed to this degree of technology distribution that is so fast and which is difficult for us to even, you know, comprehend. Going back to your generative AI based scientific result production, if the results are reproducible, in the research context, but not really explainable. how do you go about it? And also validation comes into play, So could human validation itself become a form of ethics watching if the validation is insufficient for magnitude of decision? It could be. Yeah, it could be. it's a fair concern to have, right? if you're kind of like excluded from the scientific process of research and you're only using the findings as a form of validation of some sort, right? You're kind of watching it for being an attentive research process. But I'm also concerned that the replication might happen maybe more nominal. So that you might take a study that has been done by a generative process, you adapt it and now it becomes your own study. And you could argue that we're fundamentally defeating the principle of scholarship, where scholarship was about, if you think about it, building on other people's discoveries, right? And the entire idea about the literature review is built on adding your own brick to the wall of knowledge, right? And that brick is really how you're running synthesis, analysis, right, and then research methodologies that are eventually trying to either measure or test a specific phenomenon in in reality. So I think it does remove or deprive us from a sense of of Ownership that we had in on the process of knowledge that clearly belonged to humans because we never had a challenger before. so clearly it's I think it's a shock to the system in the way we are able to assess the system. Back to what I was saying before, whether or not that is by itself a moral problem, I don't know. I I think that Only time will be able to help us in understand how our own sense of understanding of these technologies will eventually place us in in a way that is not a technology that has downsized our ability to fulfill our ambitions as humans, but something that instead has you know deprived us from it. So I think this is where you know, if you read and you know, Dolly, you've been reading some of my work, right? I've been I've been advocating for the fact that technology should expand us. It should not downsize us, right? So if the the question was, is generative research a form of expansion? I don't know yet. I don't have a way for me to assess it yet. I think we are too early in the process. I also have been I've been a scholar for 23 years now. I have seen a lot of really imperfect processes in the way we run research, largely driven by the publishability of an article that is not necessarily following a scientific process, but there is a lot of alteration of data to please an editorial line. So you could argue that while we have preserved the original process, we have also adjusted, Tinkered, to some degree corrupted it for specific purposes. So it's not that the fact of generating research by people by itself is absolutely virtuous. Right. There's a lot of flows there as well. Right. So we haven't perfected that process and now we are left with humans versus AI when we were still debating humans versus humans at large. going back to one other thing is accountability problem, should there be any traceability across the, for example, students are using AI assisted research, so our professors or educators. Should there be a traceability across the entire research cycle? And when I say traceability like Some sort of auditory? Okay, I think the mindset of a forensic AI is actually a healthy, a healthy process to have. simply because in you know, if if we do believe
speaker-1: Leave AI being.
speaker-0: in computational statistics, right? Operating at a pace and magnitude that we had never seen before because of the supercomputer that we currently have, with the amount of data that we could never store before because we we didn't have the capacity. So if you're if you assume it to be an extremely complex infrastructure reference, right? Yeah, and then you can just say, okay, well, how do I how do I make sense of all of this? Any degree of forensic approach which allows you to have some degree of explainability does help us to understand the Web of interdependencies that AI is is actually implying. Let me try to be a bit more specific and I'll paraphrase. Traceability will not fundamentally be the solution to the problem we currently have on explainability, but it's a good practice to initiate. It does help a lot that we have an opportunity to talk about authenticity or originality now. It is the right level of conversation to have right now when this degree of ambiguity caught us all off guard. Whether or not it's a permanent solution to the issue we're having about authenticity, originality, ingenuity, right? I don't know. And I don't think so. I I don't think that this kind of incremental transactional tools of of oversight, they're fundamentally improving. some the more fundamental problem that this technology is posing for us. But, you know, put it this way think as a guardrail on the road. It cannot prevent a large crash from getting off the road and maybe off to a precipice. Yeah. But it can help with the minor crashes so that small accidents do not need to escalate into tragedies. It's the very same principle. I find that traceability is a guardrail that helps us to mitigate minor issues. It doesn't help us to mitigate issues of a greater scale. And so will it help us with a conversation about privacy and digital identity? Of course not. It's the very same guys who are producing the technology who will produce not only the traceability but the tools to trace. So are we solving the economic conundrum of an oligopoly or concentration of power that is giving s so much? political power in the hands of few companies. We're not. On the contrary, we'll keep on becoming overly dependent on them. But are we, for example, preventing an unfair treatment of students where a lazy student can get by versus actually you know, motivated students. Sure, we can now give to a teacher the tool to assess it differently. And that tool can provide us with opportunity for educational engagement, retraining, tutoring, mentoring. So I think it's really the scale that Santain gets gets out of proportion. So I see this as guardrail preventing small accidents rather than guard will preventing big accidents. Yeah, it brings me to other conversation a separate conversation from ethics, looking at people who don't have accessible technology, in different countries. the gap between the haves and the have nots technology-wise is going to be further widened if a student who is not intellectually Capable has these tools to excel versus not. And then there is ethics on top of that. going back to your conversation on guardrails I want to focus a little bit on governance and standards, are there any existing AI governance frameworks that adequately address this? Autonomous scientific research or AI reviewing AI or AI generating data that cannot be explained, any of the frameworks that you can talk about? So look, there are, I think there are initiatives, right, Dolly, that are happening across, right? So the UN has one-level frameworks that are basically designed to try to integrate humans in the loop. So if let's say that at any level, If you are preserving the constant, in this case, it will be human in the loop, right? So you wanna have a human in the loop that is checking research publication or a scientific one. You wanna have a human in the loop that is assessing the quality of the data, you wanna have a human in the loop that is able to navigate whether organic or synthetic data, they constitute a good source of replicability or not, right? In our scientific experiment, right? You want to have humans in the loop in assessing students' performances, you you name it, right? So let's say the humans in the loop is kind of the constant. So the UN has some level frameworks that are are currently being pivoted, right? The OECD, the entity of the largest 29 GDPs in the world, they they do have a framework as well. The UNESCO has stipulated, and UNESCO has been quite vocal about this. And then you can say that a foundation and not for profits, World Economic Forum, also like survey companies, to some degree, consulting companies. They're all coming up, right? We way to define it. And equally at the government level, you will have governments, the South Korean government, the UA government, right? they all have stipulated forms of integrating a way of assessing the level of intervention that humans have in, for example, research. So you could argue, did you use it for brainstorming? Did you use it for literature review? Did you use it for editing? Did you use it for research method, did you use it for statistical inference, right? Did you use it for polishing or parsing data? So you could argue that the use determines the the legend or the rubric, right? Basically where you're reflecting on what has been the degree of dependency you had with technology and you can assess it in a way that is becoming fair. So you do have some, I would say, examples where this has gone from kind of the lip service to the application in in real world. And you could argue that, you know, as much as we have seen, for example, well, back in the US when we used to have DEI, right, that eventually got phased out, but at least it it transformed itself a lot. Or ESG, right? Where we were asking organizations to perform a certain level of assessment on their environmental, social, and governance metrics. You could argue that compliance Will play a meaningful role in defining these humans in the loop. Again, for me, these are temporary short-term solutions. They're trying to make some form of progression towards a problem that requires engagement, but it's not going to be addressed by any of this more cosmetic, I would say, tools. So I think they requires a deeper level of foundational underpinnings, right? but I think, this is where we want to eventually continue to engage. By having organization, no matter whether it's intergovernmental, governmental, municipal, regional, whatever, corporate level, engaging and back to your original question, in defining what is the ethics reflection between what I value for and how do I reflect it into a process that is now largely agentic and where I want to preserve humans in the loop. And I think if we understand this process, then this grassroots of initiatives that are going to initiate and provide people with the opportunity to reflect on this, that will eventually create I think a a movement of sensitivity about the fact that we do care about humans in the loop. We care to make that a fair role that is not necessarily becoming back to your kind of one of the premises of your work in this conversation in the ethics question. we want that to be a form of augmentation of humans to the point that we are in the business of improving human judgment. We're not in the business of depriving people from human judgment, but still give them a form of assumption that they have. I think if you if you navigate it this way, then the greatest key, the greatest scheme of things, right? The greatest transformation we want to achieve might be achieved because you are created a spillover effect. across a number of different initiatives that are understanding the engagement required. So this is where I think there's been a lot. I think in the last year or so, the number of initiative that came across are are today, I think is is becoming almost like you know, difficult to distill them. But there has been a clear response to the action that is required for this to become to some degree much more participatory. So we are moving towards the end of the discussion. I want to ask you for our community, what three questions should every researcher ask before using AI in their work and also the role of funding agencies that are probably also using AI to review these grant proposals? Yeah. So look, I think so imagine a scientific community that is to some degree And ties and excited about the fact that now we have access to data, right? Because it's easier to get data. I think the question, the first question to ask is how reliable is this data for the purpose that I have? And and whether it's synthetic or not synthetic, does the data fits the purpose, right? Of of providing a scientific advancement to a question I want to bring forward. I think that the engagement and the interrogation on the data is a starting point that should never be discounted. I think the second one is what we discussed throughout the conversation, right? Now that I have oversight, am I looking at this oversight as a way of expanding, amplifying my ability to contribute? Or is the oversight actually a prison in which I am confined to a number of different variables that are controlled basically agentically and predetermined before, where I'm only an executor. I'm executing basically what has been predetermined. But I don't carry any real critical judgment in the assessment of what is ahead of me. And so I think that would be the worst case scenario where we have used technology to make humans even least or less useful, right? In that case, because we have made them fundamentally just executor of a decision they have never partaken in. And we'll see that would be so sad. And I think the question is always about, and I I say this always to my PhD students, right? It's always about the defensibility of the ideas. And and let me try to be specific on that. Ten years from now, we will have technology able to trace, assess, reverse engineer a lot of what we do right now. Ten years from now, many of the people doing research might be at the peak of their career. It would be terrible if now they discovered that during the process of inquiry, they have abandoned and surrendered their own sense of research to technology that was doing on behalf. And so they have to some degree dismembered themselves from the research that carried their names. So, in light of technology that will be made possible today to discover much more, what was the process of inquiry that was applied in this specific paper? How defensible is that paper for me or that research? And so the more people realize that defensibility is coming from preserving the degree of agency, the more I think they will be able to delegate what needs to be delegated and keep what does not have to be delegated. And I think this is our, I think there's a mechanism of defense and equally of offense at the same time. In the sense you're you're engaging the lack of knowledge by simply engaging because it's important that we do. We are also making sure that there's always the fensibility around anything that you're producing, to the point that no one can ever say that the process that you use to produce the specific findings does not have your voice on it. and then the question we're saying before, right? The data part and and whether, you know, we are we are expanding or we are downsizing. I I have a lot more questions, but I know our time is limited today and follow up questions too. So hopefully some of the community members who are going to reach out to us will be passing on those questions to you, Mark. So do you have any final thoughts? I know we just scratched the surface of this topic. I think Jolly, it's important that you know governance, governance in general used to be part of you know a specific group of people, right? So you could argue that many legal scholars were studying governance. today in organizations governance is fundamentally carried forward by compliance officers. I would imagine governance has to be transferred. much more seamlessly across the entire organization, organizational design. So anyone should be able to impose a governance reflection of some sort. It doesn't have to come from HR, right? Or from the office of institutional effectiveness or whatever they're gonna call it, right? It should be a much more distributed sense about understanding this question. So hopefully when governance is becoming decentralized, it is becoming much more effective. when governance is centralized and it basically try to get distributed through the peripheries. So I hope that's probably where I love this to inspire more people. and that's for another time. All right. Well thank you Mark. the one idea that stands out from your conversation is that having the human in the loop is not necessarily the same thing as having meaningful human responsibility and accountability. And as AI becomes increasingly embedded throughout research, it's no longer enough to ask, is there a human in the loop? Perhaps we have to ask who is accountable for the loop? And human oversight must be meaningful and not merely present. So thank you so much, Mart, for joining us and thanks to everyone for joining this conversation as part of the inaugural IEEE Ethics Week.
speaker-1: Leave AI being.
speaker-0: in computational statistics, right? Operating at a pace and magnitude that we had never seen before because of the supercomputer that we currently have, with the amount of data that we could never store before because we we didn't have the capacity. So if you're if you assume it to be an extremely complex infrastructure reference, right? Yeah, and then you can just say, okay, well, how do I how do I make sense of all of this? Any degree of forensic approach which allows you to have some degree of explainability does help us to understand the Web of interdependencies that AI is is actually implying. Let me try to be a bit more specific and I'll paraphrase. Traceability will not fundamentally be the solution to the problem we currently have on explainability, but it's a good practice to initiate. It does help a lot that we have an opportunity to talk about authenticity or originality now. It is the right level of conversation to have right now when this degree of ambiguity caught us all off guard. Whether or not it's a permanent solution to the issue we're having about authenticity, originality, ingenuity, right? I don't know. And I don't think so. I I don't think that this kind of incremental transactional tools of of oversight, they're fundamentally improving. some the more fundamental problem that this technology is posing for us. But, you know, put it this way think as a guardrail on the road. It cannot prevent a large crash from getting off the road and maybe off to a precipice. Yeah. But it can help with the minor crashes so that small accidents do not need to escalate into tragedies. It's the very same principle. I find that traceability is a guardrail that helps us to mitigate minor issues. It doesn't help us to mitigate issues of a greater scale. And so will it help us with a conversation about privacy and digital identity? Of course not. It's the very same guys who are producing the technology who will produce not only the traceability but the tools to trace. So are we solving the economic conundrum of an oligopoly or concentration of power that is giving s so much? political power in the hands of few companies. We're not. On the contrary, we'll keep on becoming overly dependent on them. But are we, for example, preventing an unfair treatment of students where a lazy student can get by versus actually you know, motivated students. Sure, we can now give to a teacher the tool to assess it differently. And that tool can provide us with opportunity for educational engagement, retraining, tutoring, mentoring. So I think it's really the scale that Santain gets gets out of proportion. So I see this as guardrail preventing small accidents rather than guard will preventing big accidents. Yeah, it brings me to other conversation a separate conversation from ethics, looking at people who don't have accessible technology, in different countries. the gap between the haves and the have nots technology-wise is going to be further widened if a student who is not intellectually Capable has these tools to excel versus not. And then there is ethics on top of that. going back to your conversation on guardrails I want to focus a little bit on governance and standards, are there any existing AI governance frameworks that adequately address this? Autonomous scientific research or AI reviewing AI or AI generating data that cannot be explained, any of the frameworks that you can talk about? So look, there are, I think there are initiatives, right, Dolly, that are happening across, right? So the UN has one-level frameworks that are basically designed to try to integrate humans in the loop. So if let's say that at any level, If you are preserving the constant, in this case, it will be human in the loop, right? So you wanna have a human in the loop that is checking research publication or a scientific one. You wanna have a human in the loop that is assessing the quality of the data, you wanna have a human in the loop that is able to navigate whether organic or synthetic data, they constitute a good source of replicability or not, right? In our scientific experiment, right? You want to have humans in the loop in assessing students' performances, you you name it, right? So let's say the humans in the loop is kind of the constant. So the UN has some level frameworks that are are currently being pivoted, right? The OECD, the entity of the largest 29 GDPs in the world, they they do have a framework as well. The UNESCO has stipulated, and UNESCO has been quite vocal about this. And then you can say that a foundation and not for profits, World Economic Forum, also like survey companies, to some degree, consulting companies. They're all coming up, right? We way to define it. And equally at the government level, you will have governments, the South Korean government, the UA government, right? they all have stipulated forms of integrating a way of assessing the level of intervention that humans have in, for example, research. So you could argue, did you use it for brainstorming? Did you use it for literature review? Did you use it for editing? Did you use it for research method, did you use it for statistical inference, right? Did you use it for polishing or parsing data? So you could argue that the use determines the the legend or the rubric, right? Basically where you're reflecting on what has been the degree of dependency you had with technology and you can assess it in a way that is becoming fair. So you do have some, I would say, examples where this has gone from kind of the lip service to the application in in real world. And you could argue that, you know, as much as we have seen, for example, well, back in the US when we used to have DEI, right, that eventually got phased out, but at least it it transformed itself a lot. Or ESG, right? Where we were asking organizations to perform a certain level of assessment on their environmental, social, and governance metrics. You could argue that compliance Will play a meaningful role in defining these humans in the loop. Again, for me, these are temporary short-term solutions. They're trying to make some form of progression towards a problem that requires engagement, but it's not going to be addressed by any of this more cosmetic, I would say, tools. So I think they requires a deeper level of foundational underpinnings, right? but I think, this is where we want to eventually continue to engage. By having organization, no matter whether it's intergovernmental, governmental, municipal, regional, whatever, corporate level, engaging and back to your original question, in defining what is the ethics reflection between what I value for and how do I reflect it into a process that is now largely agentic and where I want to preserve humans in the loop. And I think if we understand this process, then this grassroots of initiatives that are going to initiate and provide people with the opportunity to reflect on this, that will eventually create I think a a movement of sensitivity about the fact that we do care about humans in the loop. We care to make that a fair role that is not necessarily becoming back to your kind of one of the premises of your work in this conversation in the ethics question. we want that to be a form of augmentation of humans to the point that we are in the business of improving human judgment. We're not in the business of depriving people from human judgment, but still give them a form of assumption that they have. I think if you if you navigate it this way, then the greatest key, the greatest scheme of things, right? The greatest transformation we want to achieve might be achieved because you are created a spillover effect. across a number of different initiatives that are understanding the engagement required. So this is where I think there's been a lot. I think in the last year or so, the number of initiative that came across are are today, I think is is becoming almost like you know, difficult to distill them. But there has been a clear response to the action that is required for this to become to some degree much more participatory. So we are moving towards the end of the discussion. I want to ask you for our community, what three questions should every researcher ask before using AI in their work and also the role of funding agencies that are probably also using AI to review these grant proposals? Yeah. So look, I think so imagine a scientific community that is to some degree And ties and excited about the fact that now we have access to data, right? Because it's easier to get data. I think the question, the first question to ask is how reliable is this data for the purpose that I have? And and whether it's synthetic or not synthetic, does the data fits the purpose, right? Of of providing a scientific advancement to a question I want to bring forward. I think that the engagement and the interrogation on the data is a starting point that should never be discounted. I think the second one is what we discussed throughout the conversation, right? Now that I have oversight, am I looking at this oversight as a way of expanding, amplifying my ability to contribute? Or is the oversight actually a prison in which I am confined to a number of different variables that are controlled basically agentically and predetermined before, where I'm only an executor. I'm executing basically what has been predetermined. But I don't carry any real critical judgment in the assessment of what is ahead of me. And so I think that would be the worst case scenario where we have used technology to make humans even least or less useful, right? In that case, because we have made them fundamentally just executor of a decision they have never partaken in. And we'll see that would be so sad. And I think the question is always about, and I I say this always to my PhD students, right? It's always about the defensibility of the ideas. And and let me try to be specific on that. Ten years from now, we will have technology able to trace, assess, reverse engineer a lot of what we do right now. Ten years from now, many of the people doing research might be at the peak of their career. It would be terrible if now they discovered that during the process of inquiry, they have abandoned and surrendered their own sense of research to technology that was doing on behalf. And so they have to some degree dismembered themselves from the research that carried their names. So, in light of technology that will be made possible today to discover much more, what was the process of inquiry that was applied in this specific paper? How defensible is that paper for me or that research? And so the more people realize that defensibility is coming from preserving the degree of agency, the more I think they will be able to delegate what needs to be delegated and keep what does not have to be delegated. And I think this is our, I think there's a mechanism of defense and equally of offense at the same time. In the sense you're you're engaging the lack of knowledge by simply engaging because it's important that we do. We are also making sure that there's always the fensibility around anything that you're producing, to the point that no one can ever say that the process that you use to produce the specific findings does not have your voice on it. and then the question we're saying before, right? The data part and and whether, you know, we are we are expanding or we are downsizing. I I have a lot more questions, but I know our time is limited today and follow up questions too. So hopefully some of the community members who are going to reach out to us will be passing on those questions to you, Mark. So do you have any final thoughts? I know we just scratched the surface of this topic. I think Jolly, it's important that you know governance, governance in general used to be part of you know a specific group of people, right? So you could argue that many legal scholars were studying governance. today in organizations governance is fundamentally carried forward by compliance officers. I would imagine governance has to be transferred. much more seamlessly across the entire organization, organizational design. So anyone should be able to impose a governance reflection of some sort. It doesn't have to come from HR, right? Or from the office of institutional effectiveness or whatever they're gonna call it, right? It should be a much more distributed sense about understanding this question. So hopefully when governance is becoming decentralized, it is becoming much more effective. when governance is centralized and it basically try to get distributed through the peripheries. So I hope that's probably where I love this to inspire more people. and that's for another time. All right. Well thank you Mark. the one idea that stands out from your conversation is that having the human in the loop is not necessarily the same thing as having meaningful human responsibility and accountability. And as AI becomes increasingly embedded throughout research, it's no longer enough to ask, is there a human in the loop? Perhaps we have to ask who is accountable for the loop? And human oversight must be meaningful and not merely present. So thank you so much, Mart, for joining us and thanks to everyone for joining this conversation as part of the inaugural IEEE Ethics Week.
