00:00:14.132 --> 00:00:17.693
Welcome to Ethics in the Age of AI from the CCAB.
00:00:18.173 --> 00:00:18.763
I'm Sam.
00:00:19.553 --> 00:00:20.303
And I'm Alex.
00:00:20.753 --> 00:00:32.503
In today's episode, we're talking about data, specifically the data of clients and students, and what happens when AI enters the picture and that data starts being used in ways that weren't originally agreed to.
00:00:33.203 --> 00:00:35.323
We've got two very different settings today.
00:00:35.743 --> 00:00:39.482
One is a big global accounting firm building its own AI model.
00:00:40.033 --> 00:00:43.814
The other is a university accounting department trying to modernize its teaching.
00:00:44.603 --> 00:00:49.844
Both run into some uncomfortable questions about consent, privacy, and where the ethical lines are.
00:00:50.563 --> 00:00:53.802
And in both cases, there's a real temptation to say, "It's fine.
00:00:53.832 --> 00:00:58.152
We've sort of addressed it." That temptation is exactly what we're here to challenge.
00:00:58.863 --> 00:01:03.192
But before we go on, we need to make the same disclosure we've made all series.
00:01:03.802 --> 00:01:08.192
The two of us are AI voices, and this podcast was created using AI.
00:01:08.992 --> 00:01:12.673
Worth holding in mind for an episode about who owns data and how it gets used.
00:01:35.391 --> 00:01:36.971
Right, let's set the scene.
00:01:37.611 --> 00:01:42.331
You're a partner at a big global firm responsible for innovation and technology.
00:01:43.182 --> 00:01:50.462
The firm is developing its own large language model, its own AI, to enhance audit, tax, and advisory services.
00:01:51.091 --> 00:01:54.141
Genuinely exciting, potentially transformative.
00:01:54.700 --> 00:02:00.641
And to train a large language model effectively, you need large amounts of high-quality, relevant data.
00:02:01.210 --> 00:02:04.581
Which a big global firm has in extraordinary abundance.
00:02:05.301 --> 00:02:09.670
Years of client financial data, transaction records, contracts.
00:02:10.230 --> 00:02:18.192
The technology team comes to you with a proposal: use this client's data to create what they're calling synthetic data.
00:02:18.741 --> 00:02:28.881
Essentially, a statistically equivalent version of the data that doesn't contain actual client information, and use that synthetic data to train the model.
00:02:29.531 --> 00:02:36.580
And on the surface, that sounds like a reasonable way to protect client confidentiality while still benefiting from the richness of real-world data.
00:02:37.341 --> 00:02:39.920
It does, and the technology team is confident.
00:02:40.341 --> 00:02:44.292
Their team leader says, "We're not using actual client data.
00:02:44.521 --> 00:02:46.020
We're using synthetic data.
00:02:46.591 --> 00:02:48.421
Confidentiality is maintained.
00:02:48.901 --> 00:02:49.572
Let's move."
00:02:50.230 --> 00:02:52.752
But there are people in the room who aren't so sure.
00:02:53.451 --> 00:02:54.920
Several concerns surface.
00:02:55.820 --> 00:02:59.871
First, synthetic data is created from actual client data.
00:03:00.771 --> 00:03:14.471
Even if the end product doesn't directly identify clients, the first step of the process, processing the original confidential information, is itself a use of that data, and clients have never consented to that.
00:03:15.241 --> 00:03:17.401
And this isn't a theoretical concern.
00:03:17.911 --> 00:03:24.581
The IESBA Code, the International Ethics Standards for Accountants, is explicit on this point.
00:03:25.300 --> 00:03:31.822
It requires client authorization before confidential information is used in the development of technology, full stop.
00:03:32.846 --> 00:03:35.566
The Financial Reporting Council has also clarified this
00:03:36.225 --> 00:03:44.066
So the technical argument, we're only using synthetic data, doesn't actually hold up against the ethical and regulatory standard.
00:03:44.694 --> 00:03:46.716
The processing step counts.
00:03:47.485 --> 00:03:52.195
What about the risk of the synthetic data still containing traces of actual client information?
00:03:52.965 --> 00:03:55.335
The technology team themselves acknowledge this.
00:03:55.776 --> 00:03:58.336
Synthetic data generation isn't perfect.
00:03:58.944 --> 00:04:03.844
There's a small but real risk that actual data occasionally leaks through.
00:04:04.596 --> 00:04:12.155
And for clients in regulated industries, financial services, healthcare, even that small risk may cross a line.
00:04:12.776 --> 00:04:14.305
And then there's competitive pressure.
00:04:14.944 --> 00:04:16.206
Competitors are doing this.
00:04:16.745 --> 00:04:19.365
The firm risks falling behind if it doesn't move forward.
00:04:20.014 --> 00:04:21.995
Which is a classic objectivity threat.
00:04:22.714 --> 00:04:31.444
When commercial urgency starts to feel more pressing than your professional obligations, that's exactly when you need to slow down and apply the conceptual framework.
00:04:32.055 --> 00:04:33.524
So what should the firm do?
00:04:34.125 --> 00:04:37.764
Start by acknowledging the legal and ethical position clearly.
00:04:38.574 --> 00:04:47.834
Explicit client authorization is required, not assumed, not inferred from vague engagement letter language, actually obtained.
00:04:48.665 --> 00:04:56.175
And that means going to clients and having an honest conversation about what you'd like to do with their data, why, and what safeguards are in place.
00:04:56.834 --> 00:05:01.985
The FRC has actually noted that clients may be quite open to this if it's explained properly.
00:05:03.014 --> 00:05:09.148
The issue is doing it without asking What you need to do is develop a consent framework.
00:05:09.908 --> 00:05:12.367
Give clients the option to opt in or out.
00:05:13.067 --> 00:05:30.476
Begin with a pilot using only data from clients who have genuinely agreed, and explore whether you could create useful synthetic data based on patterns observed through human expertise over time, which, under the IA-SBA provisions, can be done without specific client authorization.
00:05:31.276 --> 00:05:35.267
There's also something to be said here about the integrity signal this sends.
00:05:36.016 --> 00:05:45.026
If a firm is willing to use client data in ways clients haven't agreed to, even for ostensibly good reasons, what does that say about the trust relationship?
00:05:45.817 --> 00:05:46.497
Exactly.
00:05:46.877 --> 00:05:51.708
Clients entrust firms with some of their most sensitive financial information in the world.
00:05:52.237 --> 00:05:54.726
That trust is foundational to the entire profession.
00:05:55.166 --> 00:05:57.927
You don't erode it quietly for a competitive advantage.
00:05:58.617 --> 00:06:08.416
Now let's zoom out from big global firms to a university accounting department because the students who'll be practicing the profession in five years are already navigating these questions.
00:06:09.047 --> 00:06:12.966
Let's imagine that you're a professor of accounting at a well-regarded university.
00:06:13.918 --> 00:06:23.507
The department has licensed a third-party AI platform designed to help students with accounting exercises, give instant feedback on assignments, and check for plagiarism.
00:06:24.326 --> 00:06:26.497
And the rationale is actually quite sound.
00:06:26.997 --> 00:06:29.677
Students are already using AI tools anyway.
00:06:30.307 --> 00:06:35.096
A bespoke, free, vetted tool should at least give the department some oversight of what's happening.
00:06:35.846 --> 00:06:36.706
That's the logic.
00:06:37.396 --> 00:06:39.747
But during the pilot, things get complicated.
00:06:40.377 --> 00:06:46.067
The AI's feedback on complex case study assignments is often generic and occasionally just wrong.
00:06:46.536 --> 00:06:51.437
It can't grasp nuanced arguments or context-specific applications of accounting standards
00:06:52.245 --> 00:06:56.836
Which matters enormously in a subject where the whole point is developing professional judgment
00:06:57.605 --> 00:06:59.026
Then there's the plagiarism checker.
00:06:59.716 --> 00:07:07.586
It starts flagging submissions from international students, students whose sentence structures and writing patterns differ from typical native English conventions.
00:07:08.295 --> 00:07:12.834
These students suddenly find themselves under investigation for plagiarism they didn't commit
00:07:13.536 --> 00:07:19.814
That's a genuinely distressing situation for a student, and it's a textbook example of algorithmic bias.
00:07:20.305 --> 00:07:25.995
The system wasn't trained on diverse enough writing patterns, so it treats linguistic difference as suspicious
00:07:26.586 --> 00:07:38.774
And meanwhile, students are beginning to over-rely on the AI to generate answers rather than developing their own analytical thinking, which is precisely the opposite of what an accounting education is supposed to do
00:07:39.535 --> 00:07:41.074
Then comes the data question.
00:07:41.565 --> 00:07:49.654
The terms of service from the AI vendor are vague, particularly about whether student assignment data is being used to train and improve the AI model
00:07:50.305 --> 00:08:03.795
So potentially, student work, which may contain original analysis, personal reflections, even sensitive contextual information, is being fed back into a commercial AI system without meaningful consent
00:08:04.574 --> 00:08:07.605
Under GDPR, student data is personal data.
00:08:08.115 --> 00:08:16.264
Processing it for a purpose students haven't consented to, like improving a commercial AI, is a compliance problem, not just an ethical one
00:08:16.886 --> 00:08:29.276
And then there's the fundamental question: Is this tool actually preparing students for the ethical demands of the accounting profession, or is it training them to short-circuit the very critical thinking the profession depends on?
00:08:30.168 --> 00:08:31.557
What should the professor do?
00:08:32.258 --> 00:08:36.878
First, evaluate the tool's accuracy rigorously and systematically.
00:08:37.628 --> 00:08:43.388
Compare its feedback against expert human assessment before relying on it for anything consequential.
00:08:44.417 --> 00:08:47.918
Address the bias in the plagiarism checker directly with the vendor.
00:08:48.697 --> 00:09:02.758
And critically, use this tool for formative purposes only, practice and feedback, not for summative assessment, not for final grades, until accuracy and fairness can be properly validated.
00:09:03.466 --> 00:09:07.177
Ensure human oversight of all AI-generated feedback.
00:09:07.647 --> 00:09:13.947
Faculty should always review before a student sees a result, especially for anything that carries stakes for the student.
00:09:14.527 --> 00:09:21.388
Review the data privacy terms with the university's legal and data protection teams, not just on your own.
00:09:22.197 --> 00:09:27.777
Get explicit consent from students if their data is going anywhere beyond the immediate marking function.
00:09:28.356 --> 00:09:35.597
And create a clear, accessible appeals process so students have recourse if the AI makes an unfair call.
00:09:36.326 --> 00:09:39.716
There's also something worth saying here about what we're modeling for students.
00:09:40.317 --> 00:09:50.037
If we teach future accountants that it's acceptable to delegate judgment to an AI and not look too closely at how it works, we've given them a very dangerous professional habit.
00:09:50.787 --> 00:09:59.866
The accounting profession requires an inquiring mind, the exercise of professional judgment, and ethical decision-making with clear accountability.
00:10:00.807 --> 00:10:02.518
Those things can't be automated.
00:10:02.996 --> 00:10:05.986
They have to be taught, practiced, and earned.
00:10:06.706 --> 00:10:10.096
Both of today's scenarios are really about consent and respect.
00:10:10.716 --> 00:10:12.486
Client data doesn't belong to the firm.
00:10:12.817 --> 00:10:15.147
Student data doesn't belong to the university.
00:10:15.746 --> 00:10:18.547
Trust is extended to you for a specific purpose.
00:10:19.057 --> 00:10:27.086
And when AI starts using that data for purposes that go beyond what was agreed, you've crossed a line that professional ethics doesn't let you quietly uncross.
00:10:28.096 --> 00:10:45.355
The hard truth is that it's technically permissible and it's ethically sound are not always the same thing The CCAB codes ask you to apply both the letter and the spirit of professional standards, and sometimes the spirit asks more of you
00:10:46.155 --> 00:10:47.355
That's it for this series.
00:10:47.754 --> 00:10:54.865
We've covered AI in strategic decisions, AI in audit and financial reporting, and AI and data privacy.
00:10:55.436 --> 00:11:00.275
Three episodes, six case studies, 15 very uncomfortable questions
00:11:00.956 --> 00:11:05.566
and hopefully some useful frameworks for thinking through the ones you'll encounter in your own practice
00:11:06.174 --> 00:11:16.044
The full CCAB case studies with all the ethical considerations and recommended courses of action are available at ccab.org.uk.
00:11:16.855 --> 00:11:18.775
We really do recommend reading them in full
00:11:19.566 --> 00:11:23.076
Thank you for listening to Ethics in the Age of AI from the CCAB.
00:11:23.855 --> 00:11:30.105
Until next time, keep asking, not just can we do this, but should we?