PegaWorld 2026: Ethical AI in Action: Building Trust and Transparency in Innovation

I'm so delighted that you're here because it means you care about ethics. Thank you very much. Thank you for coming to our panel. We're very excited to have these subject matter experts. I'm going to introduce them by name, and then they will introduce themselves and tell you a little bit about what they do every day. So first we have Rajesh Pitchai from Wells Fargo. Rajesh, what are you working on? Yeah. Thank you Tara. Thanks everyone for joining us today. Um, I am Head of Customer Decisioning, um, lead the decisioning implementation team for Wells Fargo. Um, so we have been at this journey for like five years now. We have expanded into different channels. Um, we use AI in the Decisioning Lead space. Marketing offers and decisioning for customers. You'll get to hear from my colleagues tomorrow. We have a keynote session as well tomorrow. But predominantly I'm proud to say we are one of the, if not one of the biggest by scale decisioning implementation.

Uh, like we run 6 billion transactions in a month for 70 million customers. Um, so I'm happy and proud to be part of that wonderful team. Thank you. Tara, thank you for coming. Next we have Pawan Verma from Bupa. Thank you Tara. I'm Pawan Verma. I'm head of MarTech and delivery at Bupa. Those who don't know about Bupa Bupa is one of the largest health insurance organization in Australia. But we're also quite diversified and we are quite big on to our health services side such as Bupa optical Bupa, dental Bupa, medical centers, etc. our big goal is to transition from a health insurance organization to a health company. That's where the whole organization is going towards. We have been on to Decisioning Journey for almost five six years now. The whole idea is to put customer into the center. I don't know how many of you love your health insurance, but that's, uh, that's where we are.

We are not as big as Wells Fargo, but it's, uh, we are not a bank and we are not America. Right? So, but we are also quite 4.5 million member base, uh, across our, our, uh, our different lines of services and that we are trying to tailor our recommendations at the back of customer decisioning engine. So very excited. Great results so far and on to this continuous decision journey. Awesome. welcome. And then we have Eti-osa Agbonlahor. Thank you. And she is from Decision Alpha. I'm the CEO of Decision Alpha. We are a behavioral economics firm. We partner with financial institutions to build the experiences, the tools, the services that drive our financial well-being. And so with more of our partners kind of leaning into the guided AI experiences, we're partnering with them around, how do you do it in a way that truly delivers customer value while delivering commercial value and making sure we're doing it in a way that's still risk savvy?

That's kind of where we sit. Amazing. We're so excited. To have you. That's really loud. Um, awesome. Great group. I'm going to kick us off by just talking a little bit about the responsible AI framework that Pega stands by. So there are four areas that we really think should be embedded in all of your strategies. The first is fair and unbiased balance for all groups, right? This helps protect your brand. This is about keeping biases out of your data sets so that you know, hey, this is why my decisioning made the decision. And it's not based on a field, for example, a data field that may have the propensity to carry bias within it. Right? So we're talking about things like age, gender, region, location, that kind of thing. The next is transparent. So this is you guys are going to hear a lot about this, especially with Agentic AI.

Every decision has to be explainable to a human audience. Um, we have bankers in the room. We have health care providers. We have all kinds of industries here. And some of these industries touch very important pieces of people's lives and are highly, highly regulated. So making sure you can understand every single decision down to the source. Next is empathetic. So what does that mean? Right. Um, AI can't feel things. But what we want to use AI for is, you know, interacting with your customer in a way that is both of value to the business and the customer, right? So this is the area where I say, just because you can do something doesn't mean that you should do it right. We want to make sure that our customers are solvent so that they can stay customers with us and that our brand reputation is intact.

And then the last is robust. And so that means hardened to real world exposure and doesn't have unplanned behavior. Um, the example that I always use is you guys remember during the pandemic, early breakout of the pandemic, Amazon was not expecting a pandemic, right? So when you went to order online, anything that you would normally get served was not popping up. There was the algorithm was not working because it had never been tested with this type of behavior before? So it's testing with all kinds of scenarios that maybe you're not expecting, right? Okay, let's get right into it. So Pawan Bupa uses Pega CDH to power millions of interactions and conversations. What's the line before between a helpful nudge and an unwelcome intrusion? Because that's a really, really thin line. And how does AI help you stay on the right side of that line? Yeah.

Sitara. It starts with the trust. The trust is the main important factor, right? If you look at a healthcare industry, it's different from, say, retail, e-commerce or say, your streaming industry. If Netflix gives me a wrong movie recommendation, like it's mildly annoying. But if as a healthcare organization, I start sending some wrong communication to our customer. It can cause confusion like the customer can get overwhelmed. Uh, or the trust starts getting eroded, right? I was using this analogy with someone today morning. Uh, gaining trust is like climbing the stairs. You climb one step at a time, right? It's hard. It's slow by design. But losing trust is like taking that elevator to the ground. Ground floor. You directly come to the lobby, right? So I'll perhaps repeat what you just said for us. The trick is we can personalize a lot of things, but should we personalize those things?

Like this is the line that we generally try to drive. You also talked about empathy, right? Empathy is not some magical algorithm. It's not like another empathy model that we kind of implement. It's a series of actions like controls and review processes and governance mechanism that we have put, um, which kind of ensure that it's customer interest that are into the heart. So yeah, that's how we kind of translate our, uh, um, our, this whole use cases at scale and AI helps us to do that. So this is where the unwanted nudges versus some of the helpful interactions they come into picture. I'm going to start using that analogy all the time now. Um, yeah, we know, right, that customers have one bad experience with trusting your brand and they're gone. They're gone. Your competitor. Just takes one bad interaction. One bad for all the good, right?

Uh, so speaking of the line that we just talked about, the goal of most brands that are using AI to scale personalization, What point does that cross from harmful to manipulative and who gets to draw that line? So in the past, we've talked about, you know, making financial decisions. If we start relying on AI for financial literacy, when does that cross the line into manipulation and who is responsible? I think that the the line is intention, right? Intent is the line. We wrote a paper on our responsible AI, and we named five different areas to keep an eye out around user agency. But running through all of that is this notion of what is the intention. If the intention is to help customers, then sure, if the intent is to want to extract more commercial value from the customer, then we really should be. Then we're probably in manipulation territory, right?

So the line is what is the intent here? Now, in terms of who's responsible, I think there's a there's a shared responsibility, right. It's incumbent upon me as the user to do what we call calibrated trust, meaning I interact with AI as a thinking partner, a sparring partner. I'm asking it to stress test things. I'm asking it to show me its work. I'm asking it. I'm saying, hey, my context has changed in XYZ ways. So I'm constantly looking at the AI not as a single source of truth, but as this is going to help me make better decisions. It's also then incumbent upon the actual institutions building the AI to build it into the UX right now. Right now, we see the big companies say AI can make mistakes, but it's in gray text, super tiny at the bottom of the screen. Yeah, right. We want to see that being built more meaningfully in the actual UX so that people using AI can understand these are the guardrails.

This is where I can trust it. This is where I need to, you know, stress test some more. And then of course, there is also a regulatory level, right of drawing that line, which is we see Europe right now has come out with its AI act. And transparency is also one of the things they're talking about in that act. So I think there's a shared responsibility as to who draws the line. But intent is what matters at the end of the day. Amazing. Rajesh, you're in a very highly regulated industry. So and the regulators are watching. Financial Services how does this scrutiny shape how wells adopts. AI. Yeah. Um, so as everyone would know, Wells Fargo is one of the big four banks in the US, right? So we have all the eyes on regulatory authorities and everyone. So even within internally, right, there is a lot of regulatory and compliance needs which are much needed for us.

But at the same time, like what we have done in the last five years, scaled decisioning and AI for Wells Fargo, it's been tremendous. Right? See, we don't we didn't see it as a speed bump. Right. Like, like we saw it more like a guardrail for us to ensure we are successful. Right? It's almost like going in a highway. There is guardrails, speed limits, but we're still going in a certain path to success. Right? So regulations has been that for us, right? There was initial periods where we had to earn their trust. But as time went by, like it just became trust is given to us, right? Um, touchwood nothing major happens to a landscape. It's very sensitive. Um, but regulations has been that um um guardrails for us to success. That's how I see it. Um, and there is tremendous leadership commitment around all of this as well, right?

There's like, it's all a nice harmony and a partnership within the landscape. We don't fight with each other that much. We understand each other's point of view. And that success has, uh, you'll hear more about that success tomorrow in the keynote. Yeah. Awesome. So it's more of like a roadmap of where you should say exactly. Love it. Follow up question. So if a model makes a consequential decision, Um, who's accountable if something goes wrong? Because, you know, we all get our credit checked. Um, maybe fraud is flagged when you're like, in a moment where it's not, you know. Yeah. Yeah. I. I don't think AI has reached that stage where it's going to sit in front of a regulator and explain itself, right? Like that's, that's not happening in the near future. So it's always we as owners of what's happening, right? Uh, me, my boss and, and the leadership beyond, uh, we are all responsible.

We are sitting there with the regulators and compliance answering questions. Right? It's almost like, um, you have your kid did something naughty in school. They're not responsible. It's the parent. Right? So fair enough. Fair enough. It's almost like that. Right. Um, we have been in situations where we had to explain. So the key is um building explainability, transparency right from the beginning. Like we did not invent this on the way. After a few errors like explainability and transparency, where one of the key objectives right from the start of our AI initiatives. Right. And and so that's how it goes. It's always humans in the loop. We are responsible. And we have again, point of we have some wonderful leaders inside the organization. They don't like pass the buck around. Like we all take responsibility when it matters. Yeah, that's how it goes. That really reinforces at Pega we say that who's accountable?

It's actually the whole organization, even if you're not every day touching those things. So we all have to be good stewards. So Pawan, what does a human in the loop actually look like in principle at at Bupa and what do you let the AI do autonomously? Yeah. So human in the loop is, is not a slogan, right? Like it's a, it's a design principle. And I agree with what Rajesh just said in the discussion that just happened. It's always the humans who are accountable for the decisions, right? AI can help us synthesize information, summarize things, make some recommendations. But ultimately, the accountability always lies with the human or with certain leadership, right? In practice, what it means, it means having the right control tower in place like a governance council. Your, um, uh, controls that you've put into place, your reviews, your testing, your monitoring, uh, your checks and balances time to time.

Uh, this for us at Bupa, we believe that trust is earned every day. It is not granted. Right. So that's the principle. Now, to answer your question about the autonomous AI, it depends upon the stakes. right? Like if it's say, content and we have to do some sort of content testing or content. AB testing. Yeah. That's fine. Like we can have that quite autonomously because the stakes of two. Of that particular tasks are very low. And if the content gets strong because there's a spelling mistake or something like that, right. Like that's an easy fix. And they are controls generally in place in order to avoid those gaps. But as customer stakes increase, we definitely would like humans involved, especially when it comes to their cover reviews. When it comes to their benefits, what they're eligible for, what they're not eligible for. So fairly important.

We have this running joke, um, that is a health organization. We have a very healthy respect for caution. And I am sitting into a tech conference and generally we say that we move fast and break things. We don't want to break things, right? We we would. The industry and the space in which we are operating is fairly sensitive. People don't want us to break things. Right. So I understand sometimes it may it may come as. Are we too cautious? Uh, no, we are not too cautious. We just want to do the right thing. So this is how I wrap this, uh, this answer, that AI. Help us quite a lot. It's the way to go. We are kind of doubling down onto this, but it depends upon the tasks, the stakes, uh, the intention. And it's ultimately the humans who were always responsible for the end outcomes.

Fair enough. Rajesh, what about you? At Wells, what is allowed autonomously? And what does human in the loop look like? Um, we are big on human in the loop, right? Like, um. Right from the first step of designing a model. Right. What goes into the model? Um, the predictors, every single piece of data element goes through scrutiny, right? So we have an in-house data science team who looks at everything. They take it to the compliance reviews they vetted out, discuss it through even before adding all of those predictors into the models we use. Right? So it starts from there, right? Um, then we roll out the models. Um, and there is a team who is like doing a dipstick check into the models in production on a daily basis, almost on a daily basis to it's like a quality check of how the decisioning engine is, is performing for us.

Right? Um, so there is, there's humans that loop in that process. Then there is, um, we have a process where when we roll out changes, new, new actions, new new conversations into, into production, right? We run simulations, um, in a safe environment and see how the distribution is aligning. Is there something off and all of that that's still monitoring the quality of decisions we make still humans in the loop, right? Um, it's right along, like even decisions to retire down certain models and even decisions where the business wants to override the models. Like, um, our team is involved and we give the right recommendations and whatnot. Right. So, um, so it's pretty much human in the loop all through the start to end, right? Coming to autonomous AI, I think we have a journey there to go through. Um, we are still exploring there, right?

I'm sure we'll get there at some point of time, but it's still exploratory in nature for us. Needless to say, we are a bank. Yes, highly regulated and any sort of autonomy is against a red flag for compliance and regulation. So there's quite a bit to go there. But I'm sure we'll get there. Yeah, yeah. Fair enough. Um, so you deal with all kinds of brands in the marketplace. Is responsible AI a real differentiator and organizations looking at it as that, or is it kind of just like a check in the box? We have to do this? Yeah. I think at this stage, we're still seeing a lot of organizations focusing on the, the technical accuracy, meaning can we do this right? How do we build this in a way that's going to deliver value to our customers, deliver value to the organization? Can we build it properly to start with?

Right. And so I think a lot of people are still in that kind of neck of the conversation and not quite yet. Okay. Now that we can do it really well, how do we make sure that we're doing it in a way that's ethical, that won't damage our customers? That will. It won't just be that we're taking the legal risk compliance box, but we're really being able to differentiate ourselves with it. So if you look at it similar to like GDPR, we're still at that stage where GDPR was rolling out and people were going, what is this? How do we make sure we do it? Okay? We're not yet using it and saying, hey, privacy is our differentiator and we really care about it. And this is how we keep you safe. So we're not yet at a stage where it's, you know, we're seeing organizations use this as a differentiator, but we absolutely will get there very quickly.

Follow up question. So there will be harm done to consumers over time, obviously. And who is responsible for measuring what that harm is. Right. Because these things tend to be sort of a nebulous, you know, will we start measuring it like we measure business outcomes, for example, and who is responsible to measure that, do you think? Yeah, I look at it kind of similar to the pharmaceutical industry. So if you think about pharmaceuticals, we've got the regulator that's the FDA here in the US who is responsible for setting the standard of drug testing and how you roll medicines out into the market? Next layer down, we've got the actual drug companies that say, this is how we test, this is the standards that we set. And then one layer down, you've got the attending physicians who are responsible for looking at the patient and going for you specifically, I understand in your history understanding the other medicines you're taking.

Understanding the long and short term effects. Feel comfortable prescribing this for you. And so I think there's kind of still a shared responsibility, especially if you look at it from the financial services perspective. We've got the regulators being able to sit and kind of say, this is how you think about harm, this is how you need to measure it. This is what it looks like, but it's never going to be as detailed and as powerful as the institutions themselves. To understand the customers who understand how customers are using their products, to understand what is the intent of our customers using the products and what does capability look like? What does financial wellbeing look like for other customers? So I think to some extent, the institutions be a little bit more responsibility in terms of harm measurement, just based off of the quality of data they have access to.

But everyone still has a bit of shared responsibility. So the institutions, the people who make the AI and our regulators as well. Fair enough. All right. Um, okay. We talk a lot about when things go wrong. We just saw this on stage, right. Here's here are the missteps we've had. But we don't often talk about the things that go right. Can you give some examples of when you're ethical and responsible AI practices have strengthened a consumer relationship? Yeah. Great question. Right. Like credit where credit is due. We talk a lot about those muffins in the morning. Right. Like and and that's real AI is not perfect, but also AI is making a lot of progress. So I'll pick two themes where AI is helping us in our journey. The first theme is relevance, right? If you have like for any organization, if you have a million member base or more than more than that, it's humanly impossible to kind of send relevant messages to them at the time when they need it.

And AI is a great enabler towards that. Like I'll give some of the examples. For example, as a health insurance organization, we have members who are entitled to many benefits. And what if we can just use AI in order to tell them that, hey, you have not used some of these entitlements that you are eligible for and, uh, or like you are onto this particular cover, um, policy cover, but maybe you may have to downgrade your policy, and it may look like contradictory, that it could be a short term revenue loss, but the kind of lifetime loyalty and the kind of customer trust that you gain at the back of it, it's like amazing. So we are quite big into that. We use CDH AI quite a lot onto the relevancy thing. We have use cases like credit cover fit, we have what the customer entitlements are, which is quite helpful.

The other thing that I will pick is the restraint, right? When not to speak these as marketeers, we have this whole big tendency that we need to constantly we are into this dopamine mode of campaigns, like more campaigns, more reach, more emails, more sense, but depends upon what kind of problems you are solving for. Our customers generally don't want to listen from us, right? Like from a health insurer. But when they need us, then they want to over listen from us, right? Because they want to know all the steps, what they are covered for, what they're not covered for. Uh, what's the process if there's a pre-admission process, what it looks like, what's the post admission process look like? Right. So we need, we cannot just have a measure them with the whole, um, same lens. We need to know when we have to be silent.

We have to be restrained. And when we kind of go and do some sort of upsell, cross-sell loyalty, those kind of communication. So I'm just going to reiterate what you're saying about silence, because I talked to a lot of marketers out in the world, and silence is an undervalued offer, right? You know, we overserve so much. We're so used to overserving that I think you just really hit the nail on the head right there. Um, Rajesh, I want to ask you the same question. Yeah. Um, I'll I'll follow on what Pavan said. Right. Being a banker and a marketer is a deadly combination. Like you want to land a credit card in everyone's hands? Like, as much as possible, right? There is a natural tendency for us to do that. Right? But, um, we understood customers nature and like how they interact and how they like to interact with the bank.

Right? So we have a healthy mix of, um, acquisition type of offers and like servicing type of offers, right? So it becomes a natural way of interacting with our channels than like, oh, this is a marketing channel coming at us, right? So, and our engine is optimized and tuned to ensure that the right balance of these messages gets out to the customers. Right. So that's, that's, that's one thing. And second thing, what we have seen is like, we, we measure our own impact, right? So we do have a control group kind of a setup in our implementation, which doesn't go through the decisioning engine, right. And then there is the rest of the customers who go through the decision engine, right? So when we see comparison between the control group and the customers who go through the decision, we have seen some huge lifts, right.

That tells me that my decisioning engine, my CDH implementation, is giving out more relevant messages and offers to the customers. Right. And and there is a direct correlation to the value we recognize as, as business. So, um, so definitely measuring, measuring that lift is so critical, uh, which helps you optimize, right. And then we slice and dice the lift into, um, into multiple portions. Like, um, my, my boss reaches out suddenly one morning and says, hey, this section of the customers is not getting these certain offers like that. That triggers a sequence of events within my team to investigate what's happening. So, so measurement is a big part of what we do. Um, like both for us to continue operating this program and prove that there is the success is real, right for us. So these two points I wanted to highlight. Yes. Awesome. Okay, you guys, this is the last question.

And then we're going to take questions from the audience. So I'm going to start with Atossa and I'm going to get all of your opinions. And then I'm going to give my opinion too, because I have an opinion on this. Um, in five years, how will we know if the industries that we're in got responsible AI. Right. And what is it going to look like? Mhm. I think the intent of AI is to make our lives better, right? Ultimately, that's the point of why we're doing all of this. And we measure the things that matter. So I think in five years, if we get this right, the metrics we're talking about should change and should be mature, right? As we're reporting NPS and Rs and AUM. We're also talking about financial well-being scores and harm reduction scores, right? We should see a maturity in the metrics that we're talking about around responsible AI.

And it should be almost intuitive. It should be like, yeah, of course we're talking about financial well-being. It shouldn't look like that one special thing that this bank did with this one time, it should feel, of course, it's Bau to talk about these metrics when we discuss AI and its impact on our customers. So I think that's what it will look like from a won't be sexy, but we'll see the metrics change. Yeah. Boring. Boring. Um, so in five years time, you should not have this panel. That's my wish. Great answer. Right? Uh, AI is same as responsible AI. They are not two different things, right? And today, what we are discussing and this is this whole talk, right? Um, we think about innovation, we think about speed, and then we think about responsibility and ethics. And they are generally thought as two sides of the coin.

They're not right. They're on the same side. So for the sustenance, for the long term success of AI, it needs to be responsible. It needs to be ethical. That's the only way to go about this. So that's my think AI and responsible AI and ethical AI are all one and same. And we will not be kind of discussing about, hey, how do we kind of build responsible AI? It's not an afterthought. It's kind of becomes, for the lack of better word, I'll just use Bau. Right? Boring as you kind of designing your processes, you're just thinking about what's before you even design a technology model. What's the governance of this model look like? What's the responsibility? Things that going to look like in this from a customer perspective, I would say, um, our customers will understand. And if the customers understand that we are using AI and they respect that and they are comfortable with that and they trust the organization, I think that's the success.

It's not like AI is going anywhere. Our customers will also understand it, but we need to cross that chasm of that trust, and they need to be comfortable into that journey. Mhm. Mhm. All right, Rajesh, you're up. Yeah. Um, I'll give a two part answer to this. Like first is a customer's perspective. Our end customers, like, as we speak, um, my customers are experiencing my AI every second right now as we speak, right. Five years down the lane, Like if they don't feel that there is an AI sitting behind their interactions right then I feel that we have succeeded in responsible AI, right? Like the messages I put out there should be so relevant. It should feel so natural. And they should develop love and affection towards this brand than thinking, ha, there is an AI sitting behind and doing this thing. Like if they don't feel that way, then I think we have succeeded in AI in five years, right?

So then internally, um, if I don't get anything from my regulatory and compliance people, I've succeeded, like, and everything is explainable. Everything is is audited. I can pick any decision and say, how was the decision made in a, in a very easy fashion as an internal stakeholder? Um, that will be success for me, right? So both perspectives. I love it, I love it. For me, it's also a two part answer. So to echo, um, you know, if I don't ever want consumers to be surprised in the same way they were surprised around what happened with Cambridge Analytica and GDPR. Mhm. I want them to have literacy so that they know that most of us are are trying to do the right thing at all times. Right. And then secondly, this is kind of a pipe dream, but I would love if we could all get on the same page with a framework for responsible AI that is widely shared and adopted.

Um, so we're all kind of moving in the same direction, moving with the same wave. So all right, with that, are there any questions out there in the audience? There's microphones right here on either side of the room. So. Hello? Yes, right behind you. Yep. Okay. Um, so yeah, James Smith from Natwest in the UK, uh, bank. Uh, I was just curious, um, a lot of the conversation has been around the ethics of, um, applying AI to do the right thing for customers, etc., but kind of the second half of the title of the session around kind of the ethical innovation of AI. I was just curious maybe what the thinking you've been doing around how do you innovate ethically with something where I think more and more through time, you'll be thinking about environmental impacts, kind of operating impacts of like the succession of your people from kind of new entrants that need to learn to highly specialized people, I think, feel like there's a broader spectrum of ethical, um, factors when you're thinking about innovation of AI, not just the application of it.

I think that it starts with, and I think that's a great question, right? But it also starts with understanding how do we want our customers to grow? What does innovation look like in what is the best way to serve our customer? What does financial well-being look like for our customer base as they mature with us, which then starts to pinpoint whether the innovations they're going to need, right? I expect that I want to see more of my customers or a few of my customers spend more time in early customer arrears, in debt, in collections. Maybe we need innovations there. What does that look like? What kind of interventions do we need to start plugging in to make sure we see fewer customers in debt? So I think it actually take a step back and look at the customer lifecycle and understand with this future, we're rapidly headed towards what are the new customer needs at each section and how are we serving them today?

What is best to look like? What is what are their future needs going to look like? I think that's where the innovation is going to come from. And because you're doing it from a customer perspective, you almost have that ethics built into it. You almost have that sense of, we're starting by wanting to do right by customers within. We start with the right within the box. But because that's our foundation, it helps us start to think through building these innovations, but in an ethical manner. So that's where I would start with. Awesome. Hey everybody. Thanks for doing this. Um, there was mentioned that we can have confidence that we're doing the right thing based on propensity, because we're acting in the customer's interest, which is baked into that. A lot of those interactions today are with very clear communication collateral, like a placement on this or an email or something, a thing where the customer knows they're being spoken to in this context and they can accept it or reject it.

And that's baked in the propensity as, um, more convincing and natural LLM chat and even like voice chat with an LLM becomes more of the norm. How do you see that evolving? Like the sense of responsibility that we all need to take? Maybe I'll start. And then then you guys build. From my perspective, a propensity is a great. Point to look for the thresholds and the reliability. But we actually even proceed and we put it into the rule sets like the eligibility applicability and suitability rules. Right. And if it comes to your propensity model, then the hypothesis is that this is actually can be given to you into this particular situation because it has bypassed. Is it are you eligible for this? Is it applicable for you. Is it suitable for you? Right. So even if we give it, even though it could not be a high propensity score, it's not a bad thing.

But there could be other actions with a much higher propensity score which are better suitable. And that's what we should be giving to you. So if I extend the same analogy even to other channels. Like that's what, as I understand, your question is to voice and to, to chat and others. I think the same logic applies if we, especially in our case, we, we are generally more cautious. And if we have more cautious, then we try to be a little bit cautious. On to the rule side, rather than arbitrate onto the propensity model, because they could be a chance that there could be bias, there could be few other things where things can go wrong. That's how I would take it. Yeah. By the way, he was my boss. He found a way to ask me a question today as well. So. So to, to answer this question, um my perspective is yes, propensity models are just scratching the surface right now.

Like what we do as propensity models is kind of easy from an AI paradigm perspective. But when we go into unstructured voice and chat transcripts and unstructured way of getting inputs into the models. I strongly feel there should be a framework on top which which consumes all of these inputs and translates them into meaningful predictors, which you feed into your other known models, right? I feel feeding these directly into these kind of propensity based models would be detrimental. Um, and I feel there should be a framework on top of it, um, generating predictors for these kind of models. I'll give you a better answer in office. Fair enough. Hi, my name is Laura McGinn. I'm the vice president of customer experience with CSC. And we provide business services to global companies, many of whom are highly regulated banks. Some of you guys are our clients.

One of the challenges that we've been having is our customers concern about having their data in the cloud. If you think about it, some of the services we provide require some of your executives ID that we need to have on file, right? So we need to be able to protect that information without moving to the cloud. We can't process the way we need to to be able to leverage AI. So just curious on your perspective of how you've if you've faced those kinds of objections from your own customers, especially during the RFP process, how do you overcome those? And what are some of the things that you've done to protect yourself? Um, I can go back because, um, we are one of the biggest cloud implementations of CDH today. Right? Um, again, it comes to regulatory and compliance kind of thing, right? The way we see it is if it is, um, if it is good enough to be used for marketing purposes.

I think it's good. It's good to be on the cloud as well. Right? So then there is a classification between sensitive information and what you can widely use for marketing. Right. So we have classified that in at least in our implementation. We have gone through our internal approvals and whatnot to classify our cloud into secure enough to even hold sensitive information. But in general, um, the data we use in our implementations is good for the cloud environment. Yeah, like 100% kind of aligned with that. The only other thing which I will add is, um, about this whole cloud infrastructure, right? Like there's this whole private cloud, public cloud, all of those things. Do you want to, uh, maybe they are nuanced into your, into your question if, uh, what's the discomfort of the customers? Right? Like if they are uncomfortable with the data is going out of the country.

And I think most of the countries have these regulations that the data centers should be within their own country, right? Like in Australia, we were lacking initially, but now most of our data centers for all the big SaaS platforms are generally into the country. And then for some of the other kind of data sets, we have also established some private data cloud. So we thankfully do not face any problems because I have no idea if I start facing those problems, how do I deal with that? Thank you. Thank you. Thank you. James Kilcoyne Wells Fargo. Is there a room for a maker checker pattern? Thinking about your framework for ethical AI and something you'd like to see kind of rolled out? When you think about what we've learned about unit testing and what we saw today in the keynote, the developer does, the engineer does the work, or the user does the work, and unit tests are automatically generated to verify that the objectives of the code are being met.

Would it be possible to think about some kind of AI framework where when you're building models, you're automatically considering what are the negative outcomes and automatically spinning up cases to look for those, or potentially some kind of analytics approach to look at the outcomes of what's happened, sort of regardless of intention. How did it play out in real life? Is that something that is useful to think about? I can attempt. So from my perspective, one of the big things for responsible AI or decisioning in general is what we should not talk to our customers. And most of the time we are very clear about that, right? Like they are codified as rules like eligibility rules, applicability, stability. Again, I go back to this example. Now, if I extend the same analogy to your question about unit testing framework, and most of the time we know as part of unit testing framework, what's the happy path, right?

Like, and most of our devs and tests, they kind of go about that, but we also technically know what's not the happy path, right? As part of the rules, right? Like applicability, suitability, eligibility that anyone who is a minor should not be getting a product. So if we are doing some unit testing, we are kind of testing it for like some of the people, they should be a negative test case that should automatically be generated because AI or Pega knows about our whole rules catalog, and they should start saying, okay, as part of this, if I start speaking to the decisioning language, um, this particular business issue, these are some of the rules that should never be applicable. I think they should be some negative rules that should be codified. So yes, it's 100% possible. Should be on the roadmap. And I'm sure somebody is already working on it.

Yeah. Tara from your team. Yeah. So my, my perspective on that is, um, AI testing for me needs to go further, right? It's not a typical software testing where we know the outcomes and we can mark it as a pass or fail, right? AI testing for me is you almost need another AI which is sitting on top and testing this, right? So I don't think we have made enough strides in that area. Um, it's like, how can I call what is right or wrong in an AI, right? Like to define a test case, right. So I think it needs a lot of thought in that space. I. I will say on a granular level, so we work with a fintech and part of what they do is, um, they help college students understand their finances better. And so one of the ways they extrapolated from our responsible AI framework was simply to have experts compare the answers AI gave with what the expert would give.

Right. Very good. So fintech. So they're always thinking, how do we do this as quickly as possible? And like, you know, move forward. But they've got a bunch of experts and they compared, all right, for this standard questions that measure financial literacy, that measure X, Y, Z, what did I say? What did we say? Right. So it's a way of starting very small. And then you can build up and become more robust in that kind of testing as well. That's interesting. Awesome. Well we're about. Oh. One more. Yeah. Come on up. Hey. Wes Hatton. Leidos. Um, earlier you had an example about how the AI industry could be compared to the pharmaceutical industry, where you have the government with the regulation and then the institution with like guidelines. And then below that, you have like the pharmacists who have like essentially their principles. And then but, um, my question was just that with pharmacists specifically, I feel like you have a lot of, um, actual like certifications and regulations and that's already established around that field.

Whereas with AI, um, we don't really have as strong of a guideline for like how that should look. And also there's a big push in the AI industry to like, make sure that everyone can be using it. Um, even people who might not necessarily have the experience with it yet. Um, so my question was just how much responsibility do you think the institution has to teach like ethical conduct essentially to their employees? Yeah. So, um, yeah. That's a great clarification. So in my analogy, the financial institution is the pharmacist and they bear a lot of responsibility to making sure it's done well. It's rolled out correctly. We've got the measures and the risks that we're taking and kind of looking at the right metrics consistently. So it's less really talk to everybody and like go forth and conquer and a lot more because we are the ones who, if anything goes wrong, they're going to come back to us first and say, hey, what went wrong?

We bear more responsibility and we have to make sure those standardizations and that maturity is in place. I would say to, um, you know, we have that set of employees who are power users and some maybe who are on the, you know, newer side. It's really important to have a defined company policy for just using the standard AI tools that you use every day. So helping people understand and guiding them, having it written out and accessible to people. Uh, you'd be surprised how many organizations don't have that. And it's very simple, easy thing to contribute to responsible AI practices. Thank you all so much. This was great. Thank you. I appreciate you. Thank you guys.