PegaWorld 2025: Maximizing Business Benefits with AI: How Banco BHD & Citi Drove Adoption of AI

PegaWorld 2025: Maximizing Business Benefits of AI – How Banco BHD and Citi Drove Adoption

Um, thank you all for coming to this session. I think we'll get started. I know it takes time, apparently, to get from where lunch was, but, um, I don't want us to run out of time. So, firstly, thank you to all of you for coming to this session. Um, I think we've got a really interesting topic to talk about today.

Um, we're going to have an opportunity to hear how some of our most innovative clients have changed their organizations to be truly customer centric and to take advantage of AI. We're going to cover the business drivers, um, that drive this change, the changes that they had to put in place to be really successful, and how they won over internal stakeholders to make sure that they're aligned behind customer centricity.

So we're going to start with introductions. I'm Mark Davis. I run the 1 to 1 customer engagement um business excellence team. And that's a team focused on helping our clients to maximize the value that they get from the Customer Decision Hub solution. And in that role, I've been really fortunate to work with, um, many clients around the world, across all different industries, um, getting to see and support some really fantastic innovation.

And interestingly, before I joined Pega, I was once a client of Pega, where I was working at a telecoms company in the UK, and I was responsible for both the technology and organizational transformation required to put in place a 1 to 1, um, customer engagement solution about 15 years ago. So quite a long time ago.

So now I'd like to have my panelists introduce themselves. Um, Julio, maybe we could start with you. Maybe you could describe your company, your role, and any benefits you've seen with 1 to 1 customer engagement? Yes. My name is Julio Rosario. I come from Dominican Republic. I've been working on food bank for the last 20 years.

I've been working in marketing, business analytics, and, uh, I've been working with Pega from day one on the implementation. Uh, I was the product owner for CDH. And since, uh. 2020, I've been working with, uh, with Pega. Fantastic. And, Jay, maybe you could introduce yourself as well. Uh, sure. This is Jay Gao.

I work for Citibank. I've been there for over 20 years. Had a series of different roles. Uh, most recently, actually, I am not in digital marketing. I am in analytics. Uh, and as part of my job, we have Pega CDH as a omnichannel decision engine to drive and prioritize messages across channels for all customer facing recommendations and recommendations.

Uh, Citibank has hopefully you are aware that it's one of the largest global banks and, uh, me, myself, we are part of the US operations. Brilliant. And last but not least, Joe. Yeah. So, um, Joe Alan, I'm a go to market director for 1 to 1 customer engagement at Pega, working closely with Mark as well.

But before I joined Pega, I was a client of Pega going back to 2004. So I spent 15 years on the client side. So got a great amount of experience, particularly in the kind of next best action and retention optimization space of using CDH. Brilliant. Thank you. Um, so just actually, before we get into our questions, if you do have questions, we're going to have a few minutes at the end, um, to allow you to ask them. So please just just wait until then. So now we understand your backgrounds. I'd like to start by exploring what sparked your AI journey. What were the specific business challenges or opportunities that made you consider adopting AI powered 1 to 1 customer engagement? Was there a particular pain point? Was there a competitive pressure that served as a catalyst? Um, and it'd be great if you could highlight some results that you've seen as you go through through that.

And maybe we could start with you and talk about PhD. Yes. Uh, in case of the, uh, the decision was in terms of the need that the bank had of, uh, technology aligned to the consumer centric strategy of the bank. Uh, we needed a flexible technology. We needed to integrate some channels in order to have, uh, consistency and to have a relevant message for the customer.

Each time we contact them that that was the main reason to, Up to to take this path. Okay. And, Jay, how about at Citi? Sure. Our vision is to have an omnichannel decision engine, but our starting point was in digital, and we have a good reason. We have tens of millions of customers. And over the years, they have migrated their behavior to be having a lot more interactions with us in digital, which is a real time, highly personalized from an expectation perspective.

And also we cannot really necessarily control the demand because we don't dictate when they come to our websites or our mobile apps. So we need a piece of technology that's at the center of that to determine at any given second, if somebody shows up, what do we say to them? And at the end of the day, to have great business value as well as a very pleasant and seamless customer experience.

And that's the reason we considered the omnichannel channel decision engine, and we use Pega CDH, which has the dynamic AI models, which is the core capability of that. And we honestly couldn't run these kind of interaction points without that. Fantastic. Thank you. And Joe, what have you seen? Yeah.

So if I talk about my most recent time as a client, we had a burning platform, which was is always a good incentive to change things. But more importantly, you know, a bit like these guys, we wanted to think about the customer experience and join the up across the channels. And in fact, that was one of the first times when there was a Pega implementation to join up those channels to make sure that experience was consistent using one single tool.

I think that's really important to to kind of help join up that experience, particularly in a market that I was in that was very undifferentiated. So that was our kind of way of differentiating ourselves in the customer experience. Okay. Thank you. And one of the challenges that many organizations face, many of our clients face is determining where they should start their AI implementation.

I'm curious to understand how you prioritized your use cases, what criteria you use to work out where you should start, and maybe how you balance quick wins against long term strategic value in your initial deployment. And Jay, maybe we could start with you this time. Sure. So our choices were actually pretty easy because we have had a, if you will, a predecessor to CDH that we've been using, which is the Chordiant solution that Pega also acquired, that we've been using to drive our digital messaging for a number of years.

Um, it's at the end of life for that solution. So we essentially upgraded to a Pega CDH. And the interesting thing that we did, and we confirmed to the business as we were migrating. We have set it up. That part of the traffic, digital traffic will go to the old legacy capability, and part of it will go to the new Pega CDH setup. And we allow the CDH enough time and volume to train up and the models to mature. And then at that point we compare what is the incremental lift that we get from CDH. And the results was very encouraging. Let me put it this way. That's fantastic. And Julio, what about ADHD? How did you pick your starting point? Yes.

At the time we were implementing several pega's modules, so we were expecting an impact on people. So we decided to to divide the process in in different steps. So we started with branches. We developed the models for personal customers and we started with the inbound direction. So after that we started working with small business and uh, after that, uh, we included, uh, the album and the rest of the channels, such as email, customer service, uh, call center, um, the web channels.

So we did it step by step because it was very important to see the impact first and then continue with the next step. Right. And I think what's interesting, and I think in previous conversations as well, Joe, you've mentioned this. One of the key things for you was starting with a channel that had a high volume of interactions.

Um, and you've also talked about how it's inbound conversations that you wanted to focus on. And that's important because in those situations where we have that, um, inbound interaction, we also get an immediate response from the customer, right, which helps us train our models really efficiently. Joe, what have you seen? Yeah.

Totally agree. So my experience has generally been with subscription model services. So um, again that inbound interaction is really important, but I've often started in the call center for that very reason. Because you get so much traffic, it allows you to learn really quickly and evolve, so that when you set up your next channel, you're doing so with with great insight.

So particularly when you move from an inbound channel to more of an outbound channel, use the inbound to form the inform. The outbound is really key. So you've taken those responses. Train the models and that gives you that kind of leg up to have a running start with outbound. Okay cool. So now I'd love to dig into how you quantify the true business impact of your AI investments.

Could you share your approach to measuring both tangible and intangible benefits? Have these measurement frameworks evolved as your AI implementation has matured? And maybe again, Julio, we could start with you talking about FDD. Yes. Uh, when we started, uh, we we we thought of, uh, measuring first the customer acceptance and, uh, we, we thought of also to, to establish a control group so we can test the impact and see if we had an uplift and determine the impact we would have with Pega and also with that, take other decisions in terms of the next steps we were going to to do in the implementation, because I told you before that we did it in in several steps.

Yeah. No, absolutely. And I think one of the key points you, you raised there that I think is absolutely critical is putting in place test and control so you can prove attribution. You need to prove the value that you're generating against the metrics that you're focused on. Jay, what about a Citi. So we were focusing on generating value for the business.

We also want to make sure we have a great customer experience. And Pega as an engine that captures customers reaction in real time. Almost as well as we have capabilities of tracking customers activities on our website and mobile apps that we're almost hearing and monitoring their activities instantaneously as well.

So what we do is we, on one hand, track the incremental benefits of the AI driven models from CDH, the lift that we're getting, and we apply that to all the programs that were actions that we're enabling using Pega. So that's one stream, which is hopefully a bit easier to understand from a business perspective.

On the other hand, we have a very robust tracking mechanism to understand from customer experience perspective for any given action on any specific treatment. Are we generating any customer reaction? Because we have very easy ways for us to complain for the customers, and also at the same time that we track every time you hit something that is driven by Pega.

did you get an error message across the way we track that percentage on a daily basis? Anytime, anything that spikes, then whoever owns that action needs to do something about it. So I think we have used eight the engine and the intelligence to accomplish both, which is the business value and customer experience.

And I think that's really interesting for a lot of our customers. Right. We have a number of customers that primarily focus on recommending actions that drive a direct monetary outcome, like selling a product or selling a service. But it's also really important to think about other kinds of conversations you can have with your clients, right? With your customers, there could be educational or nurturing or some other kind of service message that doesn't have a direct monetary value, but clearly has an impact on your relationship with them.

And Joe, what have you seen? Well, I've just really pleased to hear when I hear clients who are actually measuring the impact, because it's really important. It's often the one thing that people forget. Um, I certainly always, you know, when I've been a client, have left it a little bit to the end and you kind of think, oh, we'll worry about that later.

And that's, that's a problem. Right. So it's really important to, to measure and it's really important to measure, you know, apples with apples rather than something measuring apples against bananas. That's you know, this is something, you know, CDH is something that really helps you drive value within your business.

But if you aren't measuring it, it's it's easy to question. So I would urge everybody to make sure you get that measurement in early. Unless you're a supermarket, in which case apples versus bananas might. Be. Might be a good measurement. Right. Okay. So, um, securing investment for transformative technology often requires demonstrating a clear ROI.

We've talked a bit about the kinds of things you you measured. What specific value metrics resonated most with your leadership team? How did you address concerns about investment risk and implementation complexity? And maybe we could start with Citi again. J. So from a value perspective, given our migration journey.

That was pretty, pretty clearly demonstrated. I think the the piece that we kept have to go back to our leadership and convince them of is the quality what comes out of the engine. Is that something that is, first of all, not negatively impacting customers? And b is that something that we want the engine to recommend? It is recommending understanding the dynamic AI models are not always fully transparent, but we have a good way to showcase that.

Hey, there is relevance given the results, given how customers interact. So that is the piece that we track the most often, and then we go back to the leadership and make sure that they stay on board with us. Totally. And I think, again, like you mentioned earlier, because you had a solution, you had historic performance that you could base your business case on, right? Which is is not always the case for a lot of our customers.

And in those cases, we can we can provide information from comparable industries, comparable companies, to help build those business cases. Um, and what was the experience at BHP? Was it similar? Well, it was a little bit different because at the time we were implementing several modules. Pega modules.

We were implementing channels such as, uh, sales automation and also customer service. And we were also implementing CLM. So in this case, uh, it was easy to include CDH because we were looking for, uh, these capabilities where we can, uh, get value, uh, on the implementation side, uh, and, uh, the process was complex because we, we had a we had to do an evaluation of every provider, and, uh, Pega Pega CDH was one of the key factors for the decision for the bank.

Uh. Uh, what's up buddy? Thank you. So moving on a bit to the organizational transformation piece. When you, um, or successful adoption of AI often requires rethinking some of your organizational structures and some of your, your processes. And I have a two part question on this. First, how did you align your teams around AI capabilities as they existed at the start of your journey? And then secondly, given how quickly the AI landscape is changing, we saw Rob's presentation this morning about agentic and GenAI, and we know things are changing rapidly.

How do you line yourselves up to be agile enough to really adapt and take advantage of those kinds of changes? And maybe we could start with PhD this time. Yes. So every time you start working with AI, you will have impact on people. That's why we decided to implement in in several steps in order to see the impact, to take all the decisions and to be aware of what is happening with people.

We also had to think about how we were organized. We decided to put together some multifunctional teams to work on on the channels and also to work on CDH, and we have been working that way since then. We also implemented last year a governance model to get everybody together online with what we want to do.

Other departments such as risk, business, transformation, technology, the product departments are get together every every two weeks and we take decisions according to the goals we are pursuing. I think it's really important to get all of those different parts of the business to collaborate effectively, right.

To take advantage of this. What was the experience like at Citi? Sure. The question triggered a lot of thoughts. First of all, so as we heard this morning that there is this left brain AI, which are the statistical AI or predictive AI that I like to call them. They have been around or being used by Citi for more than ten years.

So we have governance compliance framework to tackle those. And in the past year or two, JNI based solutions are coming up. And quite frankly, when we first had a first use case for JNI, we couldn't even figure out a way to get them compliance approval. Now we are over that hurdle. Now come back to how you sort of evolve the organizations, get them aligned and realigned.

In my mind, I always want to start with what is your KPI? What is the business driver that you want to impact? Because people will have different opinions. What is the right setup? What is the right even governance model? And at the end of the day, how you align and get agreements from others is is this option going to give you a better results on the KPI that you want to influence or worse? And if you have a quantifiable KPI that everybody agrees to and trackable, then the conversations become a lot easier.

And then in terms of organizational setup, one thing I want to call out is, at least from our perspective, it's not once and done. So once when we move to CDH, we had an operating model. Then actually pretty quickly we realized it was not working. So we had to rethink how operationally we connect human beings into the machine.

And we realigned. We created a whole new set of procedure manual, and people get retrained and all of those. And now it is stabilized. It's working as we intend. But I would venture to say then as we have new technology being onboarded, we'll have to revisit those operating models and organizational structure constantly.

Otherwise, it will not align as nicely as we want them to be. No, that's really key points, right? That I love the idea of having a North Star that the business is focused on, and making sure that everything you do supports that, that North Star. And I also really like the idea of having a continual focus on innovation, agility and improving what you do to take advantage of those new capabilities.

That's also incredibly important. So I implementations can yield some unexpected insights or benefits, right? Could you share something that genuinely surprised you about your journey with Customer Decision Hub? Was there an unexpected benefit or challenge that wasn't part of your original strategy or business case? And maybe, Julio, we could start with you this time.

Yes. Uh, I guess what caught my attention the most was how easy it was to implement. It didn't require prior knowledge or prior experience, and we launched our first product to production in 12 weeks. We developed actions, we put models in place, and we started integrating integrating the channel. And we could do it.

We did it in 12 weeks. So my expectations before implementing were that it was going to take a long time, but it didn't. So that was the the. That's a great, great. Great. Bit of insight to hear there. And Jay, what about a Citi. Any surprises from. Well, as you were saying that I instead of thinking about surprises, I was thinking about a very nice comment because we always struggle with, uh, sort of internally that why do you need a decision engine and why it is important to only show relevant messages to people.

And actually, one of your Pega colleagues said something which I found to be very profound and convincing that if you keep showing irrelevant messages in that spot, you're literally training customers to ignore you. And it's such a powerful message that registered with me. I think I'm going to just use this moment to share.

That's a great point, I think. And it's very true, right? If you keep pushing the wrong thing to people, they just tune you out. Right. Joe, what about. What have you seen? What surprised you? Interesting fun. Fun. One to talk about. Because, you know, things often change when you're running a program, implementing a tool like CDH.

And I, when I was a master client, our CEO at the time actually went to the city or the investors and basically announced he was going to implement a new loyalty program on a certain date without talking to anybody about how we might implement this. And that date was less than three months away. I and through the agility of light CDH, we were able to stand something up really quickly to to drive rewards through, you know, an intelligent reward. So we were investing appropriately to, to individuals with the kind of loyalty program. But the funny thing, and the surprising thing was because we had started to give customers things that they were interested in, they actually started to call us and ask us to make sure that we had their data correct, so that their name was correct and their email address, because we were suddenly giving them something that they actually wanted to hear about.

So kind of like the opposite of the unsubscribe problem that most marketers have. You're selling stuff that they want, so they're asking you to talk to them more frequently. Yeah. That's amazing. Um, one of the other things that's often challenging when you implement something like a CDH solution or another AI powered solution, is managing that balance between AI and human judgment.

And Rob referred to this a couple of times this morning in his keynote. Could you maybe share a little bit around how you worked out which decisions could be fully automated versus those that need to be augmented and still retain a lot of human control. And maybe we could start with ADHD on that one.

Yes. We decided from the beginning to to go step by step. We divided the project in in four steps because we thought we would have an impact in people. So we prefer to go step by step and then do a larger rollout. It's the same. It's important with AI because you're going to have a big impact on people.

You're going to change the way they are working. So my recommendation is. Go slow, try it and then decide how do you want to to make a large scale rollout. So again, it's about taking some steps, learning from them, understanding how you optimize like Jay talked about earlier with the operating model and then leveraging that to improve on the next the next.

Build confidence and. Yeah, and. Keep on. Going is important right. What about a Citi. Jay do you have a similar experience? Yeah. So given our experiences with the AI models in the past, organizationally we were pretty clear in terms of what is the principle meaning anything that is hard and fast. As a rule, we have a lot of risk management compliance policies.

So not everyone is eligible for all actions. So anytime there's a hard and fast rule to exclude or include, that's being reflected as the eligibility set of rules. And then anything that is not hard and fast. Meaning if I show the message here versus there, that's fine. Legal compliance they're fine with either or both.

Then we put it to I. We put it to the engine to let the decision engine fully utilize its capability to maximize the outcome. Now, it's not like this concept has never been challenged by internal people because models are not very transparent. So at those times we would entertain a B testing. So humans, please come to the table.

You set up a test, then you compare against the machine outcome. And based on my limited experiences, machines always run. Well. It's a little bit worrying. That reminds me of the Terminator picture on Rob's keynote this morning. Then that scares. Me a little nervous. Okay. Um, so transforming an organization's approach to customer engagement requires effective change management.

We've touched on this a little bit. Um, but it'd be good to hear a little bit more about some of the specific tactics that you might have taken to overcome some of those objections to AI and using a data driven decisioning approach. Um, we've already touched on it a bit. Jay, when he talks about comparing AI versus people.

Um, but were there any other specific tactics you use to help drive that cultural change within your organization? Yeah. So, uh, I'm part of the analytical organization, which is data driven. We understand models to a large degree, but we also fully appreciate and understand there are people in, for example, legal organization, even in marketing, that the concept of a model, especially the concept of self-learning model, sometimes it's hard to understand.

So what we tend to do is to add some context to the outcome, even though the outcome is model driven. Meaning we will call out here are the major factors that's causing your program to be prioritized higher or lower. And then give them a sense it's not the full picture. Meaning there are. were hundreds, thousands of inputs going into the models.

But we'll highlight some of those. And we realized that by doing so, it's just giving people who are not, if you will, in the quant space a little bit more context. And then they feel a little bit more how do I say trusting the machine? Because you give them something that they can grasp and they can understand.

I think that particular technique sort of works out. I really like that. So you're showing the explainability, almost creating a story of how a particular action was, or why a particular action was presented in a particular circumstance, so people can internalize that and understand the reasoning behind it.

That's a great idea. Um, what about PhD? Were there any other aspects that you think we need to consider from a change management perspective? Yes. Uh, we included, uh, consultants for consultants for change management. Change management since the beginning of the project, and those people would see what we were developing, and they would think about how how this would impact on on the people, on the employees.

So they would communicate accordingly to minimize the impact on people. It's very important to to think about that. At the moment you are trying to to achieve an implementation like this. Yeah, I think that's really key. And in my experience, when I've implemented both pre joining Pega and worked with with Pega clients, I think it's really important to highlight what's in it for those people, how they benefit from it.

Right. Everybody is nervous about change. If you can really highlight, um, specific things that are going to make their lives easier, make them more productive, or in the case of like frontline agents, how it's going to help them earn more commission then I think you get that really helps drive buy in.

So I think the the point about using a specialist change management team is a is a really good piece of advice. Um, so as we wrap up our conversation, um, I'd like to capture some additional wisdom from you guys for those in the audience who might be early on in their journey. Um, so if you could go back in time and give yourself one piece of advice at the beginning of your AI implementation, what would it be? What do you wish you had known then that you know now? And maybe we could start with you based on your experience at Citi? Sure.

There's one thing that we wish we had done a little bit differently was to get the user community educated and engaged early on. Um, I think we missed that, but good luck that we actually fixed it pretty quickly. But had we do it again, we would definitely get that piece figured out sooner. It's a it's a really good point. And Julia, what about you? What about a PhD? Well, I would suggest that, uh, you you guys start with something that that can show value to your organization first. Prioritize something where you can show show the the company that that you see value. Also, keep in mind that, uh, we got to work with quality from the beginning.

Uh, so when you deploy to production, you have everything ready and you always have, uh, something that is not working. But if you concentrate in quality, you would be, uh, minimized. And, and, uh, also put people first. That's the third one. Yeah. I mean, a lot of these changes, it's about getting people to buy into them.

Right? To really take advantage of it. I also really like your point about ensuring the quality of the implementation, because when you go live with that first, um, phase, it is important that it that it lands well. Right. That you land it correctly and that people see the benefit quickly. It doesn't mean that you can't optimize over time to improve it, but it's great that you're able to go in 12 weeks into production with a solution that seems like it worked really well from from day one, which I think is phenomenal.

And Joe, what about you from your experience, anything you'd like to? Yeah, one thing I would talk about would be kind of communication. So communicating with other people. So coming to PegaWorld and actually talking with your peers from other organizations and learning from them, communicating internally.

So make sure that you are keeping all your key stakeholders involved and understanding what's going on, and also communicate how what you're trying to achieve with CDH kind of supports the vision of the organization to make sure it's kind of you're delivering on the organizational strategy rather than your own personal kind of strategy, I guess.

But Jay's point about the North Star, and I think when we were preparing for these, this panel, Jay, one of the other things you mentioned that I thought was quite cool was, um, using a fear of missing out. Right. So I remember you talking about how you had channels that were showing great results, and you would use that as a way to push other people to adopt it.

You'd be like, hey, they're making loads more money in this channel. You don't want to miss out on on the success they're having, which I thought was quite a powerful way of pushing people. Everybody loves the carrot and stick. Yeah, absolutely. Nobody likes to miss out. So we've come to the end of the the questions that I had for these guys.

I think we can open up the floor. Now to some further questions. Hopefully there's some people in the audience who've got a couple of questions they would like to ask. Um, and we've got some great experience on the panel here. So this is a really good opportunity to to try and extract some, some guidance and advice and wisdom from these, these guys.

Hello and thank you for your time today. I feel like I learned something from each and every one of you. Uh, I'm from Bank of America on the corporate banking side. And so I had a question specifically for you, Jay. So at Citi, I'm sure in your digital group you touch consumers and corporate banking.

What are the biggest successes you've had integrating into that corporate banking, uh, client base? So I have to confess, we have not installed. CDH for corporate bank yet, but we have had some conversations with people on the institutional side. And, uh, in my mind, the principle is the same meaning you're delivering the right message at the right moment.

The question that I posed and maybe you can help me is who is the audience? Because you're not talking about specific individual. You're talking with a business who has many individuals included. So I guess we'll make this a dialog and not a question and answer for Cash Pro on the Bank of America side.

I view our client as a couple people. One, it could be someone in the corporate Treasury Department. Someone in accounts payable. Any finance group that's signing into your banking platform and transacting. Um, I'm sure the Citi direct platform is similar, where instead of just a user ID and a password, there is also a third company ID that groups everyone together on the same online banking platform.

So I see the end customer as both it's the companies that you're transacting with. You want a view of that company, but you also want a view of the individuals that you're operating with. So I think you have to analyze it from both sides. Yeah. That's what I'm challenged with and why I'm here today at Pega.

But and I'm trying to learn how we can do both, but, uh, yeah. Thank you for the time. Sure. We do have some other customers that do, um, business or have business clients, right? So in telecoms, for example, we have um, and Steve is just right there. Could probably talk you through some of those examples.

We have customers who cover both consumer and business telecoms accounts. And in those situations, we like to say we have a combination of an organization with individuals within that organization that we need to talk to, and we're trying to evaluate both of those things at the same time. So we have the ability to have hierarchies of customers within a particular account, for example, to help manage that.

Wonderful. Thank you both. Thank you. So I was on stage last year, so I figured I'd ask a question. Jay, I'm very you know, obviously, you know, we know each other and I know a lot about the Citi implementation. Um, Julio, you know, the less less time spent with, with your organization. But on day two of PegaWorld, what's of the conversations you've had, the people you've met, the sessions you've seen? What are you taking back to your organizations to, to do more of, to do better of? That's a great question.

I thought of many things to discuss with several departments of the bank, such as the IT Department. Also the process department. Many many advantages I see using AI on those processes and. When you have someone, for example, working on the back office and he has to know all the procedures and everything, if you have in place one of these tools that we have been hearing about this this couple of days, I think it would be, uh, we would do the job faster and we can get the right answer to the customer faster.

So this is my third year at PegaWorld, and I say every year when I come, I learn a lot. I learned from the speakers. I learned a lot more. Also feel more better when I talk with my peers because, uh, we all have those moments in life at work that say it's just not working. It's so hard. But then I realized that everyone else has the same problem.

For whatever reason, it makes me feel a little bit better. And then from a learning perspective, there are a couple takeaways that I think it's worth thinking and looking into. One is, what is the opportunity with generative AI for a decisioning engine from operational perspective, from insights generation perspective? How do we think about that? How do we make our, if you will, usage of the tool more efficient and accurate? So that's one.

And then the other one that somebody also, you know, we had a discussion yesterday that it's an interesting revelation that for us who sell financial products it's not always easy to generate demand. Right. People's financial needs don't change that often. So when it does change when you have an explicit signal, double down, double down very, very hard.

Don't let it go. In that case, I'm going to have to say that this is the advice I got last night that forget about the decision engine. If you know this is very likely to happen, go for it. And then the the third thing that, um, as I was hearing from Verizon this morning, that how do you think about the whole customer experience, not just in sales or servicing, but in its totality? How do you think about this customer experience, which has a back end operational implication that leveraging case management workflow together with GenAI and mix with the decision engine? I think it's a very powerful concept that honestly, we have not really thought about at Citi.

Uh, I have to confess, I don't know exactly how it's going to evolve into, but I think it's very inspiring that and with some more dialog, uh, with us. So I think one of the points you raised there was really interesting. You were talking about, um, I think both of you touched on it. on it talking about operational efficiency in the back office.

Um, and there's a I'm going to advertise one of our booths in the Innovation Center that I thought was really cool. I saw a demo there yesterday, which is moving towards our vision for using, um, Agentic AI to generate new actions and treatments and what it does. I'm sure you saw this in Rob. I'm sure you remember it from Rob's presentation this morning, where he talked about having multiple agents communicating with each other to try and achieve a common objective.

And in this particular case, what we're doing is we're creating new actions and treatments with multiple agents, each taking individual responsibilities within that conversation. So for example, you would have a creative agent that would generate images and text for a particular action, right. And then you have a brand compliance agent that looks at the stuff that's been generated and says, you know what, I think this aligns 30% with our brand.

You need to go and do it again. So it'd send a message back to the creative agent, which would go off and work on its creative and creative, try and improve it, and it's got to get past that other agent. That's the gatekeeper. I think putting in place things like that. It's not using Agentic AI to talk directly to your clients, but it's using Agentic AI and GenAI to generate the volume of actions and treatments that you need to have in your action library to have that hyper personalized discussion with your clients in a way that's way more efficient, right? You can really speed time to market.

And we actually had a client present on this last year, a healthcare company from Australia, who talked about how they'd been using our kind of previous versions of this tooling to do just that and seen some, some really nice results with it. So I think we've probably got time for and we've got 25 seconds, so we probably don't have time for another question unless unless there's a do you have a.

Maybe a quick one. We'll try and be quick with our answers. Um, hi, I'm Melissa from Lifetime Fitness. We don't use fitness anymore, but I put it out there so people know. Um, we are a very homegrown company, and our technology team really likes the opportunity to build it themselves. And so I'm wondering as you're trying to get somebody to do something with Pega, we're using CDH but would love to use in other ways.

How do you convince some of your stakeholders that that going with something that's already built is better than their homegrown AI? Because literally, that was the conversation we had at the luncheon meeting on the table that, you know, hopefully you have an analytical team that can appreciate that because inside the CDH box, the dynamic AI models, they're literally creating one model for any action slash treatment combination.

And in because I know sometimes verbiage becomes confusing to people. Action essentially is an offer, a recommendation, a message, and treatment is where you present that message. So every combination of the two. Pega creates a model to predict the outcome of that recommendation. And for Citi, we have thousands of those combinations, which means that if we require human beings to create these models, that's thousands of models that need to be developed and maintained.

That, simply put, is just impossible. You cannot do that. Yes. Maybe at first you can manage. I agree with Jay, but in time it's going to be difficult to manage everything. It's better if you have a tool like CDH to to do it. And you were you were asking how to convince people to to sponsors to buy it.

I think when you have a, when you can have a quick implementation and you can show them that you're getting more value using CDH. They're going to buy it. You can use test and control as we do. You can use challengers models. So you can show the real lift that you're getting from from CDH. And if I could just add to that, I think, um, you can always build anything, right? Everybody will say they can build anything.

The real question is total cost of ownership, right? How much does it cost to build it? How much does it cost to maintain it? And whether it's something that your organization should even be focused on. Right. We're a software company. We build software. We build scalable, robust software that that works for companies like Verizon with 90 million customers.

Right. And the amount of investment that we put in as an organization to build that is enormous. And trying to if every if every client wanted to put that same level of investment in, they'd all be spending tens of millions of dollars trying to build something they could buy for for a lot less. So I think that that would be my answer.

Hopefully that answers your question. Yeah, I think it does. We're already using CDH. And so I think it's the opportunity to explore other parts of the Pega product. But as a company that, you know, we have our own real estate agents and we have our own construction company and we have our built our own app and did all of our own stuff.

It's hard to get them to understand that somebody who's been doing it for 30 years is going to be the right opportunity, when they feel really strongly that they could build something. I don't want to say better, but build something comparable. Yeah. Yeah. Thank you for that question. So I think we've run out of time.

Do you have another quick point or. No. Okay. Um, so thank you very much for your your time and attention. And thank you very much to Julio, to to Jay and to Joe for sharing their wisdom with us today. So thank you very much, guys. Thank you. Thank you all for coming..