PegaWorld 2026: From Skeptic to Believer: How Wells Fargo Got Stakeholders to Trust Automated Decisioning

I am excited to be here and talk to everybody about this. It's like the story of our life from the past few years. You got to hopefully you heard from Giles and Marcia at the the keynotes about Wells Fargo. This is more of a look behind the scenes on on how we did it. Less so like of a technical kind of talk and more of, of like how we got buy in. I'm the here, let me get the clicker here. Um, from skeptic to believer. So, um, I think Pega came up with this name, but it's, it's great. It's provocative. Um, and it's, I think maybe, hopefully a lot of you resonate with this. Um, that it's even the, like the best ideas, the most sound theory, the, the great product features that are baked into this kind of stuff. Um, all of that doesn't automatically mean you'll be successful in your organization?

And so we're going to focus here on the path that we took to get that buy in with stakeholders and make them believers. So, um, I won't, I won't belabor this slide. You heard a lot of bragging from Giles. Um, but we're really, really proud of the platform that we built for decisioning. Um, it's, it drives tremendous value for our customers and our firm. It makes a huge difference. Um, it's performant. Like it handles incredible scale and diversity of messages. Um, it's explainable. Like we'll get into that in a, in, in a little while. It's very configurable. Like we took all the best stuff off the shelf from Pega, put our own little tweaks on it and it works great for us. Um, but you know, despite all that, the platform doesn't sell itself. Um, I wish it did. Like it would be great if everybody kind of subscribed to the religion of it.

Um, and, and, uh, you know, raise their hands for it, but it takes, it takes more than that, more than a list of product features. It's, it's a cultural change within your company. And that happens not just through, you know, like the merits of percentage lifts and things, but about, uh, you know, relationships and trust. So, so trust, like, um, you know, generally speaking, I subscribe to this. It really comes down to a combination of three things. You have to have shared goals with the decision makers. You need to demonstrate competence and you need to demonstrate reliability. And you have to do that over and over again. It has to be a pattern. Um, the, the shared goals is, is critical here for, um, you know, not telling your stakeholders what they should be doing, but instead starting with empathy and listening and understanding what those targets are that you can share in and be an enabler of, like so that this platform, your work can be the difference maker, um, competence.

So this one's, this one's obvious, but like, you know, I think of Amazon's leadership principles of like good leaders are, are right. Often like you have to be right. You have to know how the platform works. You have to know how that translates into the objectives that you outlined with your stakeholders, that cause and effect relationship. And then you have to you have to execute on that. The execution is super, super hard. It's easier said than done. And then reliability. Um, so like the, this has to be a, a true, sustainable way of operating and not just like a one time blip where a test worked. Well, it has to be sustained and scaled up. Like if you're, if you're not growing, you're shrinking. And so the, as you do that, the reliability challenge gets harder and harder. So it's something you have to keep in sight.

Okay, so we're going to we're going to use our time here to explore like how we went on this journey to kind of like edify these three things and, and instill that with our, with our leadership. Okay. So rewinding the clock, if you, if you heard Giles and Marcia's presentation, we had like the three, the three acts, the three stages, I entered Wells Fargo kind of right at stage two, which in my opinion was, was the most exciting time because like all the groundwork had been laid of saying, look, we when we create a action, it runs, it gets shown to the right people who are allowed to see it, like all of the must do kind of operations. Um, but where that left things was, um, our stakeholders across the marketing organization, they were left saying, what are we really getting for all this?

There was a lot of opinions, but no clear strategy of saying like, what, what is this decision engine for? What is it trying to maximize? How does that relate back to what all of our personnel goals are? Um, and that then very tactically relates to the objective function. So, um, at the time we were optimizing just on click propensity alone, everything got a click propensity. It got ranked very little, um, you know, tie into value if at all there and also with, um, kind of like very little oversight of audience eligibility and the drivers of propensity, we saw huge shifts week to week as new campaigns launched of like, why is this one gobbling up all the impressions on the channel or what happened to this one? It was huge and it disappeared. That kind of volatility led people to to believe like we had no control of the system.

Um, and then the cause and effect, I think like, I love that Giles hit, um, on the black box term. Hopefully that's something you guys have ever heard. I've certainly heard it of people referring to the system as a black box. It's not. It does what you tell it to do. And it's important though, to like, perception is reality and to understand that where that's coming from and make that understandable to your stakeholders. So, um, that then demonstrates that competence, which then can be repeated and you get the business value out of it. So this is where like we found ourselves back in May 2023 when we started this, this chapter. Okay. So first for like setting a strategy, um, which then we'll talk about like how the, how that played out over the next couple years. Um, there were four things that we really needed to do.

So first, back to that shared goals idea, establish clear objectives with Marketing leadership. So it was thankfully very cut and dry. I actually wish it was a little more sophisticated, but the idea was we need more accounts, get more accounts opened. We were for Wells Fargo at the time, in a place where we were turning a corner reputationally and with standings with regulators, regulators to to go back into growth mode. And so they said we need our own channels to, to do that more. Um, so that, that led to a very clear decision that says acquisition messaging comes first. Great. So that's actually pretty easy to accomplish tactically in the system. And we could demonstrate that. Most importantly though, soliciting input from your stakeholders is important. Like I got to play the new guy card when I came. And that helped a lot to say.

All of your perceptions are reality. All your problems are real. I didn't make any of these problems. I'm here to to help you with them. It's never too late to like, turn, turn the page on that. But importantly, probably a lot of you are product managers of the CDH space. Like problems are real. Pain points are real. Solutions should come from you and your understanding of how the system works and where its strengths are. So take all of that, digest it, and build it into like number two. They're the data driven strategy. So for competence, like there's nothing better than showing objectively through the data that something's working and kind of breaking this down into a few pieces. First is like create a vision that says how you're going to use like, what is the role of decisioning at your firm according to those objectives there?

It's, I mean, for us, first and foremost, it was a way to, to make sure the products that people were in market for were surfaced to them at the right time. But then it goes far beyond that. You know, make sure you're hitting on those objectives, but then going to the target state that you want to do how things ladder together. And all of those outcomes need to be measurable. We invested a tremendous amount of time into our like our data measurement to say like clicks are not enough. We need to get all the way to outcomes and value those and create like sort of a, a way for apples and oranges to be rank ordered together that can allow us to show that we're not picking favorites. Um, that also relies heavily on the models. So like, of course, the, the models are great.

Like their value promise is that they're autonomous, but like there are seeds that need to be watered. Like you have to pay attention to them. You have to invest in the predictors that are there, that you're fine tuning them, that like the configurations on when you're getting outcomes for the objective function. All these like little details matter. And by showing that you have a plan for that, like not just doing the work, but showing others that you're doing that work, it demonstrates that confidence. It's like, oh, I have whoops, I have confidence in Jeff and his team that they know what they're doing in this space. Um, that, that telegraphing is super important here. And then again, financial value being a piece of that equation. Um, and then test and learn, like, I think that's been a huge key to our success is showing through always having like a challenge or CHAMPVA kind of dynamic that we're raising that bar that the, like, the line keeps going up and to the right in an objective way that other teams can replicate on their end to check for themselves.

Uh, another aspect of competence is just in that like, uh, squelching the volatility. So making sure that week to week, we don't have these huge swings in impressions. Um, that gives you a stable ground to then build your strategy upon. So like, you can't really, if you're always reacting to these, like shifting sands and fielding questions about that, you'll never be able to implement the data driven strategy that you've created. Number two, and then that, that black box. This is, I think, arguably the most important thing is showing through the tools that exist within the platform that it is doing what you tell it to do that through things like your product documentation, your release notes, the explainability extract, the kind of like priority score formula. There are ways to take these kind of like crunchy technical things and make them relatable to your stakeholders.

And so we've, we've took a lot of time to do that through dashboards and reporting, and it bakes into how we read our tests and everything too. Um, it's really important that we don't have voices out there that maybe like nod their head when they're in the room with us, but behind the scenes, like, say, I don't know, this thing. Like it does what? Whatever it wants. Like it's important that everybody gets behind the story of it. Okay. So our once we had outlined that of where we wanted to go, what we wanted to do, um, we took one big step. And like, this is this is always going to have a special place in my heart, not because it was like the biggest, most innovative achievement, but it was our first big step into going towards value as our objective function. Um, so the, it was very simple in implementation.

It said we have, we have these click models. Let's just apply a value per click unit that goes in there. Let's establish offline how much outcomes we're getting for that, how much they're valued. Let's put in a static, uh, you know, factor alongside that. And you multiply that together and suddenly your priority score is a marginal value. It, um, the, the details behind that, I think like how we did it. That's, that's what I'm most proud of. So first, again, like for goal alignment with leadership, um, it was, it was very important to say we're going to do this through a task and that task is going to have like wind conditions and lose conditions. This is what it means for us to roll back to the way we were doing it. This is what it means to go forward, and it related directly to that overarching strategy of opening more accounts.

So like we said, look, the the total channel PV related to those like attributed to those account openings, we're going to maximize that. Um, we did it the right way by going through very rigorous, not just like product testing, but also risk and compliance and legal review for this. It was our first time introducing a value function into the priority score. And, you know, you could argue, well, does that drift us away from being reactive to what customers actually want? Our argument is, is that the the outcomes, like propensity still drives this to a very large degree. The bands in which these values are in are actually quite tight. It's more of like a tie breaker when it comes to things that are substitutable within the customer selection. And so having that dialog out loud and establishing a precedent was really helpful to show that we were acting in good faith.

Um, the data and analytics, this is like, this is key to, to everything we did of kind of establishing, like, how do we monitor that the priority scores are playing out the way that they're supposed to, that the ranking is going well, that there is noticeable difference between our control and test group in our test, that it's like it is causal on here. Um, and so we, uh, our analytics team was kind of trained in, in like kind of traditional channel analytics, but not the depth of like the subject matter expertise for Pega and the decisioning and what propensities mean how the models work. So we partner really closely with them to, to help like cross train them and, and now like it started here and to this day, like it's been a great partnership. And so then lastly, with, with our stakeholders, we, we also like, we didn't operate behind the scenes like we did all of this in the open and we gave lots of updates to everybody we held kind of like town hall meetings about this.

The CMOs attended. We. We showed them the not just how we were constructing this, but also the test design, the measurement plan. We gave them code to say, go, you know, don't take our word for it. Have your analytics teams kind of like run this alongside us that kind of like doing it together aspect really built trust. And I can say that like the level of care that we put into this paid off that we, we don't have to do it to this degree now. Like they've given us more rope. That's the like the, the happy ending to this kind of story. But like doing it so rigorously at first it set the right tone and this whole thing paid off. It got us really well on our on our journey to realize the maximum potential of the channels. So 22% lift in MPV. It's it was material that was a game changer for us.

Okay. So once that was done, it was really important to our team to not rest on our laurels, laurels to say like, how do we keep this momentum going and make sure that this wasn't a, a blip? So first thing was about strengthening our analytics toolkit. So we said, look, we're now in a steady state. Like we're using this decisioning strategy. How are we going to measure ongoing that it doesn't take a turn for the worst and that it's still producing more and more good outcomes. So we created like a set of tools, some, some things very basic, like just literally channel performance reporting, like the bar charts there. But that's important to have one set of facts that everybody subscribes to. Like that that was something we didn't have at first at our company and then model monitoring. So think of like a set of robust features similar to prediction studio that our team can use as well as like a companion that's digestible for stakeholders, even those like outside of analytics teams.

Um, that helped to say look like we're not hiding anything. Keep your eyes on your models. We're now trying to make people owners of their models trying to say like, you're, you're responsible for them. Like, what are your seeds on this? And it started with this for win rate. This is the idea of taking eligibility of like, here's all the choices that we had versus here's what we actually picked. Um, here's where you marketer have headroom on your campaign to, um, to reach more unduplicated customers and get more frequency and the reasons why you're not like what you're losing to why there's better marginal value on those other things. And that was at first just a set of facts. And now it's a bunch of recommendations. It's here's what you can do to actually increase your propensity and drive more value. And then lastly, building upon Pega great simulation capabilities to outline even more scenarios that we could do on demand, even outside of the platform analytically.

So that gave us a lot of great iteration to plan our next big thing. Um, then it was time to get out of test mode a little bit and productionalize like how we run our operations here. So we things that we were doing sort of one off, like establishing financial values, we needed an official process for that. We needed the CFO's to sign off on that on a regular basis. We needed a way to align these so that again, like checking accounts and, um, you know, like a fixed term loan could be compared. Um, we needed fairness and doing it transparently in concert with those lobby teams helped to do that. Conversion measurements been a big part of this is saying like, we have to have a clear outcome for every action that's in the system, and they need to be measurable. If they're not like you, go last in order.

So it creates a motivator for marketers to invest in the campaigns that matter with customers, as well as establish that measurement. We need to optimize the system like that positive signal. And then and then monitoring. Like we have a great bevy of, of like Platform health measures that our product team looks at every day. Um, we actually, as a, as a company, we start our day with an 830 call where all products join to say it's the start of day call. Like we go around the room like green, yellow, red on the system. And it all comes from the like automated reporting that we have. And then, um, like I keep doing that. I'm sorry guys. Um, a personal like passion point of mine is the, is the community outreach. And that last column is like even once your stakeholders know that you're acting in their best interest and they have faith that your roadmap is good.

Like it helps your, your brand and reputation in the latitude that they give you to maintain like an ongoing relationship with them. And that takes time. Like it means office hours, answering a lot of questions, partnering to make sure that new endeavors, when they're taking advantage of a new feature, go well, like that white glove service we have like a monthly community meeting where we, we share all the cool stuff that's going on and what's coming to try to like our own mini PegaWorld, that kind of idea. It really goes a long way. Um, so this was like while we were developing, what's the next big thing in the actual like priority score and the decisioning strategy? This was being set up in the background. Okay. So, um, I mean, besides a little cynical, but it's, You know, sometimes you have days where it hits you like this of like, you're only as good as your last big thing that you've done for the organization.

So, um, it is important on your roadmap, in my opinion, to, to stage out your developments. Like if you have a rich pipeline of things that are coming next week, next month, and then in six months, there's always, um, value to add for the organization and good news to be shared. And that helps, that helps that the, all the columns that we had on that last page. So in these examples here, um, these are, these are real things that we brought to market. But I think like they illustrate that time horizon thing pretty well of creative variant optimization, the idea that you could have multiple banners for the same action or something and have the models determine the right one for each person in a way that's superior to a strict a B test where you have a winner and a loser. That's something that can be done immediately.

Like it's just about having the the treatments live and the models just do this. But the work involved was getting folks to do like take the leap of faith to take part in it, to trust the outcomes. Like there's no control group here. So understanding, what am I like? What am I really getting from that? We invested quite a bit of analytical time in it, but that's immediately available. Then there's things like moving. We talked about from like the click models before we went to conversion models. And conversion for us is often like an application submit sort of a, you know, a good measure of, of business outcome, but not perfect. Like a lot of lending products, they don't get approved, but it's a, it was a first signal that we have, but then moving all the way to completion of business outcomes. And then that's extendable to all of the non acquisition messaging, like engagement and servicing stuff too.

So that that was a deliberate step by step journey for us. But it's hard. Like you have to get the logic, you have to put it into place. You have to have those, um, like reliable counts, uh, you know, attributed to the interactions and sent to the platform every day. Um, that took some time and is in is always ongoing. Um, then we have like our model predictor. So again, at the same time, like there's always more and more customer data that could be put into the models to be predictive. Like there's, there's signals probably within your own firms you're not taking advantage of yet because they're just, there are silos for data, um, that, that you need to turn over those rocks and get it. And then there's always emergent things of like, um, there are transformer models these days that create embeddings for sequences of things.

How can we create those and put them into our adms? Like how do we take advantage of kind of like new stuff? So we have a always on pipeline of these things with our, our production data partners. Um, and then you have your big bets that might take a, like that are a big departure from what you're doing today, like prospect decisioning. Inga is here with us. She leads prospect decisioning on our team. Um, we're, I think going to be like one of the first to try to leverage adms at scale for unknown visitors, like on our website. Um, try to predict from like very loose signals. When is somebody here for checking versus credit card versus something else. And so we're really excited about that. But it's, you know, if we put all of our time onto that without those shorter horizon things like there be there be some, some quiet out there with our stakeholders.

So this is important for roadmap planning. Okay. So, um, I probably have given the picture that like everything has been rosy along the way that like you just, you just set a strategy, you chop it into parts and then you do A, then B, then C, like things go wrong all the time. And, um, you know, you could judo this into using a crisis to your advantage to gain more, uh, more like better reputation and sense of responsibility and competence with your stakeholders. So these were kind of inspired by a couple of books that that I like. Um, the first one by Admiral William, um, McAllister of, uh, like, uh, you know, managing through crises, the, the second one, Mel Robbins for let them, um, I think like the really apropos for the life we lead here day in, day out. The, the first one just with an example of, uh, fallback models, we, you know, you probably have dealt with this too, of when you launch a new action, it takes time for the models to pick up on who are the high propensity and low propensity people.

In the meantime, there could be chaos. Like while it's kind of learning that does it, does the new action take over the channel? Does it get squished and then it doesn't get the learnings that it needs? We implemented fallback models to say while something's learning, use a higher grained model as a proxy on there. So it's a brand new checking offer. We'll use like the deposits model in there until it's learned enough to switch over and graduate to its own. Easier said than done. And so there was still a lot of volatility in the system. And, and we still experienced these cases where kind of like new actions were taking over. Um, so some of the lessons from that, that book I was mentioning, um, first reports are always wrong. So like in this initial observation by our stakeholders and then our own internal reporting, like there's, there's a lot of data points, but not a story that goes with it.

It's not an explanation yet. And so it's important not to jump on those first things and burn out your team on like false paths. Um, at the same time, there's a balance to this. Like bad news doesn't get better with age. It's important to, to address this and take ownership of it fast, like when sort of bad news is out there and folks are, you know, who are out of the loop, out of the know are talking about it like it can, you know, a mountain can be made out of a molehill. And so it's important to like play a little PR and address, like acknowledge this, say what you're doing. Again, like just, just them knowing that you are taking appropriate actions and that you have a plan helps people, um, be put at ease. Um, weaponize the truth. So this goes hand in hand with that of saying, uh, like, don't let those false rumors take over the narrative.

If you're the one that people are looking for to help them through the problems, then you get to control what the the truth is. And like going back to the principles of the testing and all of that, we, we show our work. Like we say, this was the problem. Like, here's a few lines summary, but here's an addendum. Feel free to click this and kind of see the evidence of what the problem was. Um, and then lastly, move all options forward. So going hand in hand with that first one of not just jumping on that first like clue, um, it's important to establish with your trusted inner circle, what are all the ways that we can combat this and then explore all of them and then make a decision once you have the viability, um, like kind of assessed for all of them. And then that's a, that's a much better way than putting all your eggs in one basket.

Um, so then on the right hand side there letting like letting the data talk for you. So this, this lesson being, um, again, kind of back to the religion of things. I think a lot of us would say manual interventions are not good like that. The models do what they, what they do with, with a value component in a way that is better than humans picking things. That's like why we pay for this stuff. Um, nevertheless, there's always going to be maybe a new leader or somebody in the organization who wants to try putting a lever on something rather than kind of argue until we were blue in the face, we said, well, you know, here's, here's the evidence, not just our belief that that won't work. That's going to give you more impressions with this part of the diminishing returns curve. That's not going to help you at all.

Um, but we're not going to say no, let's run a test on this. And so we did that and let the kind of results of the test speak for themselves versus the theory. So, um, it showed like, nevertheless putting a boost on certain actions did not drive more accounts. It did eat away at the stuff that it displaced. Um, and, you know, we're really proud that we left that meeting with the CEO saying, okay, we're not going to do this anymore. Um, so like that was a big victory on this. And now it's a case study that lets us kind of point to that next time that idea comes up versus having to keep arguing. Okay, so I wanted to close this out with just a few more case studies of how we kind of like activated on, on that framework of the shared goals, competence and reliability.

So the first one being what we've done in our branch channel, I called out Inga before James works in our branch channel. Um, what he's done with our product team under Rajesh has been really tremendous in, um, getting them to, to believe in the decisioning. So, um, for the, for the longest time, like several years, leads in, in branches were determined by a forced prioritization waterfall. Like leaders literally picked, you know, max out on this message, then this message, then this message. Um, so what that, that does in practice is you're scraping the bottom of the barrel on that very first one. Like people who have extremely low propensity are being forced to get phone calls about a certain kind of lead. So we we knew that we were standing on firm ground that we could we could beat that we could perform better. But it was about making people believe in that.

So showing empathy of recognizing past stumbles in the branch space of where like, see our platform, we call see, like our CDH didn't hit the mark and delivering the kinds of messages that they want or the performance that they needed. That kind of led to that idea of having leadership take over those decisions and then aligning on common objectives of saying like, branch is not digital. It's different. The way that we measure success and have that human element with the bankers is different. So how do we understand that mutually and then translate that to something observable within the system? Um, so I just talked about that for what we've done in the decisioning strategy of catering that to those kinds of outcomes. And then kind of like similar principle to that value based decisioning. We didn't wait for perfect. We said like, where can we get started on this?

Um, we wanted to have conversion models because we were already there in digital. But guess what? Like the volumes are much lower on phone calls from bankers. And so our ability to go right to conversion models was limited. So we said, okay, we'll, we'll model on a likelihood for an appointment, getting a customer to agree to an appointment, because that's a, a first order kind of like step on their path to outcomes with the firm. Um, and that led to being able to test quicker and achieving results faster. So we, we again, leveraged like test and learn. That's kind of like a non-negotiable must have for these kinds of things. We had clear measurement.

And in fact, this was really the first time, not just about decisioning, but for these channels as a whole, that we understood the value they were driving, that we actually understood not just, um, kind of like activity of these many appointments, but we said they translate into these kinds of like deposits and investments, balance gains and these kinds of account openings and things of that sort that were our success criteria and are now just part of channel reporting and then our attention to like kind of like long term roadmap opportunities to say what we're building here is not just quick to get our foot in the door, but it's, we can build upon it. We can, we can like very clearly say, this is what we're going to get now, this is how I would tweak this next quarter and then the next quarter after that.

So we're really happy with this. And we we have much more coming through the rest of this year in the branch space. Okay. Non acquisition messaging. So I'm, I'm personally like really envious of some of the other companies I've talked to here that, that strike a really great balance of when to sell and when to nurture relationships, the kinds of like balance in the inventory of messages for their engagement and servicing. I told you at the very beginning, we were very laser focused on account opening and and we executed on that and accomplished it. Um, but the, the models have in the decisioning strategy as a whole has matured to a place where we don't need to be wasteful to go all in on acquisition all the time. Like we know who we're wasting our breath with at the bottom of that propensity curve. And it's, it was time to kind of like open that up to the non acquisition messaging instead of treating them as like second tier conversations.

So, um, even though we sort of always knew that was where we wanted to go, there's that idea of there being a right time and place matters like of knowing your, your opening to strike. So in sort of the CMO priorities for, for 2026, it was the first time we saw like an overture towards, um, not just growth, but, like sustainable long term relationships, like primacy with the bank. Loyalty. And we said, well, the way to do that is not by opening more accounts, it's to execute on these things that get people to use their existing accounts more. So we knew we had support from the top if we could execute on this. And that opened the stage for it. And then we showed the viability that we said, look, that that kind of diminishing returns curve of how great our our acquisition model propensity deciles are to say like, this is the open white space where we need to, um, you know, bring in additional inventory of conversations.

The decisioning strategy changed. So we, we said, and this is something our product team was very happy about because we had these levers in place that forced acquisition at the top. It was actually pretty cumbersome to maintain and update through all the Pega upgrades and the strategy changes. We said, well, this is easy. We'll just get rid of that stuff. But we have to do it in like a test and learn kind of way and, and not just take it for granted that that it would functionally work. Like there was a lot of testing that went into that to ensure the ranking was done correctly and all of that was accessible to our stakeholders. Um, so like all that preparation paid off and, and now right now we're in the middle of this task. So, um, so far, like the data is not all in, but we're seeing, um, we're seeing great uplift of things just as, as like the theory predicted and the simulations that we had done ahead of time, the vast increase in impressions that are now going to the non acquisition messaging are not coming at the, um, at the detriment of the acquisition.

We're, we're largely getting the accounts that we always have. And there's a, there's a place for both and a role for both of them within that. So hopefully what this does is like breathe new life into kind of like the whole marketing organization to see the channels in a new way, as well as like simplify our life here in product and allow us to like put sights on the next big thing without sort of the baggage of the old strategy. And then I wanted to touch on one that's still very much a work in progress. Like we haven't solved everything. There's, there's always a new horizon. So email is a channel in which it's going through our decision engine, but we're not really decisioning it. We're not making choices on targeting for that today. It's one of our next big frontiers. There's probably many of you in the room for which like your firm's getting tremendous value out of CDH and email.

But that's something like we've yet to cross. Um, we're the principles of like how we're going to get the buy in to this still apply. What we're up against is, and maybe this is relatable to some of you. Um, some of our, our marketers are kind of like rewarded on volume, just like the more they send out. Even if that's very inefficient, the the more success there. There seem to have within their corner of the shop. And so this is a, once again, like another culture change of saying like, we don't need to be wasteful like that. And there actually is a downside, not just to this channel, but in marketing as a whole when you're putting irrelevant things in front of people. Um, but like how we cross that chasm in understanding of, of getting them to take that leap of faith still comes down to like the competence and reliability of shown through the system.

So, um, I think we've, we're happy with the decisioning strategy of how we're going to handle the priority score of, of handling like net upside from the conversions with net downside of kind of like going ad blind or unsubscribed to these emails. Um, the reason to believe that we'll be able to pull this off is based on the models is like when we have the fidelity in the propensity to get both sides of that equation correct. And so that's a work in progress. That's where we put our time and energy is hopefully not into, um, you know, sustaining PR communications and goodwill towards stakeholders, but actually investing it in strengthening the platform. Um, so over time, what we're hoping to see is that we, we get a pilot going where we have a hand raiser to try this out. Um, in a world where the models are ready to handle it, and then maybe we'll be back in a future year to talk about it.

Okay, so I just wanted to end on this again. Like, um, I think, I think hit it home. There's, it's really, it comes down to these things and this, this slide, like nothing on this slide is about decisioning. This is just about how, how relationships are formed, how trust is built. Um, in a, in a corporate workplace, like these things are, are just, there, non-negotiable. And depending on where you're kind of like decisioning team sits within the organization kind of changes the dynamics of how, how much, um, you know, weight and priority, like the order in which you go about these things. But they all have to be done. And like at the bottom, the repeat, repeat, repeat. Like it has to be sustained. I mean, I wish it were easier, but like, that's, that's our job that we all have hard jobs and, and hopefully you get the, the fun out of it of like actually seeing people turn from like those skeptics to believers on this to come along for that journey with you.

Um, so that's it. Um, hopefully that was useful. Um, I think we have some time for questions. So if you, if you have any, feel free to come to the mics. Thanks. Thanks. Don't be shy. Hi, my name is Rakesh. I'm from Citizens Bank. So you mentioned that, you know, I think in the initial stage you probably prioritized acquisition. Is that something that is still in place or it's completely removed? And now you let the engine prioritize between acquisition and nurture and services and all those kinds of things. If you and maybe I'll pause over there. Yeah, that's that's kind of this stage we're entering now. That's the test that we're running at the moment is like allowing the system to just globally kind of optimize these things based on the signals it knows, you know, of the stage that the customer is in. I think what we're going to find is the decisioning works like, just like a lot of you have already discovered to allow it to do that.

But now put the onus back on Marketing. Like we have a dearth of, of good non acquisition messaging to show people. And so we need to kind of like dust that off and say, what are all those things in the like opening a new account is a rare event. Like it's not the normal day to day operation interaction with a bank or a lot of other firms. And so what are those other things that we should be talking to people about along their journey? So I think like decisioning is in good shape, but like we got to, we got to put it on steroids, right? So tied to that, you know, I've got a couple of questions, right? So when you have your mini PegaWorld sort of things, right, with all the marketing leads and all, how do you prioritize between different types of acquisitions?

You may have a card, you may have a checking account. How do you deal with those battles? Point number one. And the second thing is I saw you mentioned about net, uh, value score, right? How do you guys think about deriving a value for those kinds of service or nurture messages? Right? Yeah. So we try whenever possible to have like one objective function because it's just, it's, much more straightforward and explainable on the system. And so for us, that's like the marginal value of the interaction. So we, we tie it back to like an impression or a phone call or whatever it might be. And the way we get that value is just through outcome attribution. So it's, um, the outcomes can be wildly different, like you're saying of it could be acquisition of opening a checking account, it could be signing up for a payments feature within your existing account.

It could be a happy birthday message. Like it doesn't matter as long as there's a clear, um, sort of causal outcome that you can attribute to it and that everybody believes in and is fair because that equity of being able to compare these things on an even playing field is super important to get that trust in it. Um, I'll also say that, um, when you have these things that are very different, like these different actions that somebody is eligible for, they're not substitutable with each other. And so if you're doing a good job, like if you're high propensity on a mortgage, you shouldn't necessarily be high propensity on something completely unrelated, like, um, like signing up for payments. Um, so like we look at this kind of stuff and we try to see like, it's not, it's not bad to us when we see that a customer is eligible for a bunch of things, but is only high propensity on a couple of them.

That's good. That's a, that's a good signal that we, um, these things are sure. Marketers would love to talk to them about it, but the customer is not interested in that where they are. These things are what we need to prioritize. Thank you. Thank you so much. Yeah. Of course. Okay. Hi. Uh, myself Rashid, I'm from Emirates NBD, a bank in Dubai. I lead the Pega CDH practice over there. First of all, a pretty wonderful set of, uh, presentation. It was like a cheat sheet for me or what we have been struggling over the past one year. Like it's like you might be knowing like for the Pega CDH to get into an MNC thing. It's like a startup inside an MNC. You have to go for the correct set of angle, like one view, which I had during the startup thing, is that one thing which the stakeholders like is the time to market, which I couldn't see in your slides and all that is one kind of a major big value addition, be it all kind of scientific decisioning, whatever you do.

But the time to market is one thing which they value a lot. The other thing is to go for the right stakeholders. Don't go to the stakeholders who are experimenting on their product, and all those things go to the stakeholders who has the budget to support you because you are running a startup inside your organization. So that is one. So one doubt which I had, which you will be also having as a challenge like is to go from the conventional or the traditional market or campaign setup. Actually, how many, how much percentage of your campaigns would be more on the decisioning side and how many percent, how much percentage would be on the usual traditional Legacy based campaign style or the spray and pray type, or how what is the maturity percentage, which we can go for as an organization to reach that Decisioning Lead, uh, set of setup, actually.

Sorry, what's the delineation? The two kinds of campaigns. The usual campaigns that is the, the spray and pray, the just the rule based campaigns and the decisioning based campaigns which you run through. Pega. CDH. Yeah. Well, I think like, um, I hope I didn't paint the picture that, um, that like we've made our entire marketing organization like, like, uh, data decision architects. Like that's, that's not the case. I think it's actually like very healthy that somebody like a campaign manager still wants to maximize their campaign. And then we are the brain that can help like level that across where it's useful to the customer. It's like where customer meets that Marketing. Um, I think it gets actually the best outcomes when every marketer independently wants to maximize their stuff. It's when you sort of let that tide break through the door and kind of take over where the loudest voice controls things, that's a problem.

So I'd say like in the channels where we've decisioned, um, it goes based on the decisioning. And so, um, a voice will come up every now and then about the like, how do I get more for this? But what's, what's nice is we're not talking about like, okay, well, uh, for three weeks, we'll give it a boost or something. It's, it's more like, well, here is what the models say. This is what drives your propensity. Here are actions you can take that indirectly, raise that propensity. Here's what you can do to be more nuanced about your value. And these things directly raise your score and give you X more impressions. Like it's a very cause and effect kind of system. And, and, uh, like, we're proud that that's kind of the nature of the dialog. Now I understood that. So I don't think we should be very strict or stringent or having a kind of a decision.

So one thing which we were discussing inside our organization is to have a decisioning layer inside your architecture. If you bring in that decisioning layer, then for each and every communication which you send out to the customer needs to be validated or a tick mark from the Pega CDH team. Actually, as such, maybe that good idea would be a bit too ambitious. I don't know, like whether that. Is a good idea, like moving to the left of the campaign process and being involved in like designing the campaign from eligibility to how it's going to be successful, like, like how it will be treated by the models. That goes a long way. It's just about your bandwidth of being in the room with the marketers across your organization to, to help them with that. Yeah. Okay. Thank you very much. Really could resonate a lot on that stakeholder challenges.

Thank you. Hi, Jeff. Mark Hoffman from Kaiser Permanente. Hey, question for you. Um, around the methodology of engaging the stakeholders that you've outlined, which thank you for doing that. It's a problem we're facing right now because we have an area of our business that has, in the past, over the last ten years, not really received a lot of investment. And so the business stakeholders are generally don't have the muscle around how to even express a strategy, how to even think through an approach that leads them to these kinds of outcomes. Right. Um, can you elaborate a little bit on how do you engage when you, in essence, have business that don't quite know how to follow a methodology? Like they don't may not even see the value of, of a methodology to engage in this way and to think about their strategy or how to express their requirements and their needs in a very call it articulate type of way, right?

To even get to the point where they're saying, this is what I truly want. How do you work through that to where you may even have to interject and help them? Yeah. Um. If okay, so like if the first pillar was that shared goals, but your stakeholders don't have goals. Like if they, if they haven't articulated for themselves, like what they're trying to do. Yeah. That's a blocker. That's hard. Otherwise you're guessing at it and you don't know when done is done on this. So um, to me, if that's um, if that, if that's a self-aware kind of state that the organization's in, they would be happy to have somebody help facilitate structure to that. And where you could help come in would be like taking lessons from the, like other, like the other enterprises that you aspire to, to say, look like, look what can be done by using this kind of stuff.

Um, and only introducing sort of that how after the, the why, like, like what you're kind of going after that way. Um, that way it's less about you trying to like push a technology and more about like, hey, we all have to kind of do some soul searching here to get to like what success looks like for our organization. Um, hopefully you can get to the part where at least some corner of the, the business has set that clearly for themselves. And you can start with them like, like literally taking, doing a part like one channel, one organization or something. There is something to be said for that. Then you get a use case and a success story that you can then like get others to line up to. So, you know, without knowing more about your situation, I think, um, get yourself in that room for sure.

Thank you. Welcome. Hello, Chuck Roerig from USAA. Um, so one thing that I noticed on your slide about like, what's what you're working on now with marketing emails. I was curious if if you had operational type contacts or emails on the radar like that, you had some some day orchestrate or arbitrate between Marketing and, um, and operate and not like, uh, confirmation, but more like I've opted into a payment reminder. So it's, if you get it, great. But then the other, the second part is, is there a fatigue, uh, level that you're trying to be careful of on any given channel? So how do you define the fatigue level, like for customers or for. Yeah. Okay. Yeah. That's great. Um, for email, I think, um, we're setting our sights on the discretionary stuff because we're saying like, that's where, that's where there's a choice in the matter and we should be more deliberate about putting that there so that there's less blindness to it.

Like when an email shows up, it should be it should be noticeable in an event for somebody and not just like one of a hundred that they got that week. Um, the, I think we see the, the operational ones, like those, those like transactional ones to be, um, good, like context for how we do the propensities for the others to say like, look, this doesn't exist in a vacuum like your campaign, it's among other Marketing and among other things, and even among other channels, like the way that we have a lot of interaction history features within our models, it needs to not be blind to things like that. And if a, if a customer is getting those in a sort of like obligatory way where that like we have a, we have a promise to deliver those to people. Well, then we have to work around that in, I think strategically.

Um, and then for fatigue, I think we just haven't gotten there yet to establish something like that. We're we're still getting our organization to agree to what the benchmarks should be as a channel. Like before we even talk about decisioning as a way to reach a nirvana, it's, it's like, what does the email channel, what is it here to do for the company? What is it here to do for customers? And then what does a healthy channel look like? I think there's like a plight of the commons problem where nobody really has a skin in the game for the negative outcomes that happen. And if you're only looking at positive ones, then efficiency doesn't matter. So like we're trying to turn that ship a little bit right now. And I think your idea of like a fatigue metric that we want to, we want to stay below a threshold or something, makes a lot of sense.

Thank you. Thank you. Hi. Are you done? Okay. Sorry. I'll come talk to you after. Yeah. Thank you, everybody, for coming. Appreciate it.