Skip to main content

We'd prefer it if you saw us at our best.

Pega.com is not optimized for Internet Explorer. For the optimal experience, please use:

Close Deprecation Notice

PegaWorld | 38:55

PegaWorld 2026: The Next Best Action Transformation: RAPP’s Clean Data + Smart Decisioning Strategy for Superior Conversions

Before Pega, RAPP outbound campaigns ran on autopilot—static messages, no learning, zero personalization. Pega's Next Best Action (NBA) promised intelligent decisioning with real-time creative feedback, showing exactly which treatments drove engagement. See how RAPP unleashed Pega NBA's true decisioning power to drive conversion uplift up to 37%. RAPP's adaptive AI model, not prescriptive rules, finally learned real patterns after they filtered 30% Invalid Traffic (IVT) from the data stream. Clean signals let their treatment pool self-optimize perfectly. RAPP will reveal how NBA's intelligence found hidden high-converters, plus their IVT-detection framework that made it possible. The $71B truth no one talks about: IVT data poison AI. They found the antidote.

All right. Um, welcome for joining. We are towards the end of PegaWorld. But I really appreciate you guys still paying attention to some really cool work that's going on. I have the privilege to introduce the wrap team and talk about what they've been doing with next best action. Over the past couple months, I've been learning a lot of what they're doing, and it's just amazing the scope and scale and impact they have. So really excited to welcome you guys and kick it off. Thank you. Thanks, Elizabeth. Thank you. Yeah. So we're we're in the home stretch, right? We're almost done. So thank you all for joining. And so what we wanted to talk to you a bit about today is a question that we've had either ourselves or within our teams. And that question is really just what if we could do this better? And I think we all have done that.

We've all had those questions. And better can mean a lot of things. It doesn't just mean, can we do this faster? Can we do this more automated better means a lot of things. But really what we want to talk about today is that question we have asked ourselves and we'll continue to. And something that's been mentioned actually quite a few times in the keynotes, and I'm sure in other sessions, is that we don't feel this is just a technology story. We don't think this is just a creative story or just a data story, but we also think this is very much a people story, right? And when we talk about people, we don't just mean your customers. When we talk about people, we mean your team. So your internal teams and the teams who are either I'm not sure where we are in our NBA journey with the folks here, but I'm going to assume you're either new, you're going to be introducing your teams to NBA and aren't necessarily like full blown NBA yet.

So that's kind of the perspective we're going to have today. And what I hope that you're able to leave here today is that either in some of the questions we ask or we mention in some of the challenges that we talk about, I hope you can either recognize yourself or you can recognize your team and challenges you guys are facing, and you're able to take that away today. So first, I want to introduce you to who you're speaking to. So my name is Michelle Beck. I am a director of tech at Rapp and I lead campaign engineering. I'm Erin Sparks and I'm the director of Design automation at Rapp. So I'll be focusing on the creative stuff. Hi, my name is Karthik. I'm one of the technology leads at Rapp. I'll be focusing on data side of things. And now you know us. Give me one second.

So what we want to talk about again is the the what if right. So we talk about the static email reality. And really what that is, is it shouldn't be new to really anyone in this room, which is the traditional way a campaign's built. Right? So we start a brief, we kick off a project, we start working through all the natural steps that our teams do. And sometimes that happens six weeks early, or it happens six months before you're actually trying to launch a campaign. And to be honest, a lot of times it's like a well over a year, right? Because we have seen seasonalities, we have products we know are going to launch next year, right? So it takes time and you're doing what you're doing is you're designing and building campaigns. Well, before they were going to hit the market, well, before your customers are going to actually see the content, right?

So you're anticipating a lot of what you think they're going to want to engage with. And so the truth is, is that a lot of teams, including our teams, are very efficient at this, right? So I'm sure everyone's organization and your leadership, you have taken time and years of refining your campaign build process so that it's the most efficient, where from the moment a project is kicked off to the time you launch it to production, I mean, you can some people can do it in six weeks. Some people can do it in a month. That's amazing. If you're so in tune, you've tweaked every bit of your process that you can go from kickoff to launch in a month. That's incredible. Right? And your teams are extremely efficient at this, but efficient doesn't always equal effective, right? You can be really fast. Your teams can be very fast.

And our teams are very, very good at building campaigns and they are great at it. And you can be the fastest. But does that mean that you're as effective as you want to be? Does that mean once that campaign's out to market, it's actually bringing the value back to your, your company or your organization, your brand. But sometimes you're saying, well, speed is important and it is, but are you actually getting the value out of what's going to market? And so what the what if question does and it has for our team is it's allowed us to think differently. It gives you kind of a permission to start questioning that really effective process, right? So again, it's effective, it works, but we can start kind of finding opportunities there. And so part of it is kind of when we say, well, now we want data to guide our decisions.

Now we want customers to signal what we're sending to them. What does that do that brings change and change? If again, if we go into the perspective of our teams, change is uncomfortable and no one likes change, it brings uncomfort, right? So what do we do? We have to find the ways to bring our teams along with us. So while we're incorporating this new and exciting idea like NBA, we're bringing them along so that it's a little less uncomfortable. But first I'm going to show something that's probably very comfortable to everyone, right? And that's this diagram here, and I'm sure I don't have to explain this to to anyone in this room. We could all probably build this in our sleep and do this in our sleep. And it's the traditional way we build campaigns, which is you get kicked off. That person at the top is essentially like the audience or segment that you want to speak to, and then you predetermine this entire journey for them from day zero, all the way to maybe day 40.

And you're like, okay, I'd like them to receive this piece of content on this day and then this day, and maybe, maybe we branch off a little bit, right? So there's that branch because someone engages with something, we're like, oh, actually they should receive something a little earlier because they showed more interest. And that's, that's typical, right? That's what we're all used to. That's what we're all used to seeing, right? And this works, right. And I think we can all really recognize in this room that that works. And it will continue to work for many organizations, for many teams, they are going to continue to build this way, and that's okay. But what happens is this natural gap starts to happen. And if you can think about it, on one side is the way you're used to building campaigns. Again, that very efficient way of building.

And then on the other side, as you've heard also in the keynotes as well, is customer expectations. They're starting to change because not just the way that you are engaging with them, but the way other brands are engaging with them, right? So their expectations, expectations are changing. And what happens in that natural gap is you end up placing your customer in the middle, right? And again, this works and it will continue to work. And what happens is the gap just grows. It continues to grow. And so your customers are going. And I think we've all done it where the minute you finish binge watching a series immediately. Right. Three recommendations come up for you. Here's to binge more. Right. Probably based on the the series you just watched, how quickly you watched it. Right. So and the expectation from the customer, from the person watching, is that that's made for me.

Those recommendations are built for me. So when you continue to work this way and continue to build this way, again, it will be effective, it will work, but you're just creating a bigger gap, right? It's just going to increase. So how do we close that gap? And part of that is the original question of what if we did things differently. Right. So continuing to build this way is no I think we're okay. I think our teams are efficient. I think they're good. But if you say yes right. How then do you start approaching that change and that transformation? Because again, this transformation that we talked about or we're going to talk about is not just for your customers, it's for your team. Your internal team has to understand NBA. They have to understand the value that's coming out of changing the way they've worked with campaigns for years.

Right? So how do we do that? And part of that again, is is bringing in NBA and transitioning from schedule driven campaigns, which is what we just saw to data driven campaigns, right? Where we are reacting to what the customer is saying more in real time, those customer signals, right? And saying, well, they're engaging with this. So now let's, let's present them something else that's relevant to them and where they are in their journey. So how did we do that as a team? Right? Sounds great. How did we do that? And the way that we approached it in our team was we maintained both Bau and incorporated NBA. Because you can't break what's running right. You can't break the Bau. That's happening because I think it's very rare. I'd be interested to know if anyone in this room has ever gotten a client that has net new, has no emails, no SMS, has nothing going on.

And you're like, hey, this is a blank slate for you. Day one cold start. Have fun. Right? We always have to maintain Bau in some form. So the way that we approach it and Aaron, we'll get into more of the creative content side of it. And Karthik will get into the data side. But the way we approached it was maintaining both the Bau and when we could find opportunities, which really were challenges, right? But we approached them as opportunities and say, this is a great place for us to incorporate NBA and what that does is it allows the content and NBA to learn alongside of your Bau content, but also for your teams to learn alongside because your teams need to come along with you. So with that, I'll hand it over to Erin. Thanks. Yeah. Let's talk about how the team comes together in the creative world.

Right. Letting the system explore. This is different, and it's very hard for creatives to let that control go. Right? We've got to like, feed the model. We call it feeding the brain. We always say it right? Feed the brain, feed the brain so we can learn. But as a creative, it's really hard for me to take that learning, right? I think whatever I design and whatever my team design is the best, right? It's beautiful on my monitor. It's going to be received well. But what we started to learn with the NBA is that's not always the facts, right? So we have a little bit of that creative bias when we talk about like messaging styles or, you know, treatment plans and stuff like that, all of that's kind of like different. And we're having to let that machine tell us what's working right. So we can then as a team start shifting, right.

Um, it was a, it was a really big unlock for us is when we, you know, creative just is no longer just an output, right? It becomes a behavioral signal that's going to feed the learning system. Um, that was like really big for us, right? So, um, we take those learnings and we apply them to our treatments and, you know, while we have the Bau running, right, we'll take some learnings and we'll say, hey, this was doing really well with this audience based on the segment. Let's take some of over that and put it over into our Bau and see how that's going to perform over there. Right. And when we were doing that, we, um, we started to learn like some of the things that we predicted that were going to perform the best. The machine was saying, absolutely not. It's not, it's not doing well.

So it's like pom pom pom for creatives. Um, but, you know, it's good to learn and it was fun. And so that's how we started to like shift a little bit and take those, those hidden wins based on our different treatment pools. And then we started doing that. We started to see a 30% or 37% increase in our conversion rates, right? That's not just like operational transformation. It's like super big numbers, right? For, for us to win. Um, this optimization cycle became dramatically faster because the system was continuously Learning instead of waiting for your traditional post campaign analysis. Right? So we also reduce some manual intervention significantly. Which teams spent less time configuring that static look and more time focusing on the strategy, the testing of the frameworks and that treatment quality. And then also, I think what was most important for us too, is we started to have that confidence in the AI decisioning of it.

Um, and with that, you know, having that confidence, we were seeing those big numbers change from a business perspective. Um, the confidence and the adoption of this just kind of naturally took course for us in the team. And so future state for us, um, we're going to move beyond those isolated campaigns entirely. Uh, personalization no longer just stops at that subject line. It extends through sequencing. Your creative assembly offers timing and even your channel selection. So instead of building this one static experience for an audience segment, we're moving towards more of that modular creative systems that NBA can assemble dynamically and based on customer behavior and that journey context. So that means now your experience is going to evolve continuously. The system learns not only what content works, but when it works and what sequence and for what customer really big unlocks. And so ultimately, this just becomes less about sending campaigns and more about orchestrating the conversations with the customers in live time.

And now I pass it over to Karthik to talk about the data. All right. Last day of the conference. It's almost 4 p.m. we are in Vegas, and I got the distinct honor to be that last person between you and the happy hour. And talk about the most fun topic, AI and data, right? I know, I know, but we promise if you stay with us for another 15 minutes, it's going to be totally worth it and your happy hour will taste much better. So how many of you are currently working on an AI project? It could be Gen AI machine learning models. Please raise your hands. Great. How many of you genuinely trust your model outputs? Wow. Zero. The gap right there. That's what the next section is about. AI can go sideways for various reasons, right? Wrong features. Bad architecture. Decision. Poor system design, misaligned business goal, your leadership changing direction every two weeks.

But we can be here all day and talk about it. But today I'm just going to focus on one single element that causes the most invisible damage. That's the data feeding into your model. So for the next 15 minutes, I want you to have this one question in the back of your mind. That is can we trust our data? $71 billion. Wow. Vegas is the perfect place to talk about the wasted money. So I was I was feeling really bad this morning because I should have gone to bed like 10 p.m. I ended up playing blackjack, craps. I lost like probably $1,000. But looking at this number, that is negligible. I feel better now. Although I built this slide, $71 billion, that's how much money was spent on fake traffic back in 2021 alone, right? That's not a recession. That's not a bad strategy. That's basically the opens, clicks impressions that the marketers paid for that were corrupted data.

A research company called Luno, did a study and found that out of 2.6 billion paid ad clicks, that was done on every major platform that you could think of Google, LinkedIn, meta X. And what they found was astonishing. One in every 12 website clicks were fake. One in every 12. What it means is, in addition to the drained budget, think of the contaminated serums, distorted analytics and your sales team was chasing leads. That is not even human being to start with. And if you think about downstream effect, in addition to the drained budget, bad decisions, missed opportunities, and if you consider all those factors, the true cost wasn't $71 billion. They projected it to be $204 billion. That's astronomical amount of wasted budget spent on corrupted data, right? The research stops there. Stat stops there. But what I want to add to this section is based on what we build with the AI systems over the past few years and what we found.

So this is one of the reasons I want to rethink how we approach the AI product development. Every teams right approach AI product development as a traditional software development. You get the requirements. You write lines of code, you build it, you ship it to prod, do some basic monitoring. But the best way to approach AI product development is the 40 2040 model. So if you look at the centerpiece, which is the 20 percentage, only 20% of the effort goes into building and deploying models. I mean, but if you go to any conference, right, all the keynote sessions, break break sessions and LinkedIn posts. Everyone talks about building the most sophisticated model in the world, but all it needs is 20 percentage. If you look at the right side, which is the evaluation and monitoring. You can build the best model in the world, but if you're not evaluating, it doesn't matter.

Think of the gen AI system. And let's take rack system as an example. You need to measure the retrieval. You need to measure the generation. You need to make sure a model is not hallucinating. You need to measure the recall at K before you ship it to prod and monitoring. MIT, Cambridge and Harvard did a study and found that 91% of the model degrade over time, 91%. It's not like, phew, it's a 91 percentage. So you need to make sure you evaluate your continuously monitor your model for data attribution shift, feature shift, and you need to retrain your model. Bring a challenger model in place then so that you can challenge your champion model. Then replace your champion model by challenger. The left side is a 40 percentage, which is great data and security in order. Before you do any of those on the right, you need to make sure you get your data in place right before you start with your architecture decision.

Before you start with your system design, before you even pick the algorithm or model, you need to make sure you have the right data in place. And if you take a step back and look at this, 80% of AI investment is outside of your model. So that is where I think most of the teams are missing. And that's what we want to focus. So think of it more like NFL league, right? Everyone wants to talk about the quarterback in terms of like highlight throws, big arms, MVP trophy. You can get the best QB in the league, but if your offensive line is bad, the QB is going to be sacked left and right. Right. So your data is your offensive line. It is if it's invisible when it's working and it's catastrophic when it's not. So what is a bad data look like? So let's say your AI is making decisions out of three different channels email, web and SMS, right?

So if you look at the email channel, all links are clicked under three seconds. It comes from a legit data center IP address and the clicks are milliseconds apart. It's basically safe links and Proofpoint, your security scanners are doing their job. But the model doesn't know because the model is not going to check if it's a valid traffic or invalid traffic. Same thing if you look at the web, there is absolutely no mouse movement, right? 100% scroll and no fingerprint. It's basically crawlers, headless browsers doing their job. Similar case with SMS channel. The response is under ten milliseconds. No fingerprint. They never opt out. But if you look at the human behavior, human beings. We are slow, selective and messy, right? Because we click 1 to 2 links between like few minutes apart, we get distracted within star reviews or someone is calling you. You want to respond to a text.

So that slowness, that imperfection is the signal we need to pay attention to. The bottom section says it better than I can. Bots are fast, thorough, and perfect. Humans are slow, selective, and messy. So that contrast is the signal that we need to pay attention to. So when we started building the NBA, as Michel mentioned, initially we were not using the AI powered decisioning. Then we started using NBA and our focus was into the model. How do we train the model? How do we get the AUC score to 92 percentage? And how do we make sure the model is not overfitting in production? And we never thought of the data that is being fed into the model. Then once it went to production, once we were measuring the uplift and everything, then it occurred to us that the data that is fed into the model might not be 100% clean.

So what we did was we considered two data points and we tried to clean that. Just considering those two data points, we were able to get 30% cleaner data. And that was completely a different AI we didn't change the model, we just changed the data that is being fed into the model. So we saw you saw that 30%, we cleaned the data, then hit us really hard. So what can we do in the future so that we can get to close to 80 to 90 percentage? Of course this is a proof of concept. This is still not in production. Maybe next year we will come up with the story to the PegaWorld 2027. Maybe in the prime time if they invite us and show the output. So I mean everyone when they talk about Pega NBA, it's going to be the decisioning layer, right? That's your MVP.

That's your quarterback everyone talks about. But what do you see on the left is what needs to happen before the QB touches the ball. So your first step is the traffic is going to come from various sessions, which is web or app, and you're going to get the mixed mixed signal, which is going to be real traffic, invalid traffic. So step one is acceptance. You can't really change much. This is going to be a combination of both valid and invalid traffic. Step two is where we get surgical. So Pega Prediction Studio offers a feature which is amazing, which is bring your own model. So within Prediction Studio you have a model connector which makes a synchronous call to models, which is built out of Pega. So in this case it's suppose two major cloud computing platforms, AWS and GCP. Within AWS you have the SageMaker.

And within GCP you have the vertex AI. So we we are considering using a boosting algorithm for anyone who has a machine learning or data science background knows this extreme gradient boost. It's one of the very popular algorithms. It has won most of Kaggle contests and everything. So we are using a binary classifier approach and we will generate a score called IT probability score, which is basically a confidence score which ranges from 0 to 1. Zero means the traffic is clean, one means it's invalid or mostly corrupted data. And whatever is in the middle, which is like 0.4 to 0.6, that's your judgment call because it's a gray area and you need to do a trade off between the precision and recall and see what are the threshold you have to set. So the model conductor calls the endpoint that is being exposed. And it gets the score.

And the score gets into the decisioning engine, which is basically the fork in the road. And for the demo purpose we have the threshold of 0.5. If the score is greater than 4.5, it's going to be invalid traffic. So this is going to be quarantined. So the reason that we have both the valid and invalid traffic is because the invalid traffic is also important. You just want we want someone to look at the data, identify the source, identify the IP, identify the pattern. And this data is gold because as I said earlier, ML models degrade in production over time. So you can use this data to retrain the machine learning model that you built. Because the bots get sophisticated, they get smarter every six months. There is going to be new patterns of data that is going to come in. So you can still use this data, and you can export this data to data warehouse where your analytics team can run some reports.

They can identify the pattern, they can identify the campaigns where most of the bots or IT traffic is getting involved, and if the score is less than 0.5, that is where the clean data, right? So we still have that within the interaction history. So we are calling it as a dual interaction history track because we don't want to lose the data. The data is going to be separated based on the flag. And this clean data is going to be fed into the model. That's where the magic begins, right? Then you have your engagement policy. You have your adaptive models, then arbitration layer. Then you can use the channel that you want to. That could be SMS, web or email channels. So now we saw a solution that gives more efficacy close to 80 to 90 percentage. And I'm not saying that is the solution. That is one of the solutions.

You could also build the models within Pega. You can use the Pega, Pmml or h2o.ai. There are multiple ways. So once you get the clean data in, that's where the real magic happens. So these are the four R's that I want to quickly talk about right message, which goes to the point which Michelle mentioned in terms of letting the model pick the right treatments or right estimates and everything. Right moment. You don't want your campaign teams. You don't want your strategy team coming up with a schedule. Let the model make the decision based on customer engagement and come up with the schedule. Of course, you can put some guardrails in the contact policy and everything and right channel. So if you have multiple channels, you don't have someone prescribing, okay, this lead or this customer has to go through email versus web versus SMS based on the customer engagement.

Let the model decide the channel and right sequence you don't want like a pre-built sequence that is going on. Let the model build the right sequence before we close out. I just want to give four key takeaways. Of course, this is not theory. This is what we learned over years. Static treatments are a solved problem in Pega. If you're still running batch campaign and calling it as personalization, Pega offers solution. The question that you need to ask is do you have the right data for it? You cannot personalize your way out of corrupted signals. You can build the sophisticated models. You can come up with a better prompts, but if your data is wrong, the AI is going to be wrong. That is, there's nothing much you can do. Stop feeding your AI a blended reality. Make sure your data is cleaned and make sure your data pipeline is disciplined before the model even sees the data.

There is no point in filtering the data on the reporting side of things. Make sure the data that you feed into the model is filtered. Start with what you can measure on the day, day, day one, or the get go. You're not going to get a perfect solution. Look at one channel, one engagement signal, one metric and see how you can clean it and you will notice a difference. And the moment you walk out of this room, any AI conversations that you have, any VC pitch, any budget approval, or anyone showing the fancy dashboard of the model output? Challenge them with this question. Can we trust our data? Thank you. That's great. I think we can take the questions. Any questions for us? Hi Karthik. My name is Rakesh. I'm from Citizens Bank. So I have a question. And if I got it right let me know if I got it wrong otherwise.

Right. So you mentioned that, you know, you have the Amazon SageMaker and you have the vertex AI, which generate help you generate the IT score, identify whether it is a clean feedback that you are getting, or is it kind of like a bot based message feedback disposition that you're getting. And then you use that for auditing, but not for feeding it into E and generating the summaries and whatnot. But if you do that, does it not bring down your click through rates? Yeah. Because you're not negatively impact your earnings. When you start. But initially you're going to see a dip. The reason is you are removing the invalid traffic. You're removing the traffic, which is not even human beings. So which is good because you're getting the truth. Initially it's going to be lower numbers. Your leadership team is going to be shocked, but there is good in the long term, right?

Because you just don't want an inflated number just for the sake of it. So there's going to be a short term. You're going to see a dip. Got it. All right. Thank you so much. Hi. My name is Elisa Carlson and I'm from lifetime. Um, what? Or I guess, how did you handle some of the reactions from any of your business partners or your internal partners about their data being bad. Um, you know, the reaction to saying, I've recorded all of this data from a long time and it sucks. What was that reaction and how did you handle it? I think, I mean, kind of what was just, you know, what was just asked is a decline is going to be scary, right? Because immediately, if you're saying we have this amazing open rate, we have this amazing click through rate and all of a sudden you're saying, actually 50% of that or more was not real. 100%.

So part of it, I think something Karthik mentioned was it's kind of accepting it, right? Bringing them along and saying, well, we understand that lower numbers typically mean bad, right? They normally signal red flags and you're saying, oh no, we're down. But it's actually saying we're coming down to reality. We're coming down to stable. And this is actually our true baseline. So it's walking them through that kind of explaining it because yes, immediately it's going to be that instant shock of any anything below. Like high numbers are great. And then you come down and it's, it seems not correct, but it's bringing them along. So there was a lot of yes, it's, you know, shock, but it's a lot of conversation. It's a lot of. And that's why if your people as well as the technology are coming together, you're bringing them along with the story.

Then you're able to say, well, we're now getting a clearer actual picture of our data. This is clean data. Now the idea is being we're going to get even better, right? So then you should see the uplift again depending on how long it's going to take. But this is actually reliable. We can say with confidence this is actual engagement with our customers. Were you able to see that lift and show people that what you were doing was valuable? Yeah. If you want. To. We had I mean, we have we had lift with our content for sure. And you want to talk about some of the data sets. Yeah. So the data side of things, I think, um, right now the solution that we have is short term, right, as we mentioned, is 30% cleaner. So once this proof of concept that we have, and if it's going to give us the efficacy of 80 to 90, that's where we should be able to measure and show the difference.

And it's just not an incremental difference, right? It's going to be a massive difference in terms of AI output because you are not maybe you're feeding a blended reality to the AI in terms of data. So once you fix that, the way the model is going to perform and what we measure is going to be monumental. Fantastic. Thank you. Thank you. Hi, I'm Caitlin with Navy Federal. I have a question about the creatives. I love how you integrated the team with the transformation. Um, I'm curious, what did that really look like? Like how were you showing them the results? How deep under the hood were you going? Just like, how did you get that engagement level to then help them change their strategy? Yeah. So from a creative perspective, um, I think where we, where it was like the hardest for us is now we're not thinking of like complete, we're not building complete.

Like we know the end user is going to see this full email, for example, right? Like we're building the pieces parts together and we're allowing the technology to, to stitch it. And, and that was kind of hard for creatives to wrap their heads around it. So we really had to partner with the technology teams of like, what is actually going to be the dynamic and what are we actually saying to these end users? And then like, how is that performing? Right? So we're able to see that live on how it's going. And we just partnered with our tech team and, you know, brought our clients along for the ride to, to show them like, hey, this is what we're learning by building these, you know, dynamic modules and stuff like that. And seemed to be, it seemed to be great. You know, change is always tough for people.

But when you start to just get in there and realize we're in this together and it's going to be better on the other side eventually, you know, it was good. Great. Thank you. Yeah. And to add a bit to that, I think they mentioned the keynote today. The last one is there's a curiosity that needs to happen in your teams. And that's part of it is if, like we said, change is uncomfortable. It's scary to people, especially when it's, let's say you're bringing in NBA and it's really exciting when you bring in a new technology and you're excited about it and your leadership's excited about it, and then the people who are actually building the campaigns are like, well, that's what I do all day. So what does my day look like now that this piece of my work is now being, you know, we're incorporating a new tool.

So I know it was mentioned today in the keynote. There's also a curiosity we need to inspire or even just identifying our teams to bring them along to say, well, this is going to make sure maybe the reality is, is yes, that piece of your work is going to change, but that only opens up your day to now explore other things or have opportunities for, you know, we talk about, you know, if it takes you so long to build one email, you know, that's, that's been your day to day for so long. But then if we make that faster, you can build more emails, you can build more versions, you can have more creative fun with emails because sometimes it does get a bit redundant or repetitive. Now you have extra time to really get into your creative zone versus before all of those hours were spent doing something else.

Yeah. Hi. Um, very interesting presentation, especially on the slide on that, where NBA goes next, where the right moment and the right message should be decided by an NBA engine rather than a schedule or a calendar. Uh, at the same time where you mentioned that you also should still go on. I have a specific question, like out of all the communications which the bank does, or customer communication or marketing messages, which the bank does, how much percentage would go through the decisioning layer or would through go through the CDH system? Just take an example. About 100 SMSes or WhatsApp messages customer receives from the bank, how much would have been gone through the deciding or deciding engine, or how much would have been. Just go for a spray and pray or an awareness message or something like that. How much would be that percentage. Like what was the like the the breaking point between who went through poor and who went through okay.

Yeah. How much would be the decisioning part and how much would be the. Because that is one struggle which we are facing right now to actually stop the Bau. We cannot just ask them, hey guys, stop the Bau. Let everything go through. Pega. CDH. That's not the question. That's not the thing which you can. So how should we reach that point is my question. Like, I think the best way to start is come up with the a B testing plan, right? So you split the traffic and with your in terms of your traffic splitting strategy, whatever you want to do, like 30, 70 or 5050 and do that in a machine learning term. They have a concept called baking period where you have a certain number of months, you define that could be two months or three months and have the traffic split and measure the performance, right?

Which one is doing better by us as NBA? Then if it meets your business goal, then probably throttle up. Your 50 can go to 70, and if NBA is meeting the expected outcomes. Then eventually you can sunset the bayou and go to the NBA. Right? So it all depends on your business goal and how much your team is willing to do in terms of a B split and everything, and other parameters like the data and other stuff that is going in, also plays a key role. And good conversations to have with your marketing sciences teams, because when they ultimately have to come back around and report on those things, right? So it's, it's really that collaboration to say, okay, this is, you know, 5050, which was traditionally what we did. And then once we came back and said, this is our observation period is over, we think this is actually doing much better than we would switch to like a 6040.

And then we continue up until, you know, all of our teams are are really confident in saying, okay, we do actually see the difference. And then everyone then gets switched over to NBA. Okay. Any other questions? More.

Related Resource

Product

App design, revolutionized

Optimize workflow design, fast, with the power of Pega Blueprint®. Set your vision and see your workflow generated on the spot.

Ready to build a Blueprint?
Choose the reinvention engine for your needs.
For workflows & app design

Reimagine your processes and turn any workflow into a build-ready application with confidence.

Pega Blueprint®
For marketing & CX strategy design

Visualize customer journeys and engagement strategies across all touchpoints and activate them.

Pega Customer Engagement Blueprint™
Share this page Share via X Share via LinkedIn Copying...