PegaWorld iNspire 2023: Panel - Realize value from Hyper-personalization by operationalizing context-based marketing, sales, and services

Given the extraordinary global challenges, the need for brands to deliver Hyper-personalized Experiences (HPE) has never been greater. Not only does implementing the right technology improve customer experience, sales conversion, service outcomes, and net promoter scores – it reduces promotional and marketing costs. So how do you unlock the full potential of HPE after choosing the right technology?

Join us as we discuss with clients their journey supporting their strategic and operational activities, through continuous improvement and tuning, to help them to achieve their business goals, at speed and scale.

Transcript:

- Thanks very much for coming. My name's Nicholas Kitson. So I run Accenture's Pega Practice for the UK and Ireland, and also across Europe. So I'm here just to moderate this session. I'll introduce you to my two guests very, very shortly, but I just wanna just, just give you some context for the session. So we've obviously, we've been impossible to escape the hype around Gen AI, and we've just seen just two presentations that's just come up. Hyper-personalization has been a key part of AI for quite some time, and some of us were at the partner event last night with Pega, and they talked about the fact that, with the acquisition of Chordiant, they've actually been doing AI and hyper-personalization for over 10 years, and we heard that on the stage as well, and I think that's really impressive. So for today, what we wanted to do, rather than just go through a lot of PowerPoints, we wanted to have a really in-depth discussion from a couple of organizations who have really been through this process, right, and hopefully give you a chance to ask some questions as well, okay? But first of all, so what is hyper-personalization? So we at Accenture have our definition, as we do, right? We define hyper-personalization as the ability to provide individualized customer journeys and giving a consistent customer experience across all channels, okay. And we're gonna hear today from Bank of Ireland, who've embraced this journey, their organization, their culture, and their technology. And we're also gonna hear about how, so our expert, Piyush Vakil, talking around hyper-personalization and how that has affected other organizations, so. So, for the format for today, I'm briefly gonna introduce our guests, then we're gonna go for a little Q&A, so again, really trying to get into the meat of it so you can understand the journey that they've gone through, how that's really, the benefits they've taken out of that. And then also, I just keep peppering in the experience that we at Accenture had of working with other clients in other areas. But before I introduce my guests, I just want to give you a little context about Bank of Ireland, okay? Bank of Ireland is the oldest bank in continuous operation in Ireland. It was founded and formed in 1781, opened for business in 1783, and that's making it its 240th year anniversary, yeah, this year? So back in, I think around 2021, it had a market capitalization of 9.5 billion euros. And in 2021, it was the 16th largest exporter of financial services in the world. I think that's pretty good for a bank that probably most of you have never heard of, yeah. So without much further ado, so I want to introduce to you my first guest, Joe Madigan from the Bank of Ireland. So Joe's responsible for customer analytics, marketing transformation, and marketing operations. That's some responsibility, and we were talking about that. And Piyush, Piyush Vakil from Accenture. Piyush leads a hyper-personalization center of excellence. He has delivered over 15 projects of hyper-personalization across Europe, North America, Middle East, and South Africa. So I'm gonna dive into the first question, and this one's for you, Joe. So give us a bit more context about the bank. Can you tell us a bit more about the Irish marketplace, and how you embarked on the journey?

- Yeah, sure, Nick. And good morning, everyone. So Bank of Ireland, as he said, is 240 years old, this month, in fact, so a bit of celebration this month. Safe to say we weren't in Gen AI 240 years ago, but, but interesting to listen to Alan Trefler talk about his company being 40 years old, and all that he's been through. And we're a company that really puts, well, it's not Gen AI, we will become Gen AI, we put financial wellbeing at the heart of what we do for our customers. We're a pretty traditional bank. On the island of Ireland, we're a full service pillar bank, all of the normal traditional banking product, from current accounts to loans, savings, mortgages, business banking, wealth and insurance, mortgages; you know, full service across the island of Ireland. If you look at the UK market, we operate both direct to consumer and indirect; direct, with a couple of niche products, mortgages and loans, indirect, via our partnerships with the Post Office and the Automobile Association. While we're live with Pega on the island of Ireland, just think about what the way we operate in the UK and the challenges that's gonna bring with respect to personalizations, and we'll get to that. We're on that journey as well. We also have a corporate banking arm, and we just recently bought a stockbroker's to complement our wealth and insurance arm. In terms of the Irish marketplace, it's a pretty dynamic market. So we have one of the youngest populations in Europe, one of the fastest growing economies in Europe after the great financial crash. Having said that, we have just seen the withdrawal of two traditional banks from the marketplace. So both KBC and and Ulster Bank have withdrawn from the market, giving us a, what we would call a once in a lifetime market share gain opportunity, which we have taken. So we've got that going on in the traditional space, but there is quite a vibrant FinTech marketplace as well in more niche product areas. So we do have quite a bit of competition in things like checking accounts or loans, but that is niche.

- Brilliant. And then just moving on, the bank's vision, and when you started hyper-personalization journey, what have you deployed, and what were you really trying to get out of it?

- So our vision at the start is, if you think about, the way we describe it, just kinda simply, is we've got data, we've got our analytics practice and the platform, you know, various platforms, but in this case, CDH, and then we have our routes to market across all the bought, owned and earned channels. And the first thing we were trying to do was to remove the fragmentation across the channels, so to create more a sense of consistency of message, to orchestrate those messages properly, so to get that all done. And we're on that journey. I'll talk about, maybe in a second, about where we are with the journey. But the second thing we're trying to do is to move from being reactive to our customers' needs to becoming more proactive, and, ultimately, preemptive. And if you take an example to bring that to life, your customer, let's say, changes address. We wait for them to tell us they have changed address, and they tell us on the channel that they choose. We then have to have the right resource there. And everyone in the room, I'm sure, can feel the pain of having to meet service levels across various service channels. So we do that, but one of the MBAs we went live with was a change of address MBA, where we looked at, you know, took the data, we looked at the data with our analytics function, and we created an MBA that sought to present an MBA to a customer if we thought that there was an address quality issue. And we had a massive success with that MBA on the very first day of going live, to the extent that our downstream processes, our analytics worked, CDH worked, but our downstream processes couldn't keep up with it, and we had to adjust that as we went along. So that was us moving from being reactive to being more proactive with that particular piece. We've reused that MBA to drive up data quality, and we can talk about results maybe as well, but we've driven up data quality in various areas by being able to be very targeted with what we think is wrong with our data. Ultimately, as I say, we wanna become preemptive. So again, think about change of address. If a customer, let's say, takes out a mortgage with us, we know they've changed address, but do we present the right MBA to them in the right channel to capture that change of address, to prevent them from having to come to us? So those are sort of our visions in terms of our deployment. On the analytics side, we've had an analytics practice since 2012, full blown since 2012. We had outsourced it. We insourced it in 2018, and have grown it and complemented it with other resources. So our analytics function is commercial insights that sit as part of the cross-functional marketing squads, and also interface to the business. We have a data science team that creates propensity models. We have a decisioning practice, and we have a marketing effectiveness team that measures everything in marketing, both traditional and digital. Our decisioning practice has been live since 2021. CDH went live in January, 2022. We went live on Pega Cloud with 8-5, two inbound authenticated channels, and we found some really good success with that, compared to talking to peers who went outbound first. We have two inbound authenticated. We've upgraded, we've added more data last year. So we didn't have all of our data ready, and we didn't need to, so we added more data to be able to differentiate business customers from personal customers, Northern Ireland customers from the south of Ireland, given the different currencies and things like that. And then, most recently, we've upgraded to 8-8. It started this year. We had a pretty seamless transition, I have to say. Given some of our experiences with our technologies, it was quite good. We're adding outbound owned digital channels at the moment, so email, SMS and Push, and hope to have Paid Live by end of the year as well.

- Brilliant. Thanks. And, Piyush, I just wanna bring you in here, 'cause you've worked with a number of different organizations. But again, particularly talk to us about the vision of those organizations, and how you've seen them develop.

- Yeah, a lot of times they, the vision is, "Hey, I want to replace, bring in Pega, because it's a good decisioning tool," but then like, what do you wanna achieve out of it, right? And what we have seen is the clients who are the most successful have like this vision, "Okay, I want to make sure that," like, for example, you just spoke about where we start off with inbound, we're gonna go with outbound. They have a very specific roadmap across channels, and it's not like, "Hey, I'm gonna push out outbound, I'm gonna push out like this channel later, like, and just focus on call center for now." It's basically, "Yes, my vision is, of course, individualized customer experiences, but this is, what is my roadmap? What is my operating model change?" Because people, you know, what we have seen, I mean, regardless of the size of organization, there is a lot of different objectives of different organizational leaders, where inbound leader will have a different, call center leader will have a different organizational objective, where the, it's the outbound leader, but will have a different lead objective. So, you know, you have to, it has to start from the top. Vision has to come from the top, and make sure that everybody understands that vision, and they work towards it.

- Brilliant, thank you. And Joe, I remember that anecdote as well about when you did turn on the MBAs. I remember you had to turn it off again, just 'cause customer services just couldn't cope with it, with the number of calls, I remember that really well, and just points to the success of that. I just wanted to get back into objectives and KPIs that you were particularly trying to achieve. Can you just talk to us a bit about that?

- Yeah. It depends on the business areas, and we've different objectives, just to Piyush's point, we've different objectives for the different business areas. And so you're trying to align to all of those, while having some sort of a consistent or unifying narrative across them as well. On the sales side, again, when we went live last year, we went live for the first six months with service MBAs only, which I think was really important as well, 'cause it took the sales pressure as such off of us and we were able to learn. But on the sales side, it's standard stuff. It's legates, applications, and then measured through to draw down across each of the different units. The only different one is in our everyday banking business unit, which would be cards, loans, deposit accounts, things like that, we measure things like calls down at the contact center and self-service up as well, to give you a sense of some of the successes and divide 'em out. I talked about data quality, that was quite a good one on the address. And in fact, just on the downstream journey is what it allowed us to do is to bring in more robotics to drive better downstream journeys. And I think that is an iterative process. As we create an MBA, we deploy it, we create the best journey we can, but, you know, very often, we iterate those journeys as we go. On the service side, talking about some of the successes we've had outside of data quality, fraud is obviously big for everyone in financial services, our suspicious transactions line, we had a 15% reduction in calls on that line. And the calls down is really important, but what's more important for us is that we saw self-service going up on the back of it. So the MBA pointed to a particular webpage to help the customer self-serve, and we saw hits to that page going from 500 a day to 19,000 a day. So you had a sustained calls down. So that, that was really cool and important. On the sales side, so maybe to pick a couple, we had one personal lending MBA that we saw a 46% increase in leads and a corresponding 22% increase in draw downs. So clearly, you know, successful, but we've more to do in terms of that journey. Business banking, we, one of the successes we had, we have an on-the-ground sales team for things like asset finance, and we saw almost a hundred percent increase in referrals to that sales team in the small business as a financing space. So really good success there. And, yeah, maybe one more, on mortgages. You know, fixed rate rollers are always something that we need to manage, and we saw over 40% improvement in reduction in churn on fixed rate rollers. So, you know, good, very good successes. But you know, clearly, we have successes and we have our challenges, and we learned from the successes to apply to the challenges.

- And we'll come on to test and learn some of the challenges of it. But, Piyush, you've had lots of experience in telco. Talk to us about some of the, you know, KPIs.

- Our mantra is, "Measure what you treasure," right? You don't over-measure, right, every single time. But it's, again, what we have seen as at the program level, what kind of incremental revenues are generating, because a lot of our clients wanna make sure that they can actually continue to advocate for the expansion in other channels. Because the vision is there, but it takes time to, you know, get that vision, right? So how do you make sure that you are set up for success by measuring the incremental revenues? And, as you said, about, like, the customer satisfaction scores need to be measured, you know, when you're... Depends on where you are. Like, if you're starting with marketing, it's a different measure. For marketing, there will be a promo cost reduction. So again, that's why the vision is so important. So it depend, very depend, much dependent on the roadmap. So if it is marketing, it can be promo cost, because we have seen up to 20% promo cost reduction, if done right, in the first four months of implementations. We have also seen that a cost to save reduces by almost 15 to 20% if you're going into retention, especially call center retention. So we have seen that as well. And then the other measures are around upsell and cross sell rates, how you're improving those.

- Great. "Measure what you treasure," I love that one. That's great. And we've been talking a lot around organizational change. And, Joe, what organizational changes did you have to make within?

- So when we introduced CDH January last year, we had very little changes to make at that point, and we had made changes previously, and talk about those. So last year was really about introducing the decisioning practice to the cross-functional marketing squads, or into the marketing op model. And that was, it was quite a light touch, make sure that, you know, design and copywriting were looped in, and it fit into our overall op model. But so very little to be done for CDH itself. But if I go back to 2018, 2019, when we created the group marketing function, we created agile cross-functional marketing squads. So we aligned to the business areas in what we call value streams, home buying, everyday banking, business banking, wealth and insurance. So we had a squad aligned to each one of those that was cross-functional in nature, and then those were supported by centers of excellence. So analytics would be a center of excellence. So we have analytics people in the squads, and we have analytics people in the centers of excellence, depending on the different needs and requirements. So, and we've had really good success with that, in terms of a robust model. We see opportunities to iterate and improve it. So right now, we're looking at how do we move from being product-focused to being segments-focused. And ultimately, what we wanna do is to get to be customer states-focused, where we're talking about customers in an acquisition state, new to bank customers, or existing customers in a growth state, or existing customers in a potential churn scenario. So, you know, we're still on that journey, and I think we'll see continual changes to the op model that will be brought about by, you know, by customer demand, by focus on financial wellbeing, by the technology, by Gen AI. We will see more changes there, I believe.

- Great. And what I really like about that when we were talking about it is that you first said, "Well, actually, we didn't make any changes." But then you actually said, "Well, we made the changes, you know, a couple of years before." And so often we see that people implement the technology and use the technology to drive the change. It's a really painful experience. I know we've all been there before. So you having made that change and then putting the technology on top of the improved processes, is textbook stuff. And you talked about moving from product-centric to customer stage, and I know, Piyush, we've been talking about that. What have you seen in terms of that move from where there are organizations from that product-centricity to customer?

- I would say it's like, you know, product management is so disciplined, right? So you have to basically translate the product objectives to your customer-centric objectives, like, and that there needs to be, you know, what we have seen, and what we have seen some of, some clients working towards this, especially in Middle East, where they are saying, "Yes, product management teams are good acquisitions, let them do it. But when they become known customers, let us make sure that we treat them as our customer, as an organization, and let's upsell, cross-sell and retain them," right? Instead of like, again, doing the same product-centric marketing when they become known customers. Because you have a lot of data, you have a lot of good data at that point in time, and you don't want to keep hitting them with communications which are not relevant.

- Yeah. And does that resonate with you, Joe, in terms of what you've seen?

- Yeah, definitely. I mean, that's the piece I spoke about at the start, about having, you know, the right orchestration, arbitration, consistency of message, definitely, I mean, all the way through. Because, you know, it's so easy to confuse customers and send conflicting messages, customers that you think are in market but are not in market. So yeah, that definitely resonates with us.

- There's another thing we were talking about in amongst this, just about testing and learning. And, Joe, you mentioned that, you know, some things worked, some things did. How have you managed to do your testing and learning? What's...

- I'll talk to you about what we've done so far, and then where we're going as well. So test and learn for us is, first of all, if I speak specifically about CDH, we created a very bespoke set of reports for CDH that measured CDR on on every MBA that we brought live. So we went live with 26, we have over 200 now. So keeping control of what was happening, in terms of the responsiveness of each of those over time, sales and service, and then using the really good MBAs to help the teams iterate and improve the not-so-good MBAs. Why was one good and why was not, why does one resonate with the customer, another one doesn't? So that's what I would call a kind of quant side of what we did, measure and then iterate. On the qual side, because that doesn't always work, what we did was we saw some of the squads that adopted Pega and MBAs much quicker than others. And we have a weekly all-hands, commercial-focused marketing call, and we brought the leaders of the squads that were leaders in the CDH space onto that to talk about their experiences. And you start to see that sort of human competitive thing come alive, and people going, "Yeah, they're getting better success." You know, when we talk about metrics, revenue up, cost down, NPS up, you know, "They're getting better results across those three. How can I test and learn?" And we saw adoption improve over the course of the last year-and-a-half as a result of that. So that's definitely one. I talked about the breakout, or the subgroups of customer analytics. The fourth group I have is marketing effectiveness, that measure everything. In terms of test and learn, we take those learnings, we push 'em back into the commercial insights team that sit part of the cross-functional marketing squads, and feed that into the squads. But then we also triangulate back to the business. So it's business into squads, but then the analytics team triangulating back to the business is important as well, because not everything is a marketing output, and so you wanna be able to bring those insights to the business in a way... And we see ourselves as, you know, part educators, part providers, where we're teaching the business about, you know, the power of data and analytics, and how they can use and adopt them. So that's the sort of now, that's what we do. In terms of what we're doing more of, I mean, there's a huge suite of out-of-the box tools that come with CDH that we're really only scratching the surface of. We've just deployed a BOE, value finder, scenario planner, all of those tools that we really need to get better and better at. In fact, there's a session this afternoon with one of Mark Davies' guys, Philip Mann, at three o'clock that talks about that Pega op model and how we can adopt that op model to understand, because it can get quite complicated. How do you know which tools to be able to use for test and learn? How do you scenario plan? So there is quite a bit of education in that. And so, you know, we'll do more of that. That's where we're going.

- I mean, that's a great point on change management, by the way. Like that continuous change management, you know, it's a little bit different point from the test and learn, but that's, it's definitely absolutely essential, because you gotta, you can't just like do the change and let it be, because then that's why, I mean, I think a lot of our clients, what we are noticing is that these usage rates are less than 20%. Like, most of our clients have that issue. And so that's why you have to have continuous change management. And then, you know, the test and learn piece, and like, I think, Pega champion challenger capability, really helps in that. And especially ML ops. ML ops, comparing different machine learning models, that helps.

- Yeah. Yeah, transformation is not a one and done.

- Oh, exactly.

- It's not a one and done.

- A series of steps.

- Yeah. Yeah.

- We met... We talked about, last night, we were talking about machine learning operations, and the fact that, you know, you've got to learn how to manage those models. Could you just talk about, a bit about that for us, Piyush?

- In terms of Bank of Ireland?

- Yeah.

- There's smarter people than me that can talk about ML ops, for sure. So, for us, we look at, we look at that in terms of predictive models and adaptive models. We have a robust framework. We use TruEra to help us manage all of our models and align to a second-line risk in the bank, and ultimately, third-line risk, to make sure that, you know, we have that robust framework and we can explain what we're doing. The adaptive, a little bit differently, given the support of Pega and that. But again, a strong framework on both. I think the bigger issue for us, not so much what we do, but what's coming next. What will Gen AI do in terms of the volume of models that we'll have? Will the frameworks be scalable? Will it be robust enough? Will there be this proliferation of just loads of stuff going on that becomes unwieldy and unmanageable, which we can't afford for it to be, because we've got a good, strong foundation now that we just need to continue to work. So I think there's, there's just lots to be written on that we don't know yet.

- I see.

- I mean, from telco's context, I mean, like, model decay is a big problem, right? And then how do you really make sure that you manage the adapter models, your predictive models? That operational cycle is something which is absolutely essential to have an operating model around as well, and that's where, you know, Pega can really help. They have builtin capabilities to do that. And as I said, it's just scratching the surface, you know?

- And I know we were talking a bit more about machine learning operations and, you know, the whole organizational change around that. Do you wanna talk about a bit about that?

- Yeah. I mean, organizational change, that's like when Gen AI is coming in, you know, there are gonna be even more models, you know. If you wanna do hyper-personalization, you're gonna have like hundreds and, you know, thousands of offers, you're gonna have models, and then you are really doing like outbound, and then you're doing, generated AI and content, and then you're learning from different content pieces, then they're gonna be hundreds and thousands of models. And you know, already some of our clients have two thousand models, right, and what is, I think, an issue with them is like how to effectively manage them through ML ops. And that's where I think, you know, we, we always suggest and we work with them, and Pega and other products, like how do you make sure that the ML lops is very tight and it's like we are not losing sight of it?

- I think there's a people component to this as well, as you're talking, Piyush. When we look at customer analytics in the bank, we talk about working across three areas all at the same time, so operationally, transformationally, and experimentation as well. And you know, ops is going to be 85% of what we do. Transformation.... And that's gonna be, you know, what we're doing now for this year, transformation is what we're doing now for next year and the year after, and then experimentation is what we're doing now for three to five years' time. And we've bought, in the Pega space, what we call an orphan sand pit, a fully-functioning test bed, where we can get our people to go in and test, and try, and, essentially, try and go in and learn about these sort of areas, how to manage models, what's coming, how do you try and break the technology in a really safe environment, and understand what the impact on customer and colleague is going to be. So there's loads of different dimensions to it. I think you gotta be thinking ahead all the time.

- Absolutely.

- Oh, yeah. And then data is ever evolving, right? So you gotta make sure that, you know, for ML ops, you're gonna make sure that it's, yes, data pipelines are met, and you have gotta strive for good data. But you're never gonna get to Nirvana. I mean, I've never seen that happen, but you've gotta be, you know, good enough. You have to be good enough, and then just make sure that you're working through it and measuring, and then making sure that you're testing and learning, and going towards your vision.

- [Nicholas] Measure what you treasure, really.

- Yeah. Achieving your vision, right? So it's like, it's a series of steps, and you would have to pivot, a lot of times you would have to pivot. So again, that's where the assets and tools, which, you know, which we have developed over time, it's like, it's hyper-personalization, but it's also hyper-customization of the toolkit, right, of this entire transformation for every single client. Every single client has a different MarTech stack, and there are like 10,000 tools available in market for MarTech. Every single client will have a different combination of it. So how do you make sure that there is a system of intelligence, which is Pega, and it's working seamlessly with hundreds of other tools in the organization? Because it has to work that way, otherwise, the vision of individualized customer journey is just a vision.

- Yeah, absolutely. Has anybody got any questions, right, just that they would like to ask? Fire away. Can we get a mic, or come down? Sorry, where's that? There we go.

- [Mahesh] Hi. This is Mahesh Maskar. The question I have is a Pega out-of-the-box offers the kind of arbitration in terms of the next best of personalization. I was just trying to understand, when you refer to hyper-personalization, you know, what do you mean by that?

- For when you, when you meet... In the offers context, for example, if you, and I'll just give an example of what we have seen in a lot of, in existing clients, that the issue which we are facing is like the offers from in the inbound are like configured, right, all configured already, because in the CDH they're pre-configured. But then there are outbound campaigns going out, and sometimes, the outbound and inbound are not talking to each other. So you are getting an offer in outbound, and there were several complaints on that as well. And then the outbound, you're sending a customer a device upgrade offer of a thousand dollars, whereas in inbound, when the customer actually calls in, they're getting a $600 of. At the very basic level, just make sure that the offers are combined. And that, in fact, is a lot of, it takes a lot of work. And then the next next step is gonna be like content hyper-personalization. When I'm actually giving the offer, I know what your nano-segment is, what you are, and then I know what your offer I'm giving, and I'm giving you a specific content, not just like, I would say, a generic content about a phone, right? I can give you a specific picture, which, basically, elicits a better response from you.

- [Mahesh] So basically, you just try and do the, you know, inbound, outbound, both combined kind of-

- Yeah, digital as well, yes.

- [Mahesh] You know, kind of personalization.

- Yes. All of them are combined, not just like, the decisions are not just taking place on the,

- For one side.

- on one side of the house.

- Thank you.

- Perfect.

- [Audience Member] Hi there. Yeah, sorry, right in the front. I'm totally gonna get you now. So it was interesting that you mentioned having some of these kind of transformative cycles in process across sales, service and marketing. And thank you for outlining that. As the teams have gone through that and are now experiencing some of the changes that can happen, are you finding that your teams, that the teams across those functions are maybe changing, maybe some of their definition of what they treasure? And is it now beginning to change what they now expect of analytics and what they expect of their systems?

- Yes. Yes. And it depends on... So, sort of two parts of the answer. When you talk about colleagues, we have our marketing function, 140 people, and we have a maturity curve that that function is on. And you'll see the commercial analytics, or the customer analytics one should be more mature, let's say, than traditional marketing. And so we've got a journey to go there. But then, even across the product businesses, we have different levels of maturity there, and different levels of understanding. In terms of the measure, the measures are typically set, I mean, unless we're introducing something new. I mean, we've got P&L type measures, we've got service measures, we've got customer set measures that are more or less set. So they don't change so much. It's more the maturity of either the marketing component or the business area as to how quick they are to adopt what we can do for them. Our everyday banking team, for example, is led by the ex-CMO. So he's just, he just gets it quicker. And you can see it, that team maturing quicker, in terms of their demand on us as a provider of analytics or MBAs, or whatever it is for them. So it just, it just depends.

- [Audience Member] Oops, sorry. There was a little bit of feedback there, so I'll hold it here. So with your marketing and services systems, but let's focus on marketing, there will obviously be capabilities for testing and optimization and analytics and personalization and connections to content out there at the edge, you know, on those frontline marketing systems. And then you've got some of those capabilities with the decision hub. How do you determine where you'll make decisions about content for hyper-personalization? And then how do you align what you're doing with the decision hub with those systems at the front end, whether it's Adobe, Salesforce, whatever?

- So we're not using Adobe Salesforce, and we're live with two channels in one, authenticated today. But we do have, I talked about the centers of excellence, and one of those is our brand team. And so we worked quite well with them as a center of excellence and as members of the cross-functional marketing team, to drive consistency of message. And, I mean, that's, that works well. And we do AB testing, and we do all sorts of, I think some of the out-of-the-box stuff that's come with CDH, I think will become better for us. So that's, that's working. What I'm more interested in is where it's going. And again, back to AI and our experimentation space, we're challenging, I'm challenging my team to say, "Pega CDH makes a decision. That's a descriptive decision. Can you feed that into, let's say, DALI, for example, and Copy.ai to create, you know, a unique imagery and unique copy for that particular decision?" There is a peer Australian bank of ours that's doing that quite well, down to micro segments of 50 customers. So I think there's a challenge to get it working today and get the consistency, but I think as we introduce more channels, outbound digital, I think we'll see that that's not so much of a challenge for us. For us, it's really about what's coming tomorrow and how AI will form a part of that. But again, back to experimentation, test and learn, and see where it goes.

- And what we have seen with different clients is that they treat Pega as a systems of intelligence. So essentially, the offer, the treatment, is decided at Pega level, and then the content piece is for the content, you know, like the and other phase places. Yes. So that's what we have also seen, and it's working pretty well as well.

- Yeah.

- Hi. I manage a team. We disseminate analytics based on our CDH that's installed today, and we're a large US bank. With respect to what you mentioned earlier about going from product to segment to the customer analytics, I'll admit, I still struggle with this. We produce a lot of broad reports that show, "Here's what happened..." Yeah, I'll just take branch, for instance, for branch bankers, "Here's what happened in a day and here's the average rank across all, you know, all interactions for the day." And I will get one product team that'll come to me and say, "Why am I not number one? And how can I be number one?" I started by showing, "Hey, of the people that actually fulfilled at the bottom of the funnel, they weren't near the top. But when they didn't, they weren't. And look what else they did. You know, they took an account management for their deposit relationship, instead of this lending product." But I was wondering, do you have any advice to help? You know, maybe I should stop producing broad reports where they can dig in and ask questions to attack me. "There's just no data on that," you know. But do you have any advice to help us along that journey? 'Cause we all believe, you know, the customer-centric story is what needs to be ultimately disseminated.

- I think so. I'll go first. So we, we're on that journey, right? And the easiest thing to do is stick with product. And the product guys will always wanna stick with product, for the reasons you're calling out, and will never want to change from that. We talked about a compelling narrative top-down. We've had new a CEO since last November. We did a strategy refresh in March this year. One of the strategic pillars is stronger relationships, so now we have a top-down thing that we can point to and say, "Top table, stronger relationships," because that then aligns to that move from product. 'Cause when you think about it in the bank context, we're thinking about it in terms of how deep the relationship is with us, how many products the customer has. Are we thinking about those through the lens of financial wellbeing? So that's stronger relationships OKR with the associated KPIs is really important, because it's top-down, it's reportable back to the exec team, reportable back to the board. So there's a little bit of arm-twisting that will go on with that, to try and remove the behavior you talked about. But I don't expect it to be easy. I mean, we've started the move from product to segments, and the people that we expected to be most supportive are now going, "Oh, I'm not really sure," you know? So I think it'll take resilience and it'll take time, but I do believe, with a financial wellbeing lens for us banks, I think that we need to move to that place. But I think it'll be hard, to the point you're making.

- Yeah. It's the same thing with telcos. I mean like, you know, they always come, and I think there are a couple of telco clients here, it's like they always come and like, "Why is my telco insurance not selling, like, you know, add on insurance not selling? And can you just put it on top?" and like, "Hey, it's not selling 'cause they're not getting enough accepts," right? So yeah, it is a cultural change and it's, yeah, it's a little bit of a journey there. There's no easy way out here.

- Got one more? Thank you. We got one more in the back.

- [Audience Member] Hi, it's me. So, hello. I manage a team, multidisciplinary team for a telco in Belgium, and one of the disciplines is also analytics, so I'm lucky enough to have data scientists and data engineers within my team. And I'm also lucky enough that I work for a company where we already have a longstanding tradition of data-driven test and learn data science models, propensity modeling, customer lifetime value modeling, and so on, and even before we had Pega CDH, and so now we're looking into how can we improve our predictive models with the adaptive modeling capabilities. And I'm wondering if you have any advice on this traditional data science approach towards the adaptive, because we actually, not only marketing teams need to change, in our case, it's also our data analytics teams that kind of have to rethink their way of working. So do you have any advice for that particular set of people?

- Advice in terms of how to improve that journey from predictive to adaptive, and the test and learn? We found... It was interesting, when we went live, I didn't realize it at the time of going live, but the data science team were much more fearful of adaptive models, because they didn't understand their role in CDH, and there was a job security fear that was bubbling under the surface that we had to to surface. And you surface that with transparency and education, I think. So we've just completed bringing our data science, the data scientists to the Pega accreditation for data science, which is same name, but two very different things. I mean, you know, our data scientists are coming, you know, with double Masters and PhDs and, you know, and so there is that, education needs to happen gradually over time, but their role is really, really important. And so showing them what their role is, the outputs of the predictive models, or the predictors that come out and how they prime the adaptive models for success, and where you can see back to the concept of bespoke reporting and the different successes, different CDRs for the different MBAs, and being able to say, you know, it's because of that journey through, you know, data to analytics to Pega, to routes to market, and just bringing the more holistic picture to them, has helped. Again, like pretty much everything I've said, there's still a journey to go on. We're still... We're there in terms of their understanding of their role, but I don't think we're using the platform to its fullest, in terms of the capability that they have and how they can challenge the, or stretch the boundaries of the adaptive models. So maybe we learn from you as you go through your journey as well.

- So, in some of our clients, in some of our telco clients, actually, data science teams spearheaded the transformation to CDH. The reason being that predictive models, like, for example, you're creating churn models, like what it will lead to. And traditionally, it'll go to the marketing team, they'll generate a list, and by that time, like seven days have already passed, and then what are you retaining, right? So a lot of people you have lost. So the data science for CDH, for this, I think, to bring them to the journey, it's like predictive models absolutely are not going away, because adapter model, what that is doing is basically, it's basically telling you a lot of times, "Hey, who is more likely to accept this part of the journey, this treatment, this cycle?" But when it comes to adaptive, when it comes to the the robust, like our existing traditional models, so about the churn models, about the propensity to the product, that all can, that also is absolutely important data for us to target the right customer. So it's, that doesn't go away. It just empowers the data science team, in fact.

- Okay, guys, and I've got two minutes, we're gonna wrap up. Do you wanna say any last-

- Yeah. As you just proved to me, that what CDH does in fact is it provides a stage for the data scientists to bring their expertise to market quicker. You're right. You're a hundred percent right. Yeah.

- Brilliant. Okay. Guys, thank you so much for the questions. They were amazing. It's really good to get an audience who, you've obviously been through the journey and you understand where they're going. Look, I know Joe's, you're also talking tomorrow as well, aren't you? So if you want to catch Joe again, just, I'm not quite sure what session you're doing, but looking at, he's got tremendous amounts of knowledge in this, so I'd recommend you catch up with him. If you've got any questions, please stay behind. We'd be happy to answer any questions. And just, look, it's... The guys have really just talked, been able to talk to us about how you really embrace that hyper-personalization journey. I'm gonna go away with "Measure what you treasure" today. So that's, that's gonna stay with me, again. Thank you so much for coming. I hope you enjoyed that. Thank you.