PegaWorld 2025: Customer Obsessed: Successes and Insights from National Australia Bank's Customer Brain Journey

PegaWorld 2025: Customer-Obsessed – Successes and Insights from National Australia Bank

Hello. Hello, everyone. Thank you very much for joining. Um, we're going to kick off now for, um, thank you all for joining us. My name is Jessica Cuthbertson and this is Lisa Marchant. We are part of National Australia Bank. I look after analytics and customer decisioning. And Lisa, I'd like to say is the brains behind our customer brain. So any curly questions will get thrown at her today. Um, and we just wanted to share an update on our journey.

Um, kind of remind you of what our strategy was for the customer brain, where we are from a delivery perspective, and then share some use cases that we've taken to market and some of the results. So without further ado, for those of you who don't know. National Australia Bank we are one of the big four out in Australia. We are Australia's largest business lender. Um, and we've got about 10 million customers.

And we've just refreshed our strategy, really focused on becoming the most customer centric company in Australia and New Zealand. And our new CEO purposely said company, not bank. There was actually a typo on this slide that I had to fix earlier today because it said Bank. And I was like, if our CEO sees that we're going to be in big, big trouble. So. Massive ambition not just to compete in the banking sector, but to be the most customer centric company in all of Australia. So big ambition.

Um, but what that means for us is our Customer brain program that we started about three years ago is absolutely central to that strategy. Um, because in order to be customer centric, when you're talking about 10 million customers across your retail, business, banking, corporate and institutional bank. Um, what does customer centricity mean? Like most banks have the same products. Um, where do we want to compete? We want to compete on being personalized.

We want to compete on using our data to make sure that we are actually being relevant to customers in their situation, that we're demonstrating memory, that we're being responsive to their needs. Um, sounds really, really simple as a statement, but we're going to take you on the journey of how we're creating the foundations to build it.

And I get asked often, like, how do you how do you metricate that? What are you looking out for in your Customer brain program? Um, and there's two kind of key things that we look at. The first one is how many interactions are we personalizing? We've got lots of customers who repeatedly come into our digital ecosystem. They're they're coming to our estate.

Um, if you don't have enough personalized interactions for them, then you're not really making a decision, right? You're just serving the least worst conversation to them. And so for us, we want to have enough personalized interactions in our ecosystem so that customers do get a a very bespoke experience. And then we also look at uplift and engagement. And it's a really good indicator for our things working customers that are coming in. Are they engaging with our content.

Are they getting bored? Are they getting banner blindness? Or are we keeping them on our estate and engaged with their services and their banking products? So that's a little bit about our strategy. You can see that. Thank you for the slides. Very welcome. So we wanted to talk a little bit about our journey. Um, when we started building out the Customer brain, I hear a lot like how do you get started? Because there's a lot that you could go after.

And we made some deliberate delivery choices to try to have the most impact for our customer base, and we thought about our program and kind of four key almost functions. The first one is do we have data about our customers? I think we are so blessed to have a lot of information about our customers, and so we need to use that for good.

If customers are asking us for information, they're trying to interact with our products or services and we're not responding to them with that data, then it means we've missed an opportunity. We think about then actually building actions. What do we have to say to our customers? Let's make sure that we've got a good range of things to say across service, sales and engagement. We think about touchpoints because you've got to be where your customers are.

It can't just be in your channel of choice. You've got to respond to where the customers are interacting. And then one of the key things that we talk a lot about, which is often left out in program delivery, is kind of business adoption. This definitely isn't just a tech change. It's very much around how do we get the business on board with what we're doing? So they see it as absolutely critical to their strategy.

So I'm just going to quickly talk about what we did in our first year, which is really about how do we earn the right. And some, I think, excitement within the business around what we're trying to do and build. So I'll talk very briefly about this, because some of you may have heard this last year, but I think it's important for those that haven't heard it to to hear a little bit about our story and where we started.

We actually from a business adoption perspective, we just chose two areas to focus on in the first instance. So we had, um, a couple of really friendly stakeholders in our home lending space and also in our digital space. But the guys that ran our mobile app, um, and they were really keen to build use cases with us. Um, it was great because from a mobile app perspective, we get lower, we get 100 million, um, customers a month coming to our mobile app.

So we wanted to make sure that we had good presence there. And then, um, from a home lending perspective, obviously that's, um, a core part of the revenue driving aspect of our business. So really a good place for us to focus as well.

So we started small, um, we actually were able to leverage a lot of channels very quickly because we actually integrated with our existing marketing automation system that, from a batch perspective, was already connected to a lot of channels, which meant we could actually reach customers across all of our channels very quickly in in the first instance, in the first year, all of those use cases around digital and home lending, uh, enabled us to generate 50, 50 next best actions.

And we ingested sort of 300 data signals. So a lot of that first year for us as well was actually just about getting the infrastructure set up, setting up our data patterns, which then has allowed us to scale in the next couple of years in terms of then what? Year two has looked like for us. Um, the good news was that the first year was was pretty successful with what we did in the home lending and digital space, and therefore we had a lot of other business units actually want to jump on board.

Um, so we were actually really able to extend the breadth of what we've been doing, which is why you see this big jump here from 10 to 70% in terms of business adoption. We had our business banking jump on board. We had our, um, unsecured lending teams jump on board our deposits. Teams jump on board because they'd actually seen the success we were able to generate off the back of our sort of start, small approach. Um, the big sort of shift we started to make from a channel integration perspective was start to move our channels from an from an inbound perspective to batch to to real time. So it's a real focus on actually setting up real time with our mobile ecosystem and making sure that when a customer logged on to our mobile application in real time, we're actually making a call to Pega and generating the next best action back in a series of spaces that's actually available within our mobile app.

And I think that was a bit of a turning point for some of the kind of technology and marketing teams where they started to really understand the power of customer decisioning, because you think about that mindset from moving from campaigns, outbound push our time when we want to communicate to a customer, and when you start talking to them about real time, they're like, but why do we need to do that? What's why do we need to uplift that channel? And that was really because they kind of got the power of a customer coming in, instead of us having to do outreach to them.

If customers are already on your ecosystem, then you should be leveraging that interaction rather than trying to outbound push something to them. So I think that was a turning point for a lot of our stakeholders. 100%. And that sort of really leads us into to this is now our third year. And I think the big difference that we're starting to see now from a business perspective is that the business is really owning and investing in the capability that we're actually building out.

So we've got a much we've got a front door, um, where the business comes to us. And I think at last count, there are around 50 different engagements that are actually running and sitting on the backlog. So we're we've gone from, you know, I guess, a project driven approach in terms of our implementation to the business actually leading the investment and the development in the capability that we're actually trying to trying to build. So that's super exciting.

Um, from a touchpoint perspective, we've continued to add more touchpoints. We've continued to onboard new areas of the business. And actually, one of the big things we've done in the last 12 months is that we've actually set up a whole new application, um, to support our white label business as well. Um, it was a really. So we're sort of running through the the multi-app app approach now. So we've got an application for our core Red star business and an application for our white label business.

And it was actually really interesting journey for us to go on because two years before a lot of sort of my team and peers were doing this for the first time. And actually, you know, last year we actually started to do it now, um, for, for, for a new business, but with a whole host, more knowledge around how we go about doing this and how we actually set it up. Um, so we've, um, now got well, expectations are sort of by the end of the year, we're going to have around 400 actions.

So we're really starting exponential growth now in terms of the the actions that we're building. And um, we've got a significant number of data signals that are available now available to us to build. Okay. So um, that was a little bit around the roadmap. And as Lisa said, um, you know, three years in first year Sheer foundations. Um, but we've built really quickly and we've learned some things along the way.

So we wanted to share, I guess, some of the watch outs for those of you who might be new in the implementation, or some of you who have been through it might have, um, some similar learnings. Um, but the first one that um, we talked about is, um, hyper personalization. Everybody talks about hyper personalization. That does not mean choose your own adventure. It does not mean that you allow chaos into your system. So instead of hyper personalized I advocate for hyper standardize.

Um, oh, go ahead. It was a really big learning for us because, um, we it's really about how do you set up the right frameworks and structures and principles around how do you actually build an action and how do you build an action? I guess in terms of what we're trying to do at NAB. So we've now established and built a playbook out for ourselves in terms of how we build.

But yeah, we could have been in the situation where everyone got really excited about the flexibility in some of the system, you could have got 170 contact policies. But actually what we've done for certain, you know, how we I guess we've we've written and developed certain rules of engagement around how we actually build.

Um, and that's been really important, I think, for us, as we extend across domains, as we bring new domains within the business onto the brain, it's like, this is how we build, this is how we think about building an action. And this a standard approach that we run across the business. Yeah. And if you think about that 170 contact policies, if you think about the way that you would normally build campaigns, you just start with a brief. Right. And then you build to that brief.

And so what we've moved to, we talked a little bit about last year was that kind of portfolio of actions. Um, and you'll get economies of scale and you'll get speed if you think about different kind of subject areas in your business and try to standardize wherever possible, some of that logic up front, and then you're focused, you're mainly focused on developing differentiation in the types of things that you're talking to customers about.

So you can see how you get to 170 different contact policies if you just follow that old school campaign by campaign. So it's a very different way of setting up your ecosystem. I think the other kind of thing that we we really thought about was, um, we spent a lot of time debating which customers to include in a campaign and which customers to exclude in a campaign, and that could be lots and lots of debate about around your organization.

So we're so focused on who we're talking to and how do we optimize who we're talking to, that sometimes you lose sight of who you're not talking to. So we actually did a lot of analysis on our book around, um, who are we speaking with? Great. What does that cohort look like? But who are we not talking to? When are we leaving some of our customer segments behind because they're not in the market for a home loan, or they're maybe not digitally active, so we don't see them as much in our ecosystem.

We've got a really nice example of this in our home lending space. So when we first did a lot of our initial build, a lot of our actions were focused either on the sales, the onboarding or the retention. Um, experience within sort of, I guess, that home loan journey. And, and what we discovered is that for customers that had a home loan with us and were just sitting paying down the home loan, only 20% of those customers were being reached by an action.

So we took a step back and said, okay, well, what are the different things that we could actually be talking to these customers about? So we delivered actions that focused on some home lending milestones. We developed actions that actually ensured we we promoted the features of the actual product that the customers had and sort of really reinforced the value proposition of their product. We also did work where um, we actually, um, mind goes blank. Um, there was um, also work we did in the home lending space around, um, insights that um, we provided around, say, home loan growth in your local area. So a real range of different types of actions that we provided to our home lending customers. Um, we saw coverage go from 20% of these customers getting an action to 75% of these customers getting an action.

And not only that, the analysis that we're seeing off the back of what we've done there has not only sort of driven some strong engagement rates for these actions, but we're also seeing some NPS uplifts as well, which is awesome. Yeah. And I think that, um, you can relate that to any portfolio. So we talk about home lending a lot because as Lisa said, in the first year, we worked in like hand in glove with that business partner.

Um, but you take that to every single portfolio and every single segment that you're working with.

If you think about only engaging with a customer at the pointy end of the sale at the bottom of the funnel, you're missing out on a huge opportunity to keep them engaged and keep them sticky with your brand throughout their entire life cycle, so that 75% coverage is great because those home loan customers, you know, while they're not thinking about their home loan every single day, they're getting something that's relevant to them. And it doesn't have to be related to that product.

Often it's not. It's just a nice service message for them. And the other thing that we want to kind of watch out is always on. We always talk about always on in the customer brain. It does not mean that you set and forget your actions. And that was a big lesson for us because oh go ahead. We saw go ahead.

I was going to say like one of the things that we really started to notice this switch on as we've actually progressed in our journey has actually been we've gone from, I guess, really aggressively building actions to now we have this whole series and ecosystem of actions that we actually need to continually manage and make sure that they're working. And so, um, one of the things that we obviously look at is, is engagement.

Um, Jess mentioned sort of before, and when we were looking back in January, we were looking at some numbers and we actually started to see engagement start to dip and we were like, oh, what's going on here? Um, and we started to get in and underneath the numbers and we found a couple of things. The first thing we found, we had an action that customers had actually been, I think, very overly exposed to from an oppression perspective.

It was and it existed within our mobile ecosystem, which is where we get a lot of traffic. Um, and effectively, what had started to happen over time is that we were communicating or serving it so much that actually customers had just become bored of it. So we did some work to sort of pull back our contact policies and how often we were actually presenting that action to a customer, and that actually saw us drive uplift in terms of that engagement rate.

Um, in addition, the other thing we noticed was, um, we found a relationship between sort of engagement and how much new or how much change you had actually put into the system as well. So what we were seeing is in months where we weren't putting a lot of new actions into the system, we weren't getting the same levels of of growth and maintenance of our engagement rates there as well.

So we've really sort of tried to balance making sure that we put something new in, as well as looking at what we've got. And something new doesn't have to be new action. It could be actually within an existing action. You're putting additional treatments in, and you're really thinking about how you create variation and change. And the other point is we've got 2000 different adaptive models running at any given time.

So we've started using that as a tool in our arsenal around looking at your adaptive cutoffs and making conscious decisions where if it's below a certain cutoff, just choosing not to show it because it's worth showing something that's going to get, as I said before, that banner blindness than just not showing anything at all.

So, um, yeah, um, picking up on that creative treatment and variability that Lisa had mentioned before, The fourth kind of thing to really think about is, um, creative variation matters. And in kind of old school thinking, you think a b testing how many times a week do you get asked about a B testing? I reckon every week. Every week. Um, so the AB testing is, well, which one of my actions is winning or which one of my treatments is winning? And it's actually a different mindset shift.

Nobody wins or loses. You're looking about how do you optimize and how do you put more variability into the system. I've got a good kind of example of where we saw that. Yeah, there's a really nice example where we were running, um, a sales message for one of our deposit products, and we had two creatives. One was reasonably generic and one was actually associated with, hey, you're saving for a holiday.

Um, and what we found when we initially released these treatments, the generic message, although both worked, the generic message had always outperformed. But actually that started to dip off over time and then all of a sudden in Jan, Feb and March of this year, we saw the message around savings are saving for a holiday. Actually, absolutely go through the roof.

I'm like, well, what's going on here? Um, and what we found, um, is actually Jan, Feb, March is a time in Australia where people book holidays and they're thinking about saving for holidays. So we actually had just we had a treatment readily available that was contextually relevant for our customers at that point in time. And I guess in the old AB testing world, we launched this back in, I think it was September.

We would have turned off the savings like the, the holiday creative, um, in the old world. So it's just a really good example of actually, you know, you need a good breadth of treatments there to capture contextually relevant moments, um, when they arise. And you can't get away from all the chat around GenAI like we're just really working with our marketers around. Get all the different creative variants in, let's get them in, let's test and let's see who they resonate with.

And if they resonate with a small subset of customers for a period of time throughout the year, that's also okay. So those were our kind of four key lessons. Usually this is when I see all of the cameras come out. And because we're going to share some of our use cases. So this is probably the most exciting bit because this is what actually gets in front of our customers. And this is what makes it real for them. So we've got four of the, I think, four that we want to share with you.

And as I mentioned before, you'll always hear me talk about service, sales and engagement. We've got 60% of our actions that are just purely service based, 40% that are sales based. And we really hold true to that balance, because it's not often that you're in the market for a home loan. It's not often that you're looking to switch your credit card. And so we just want to make sure that the customers that we have, we're nurturing them. We're kind of customer obsessed about them.

And so we've got a few that we wanted to share for you. Thanks, Jess. It was very hard to choose which ones to share today, but the first one is one of my favorites. So, um, we'd identified through one of our reports that we had some of our customers who weren't fully utilizing the features of their product. So we're like, hey, why don't we remind them of the value proposition of the product and see what happens? So very, very simple message that we placed in our digital ecosystem.

And the results of this one really speak for themselves. Um, that 30% customer action effectively means that it's not only customers engaged with the message, but 30% of customers actually did something off the back of this message. Um, and there were two things that they did.

In the first instance, some customers closed their accounts, and that was totally okay, because actually, this was a nudge that reminded them that possibly there was a better product for them or this is something that they weren't using, so perhaps they didn't need it. But we also saw a very large population of customers start to use their product. So just reminding customers the value proposition actually provided some great success for for this use case.

When you think about what 1% response rates, you've got 30% customers actually going on to take the action. We were really happy with that as a result. Yeah, it's I've been around 1 to 1 for a while and I haven't seen one like that. Um, this other one is one of my favorites actually.

So the the business problem that we were trying to solve here is how could we actually keep our customers up to date when they make a transaction dispute? So, um, I guess pre implementation of, of this use case, what would happen is customers would dispute a transaction and they would have very little visibility around where that transaction dispute was at and and what the status of it was.

And a lot of times they'd be calling our call center or popping into a branch to find out what was going on. What we did is we built seven actions to actually manage customers through the, um, I guess, transaction dispute journey. So what we were doing here is once a customer had submitted a dispute, we actually had an action that said, hey, we've got it. This is your reference number. We're on it. We would do timely reminders around, hey, you know, you know, seven days, we're still on it.

We're, you know, we haven't forgotten about you. Um, if we needed the customer to provide more information, we would use and provide a nudge in our digital ecosystem to get them to provide more information. And then we would also, um, follow up with a, hey, you're it's resolved, you've been refunded. Um, and then close out as well. And these actions, we've got 55% engagement in terms of these. So customers love them. It's helped reduce calls into our ecosystem as well.

And so we think it's just a really good use case around where we're actually effectively managing, um, our customers through what can be a little bit of a scary experience for them too. And I don't know about you, but I'm never calling the call center. So this is like huge for the customers because they just don't have to follow up with us and go, where are you? Where's my money? What do I have to do next? Cool.

So, um, the next one and again, this is linked to the first where we're, we're reminding customers around, um, we're not just, uh, talking to customers about our sales offers, but we're reminding them about the value proposition of their product. And and this one up here is, is actually, um, talking to our customers in the digital ecosystem about their offset accounts. Um, and we actually started to take these conversations into our banker outbound ecosystem as well.

Um, historically, our banker ecosystem had been very, um, home loan retention and home loan sales focused. And what we've started to do is where we're seeing success in some of our actions in the digital space, actually putting those into banker conversations as well. And our banker conversations are becoming more rounded and actually customer relationship focused as well. So we're sending different types of leads to our bankers around product usage about different products.

So one of the things we're really proud of in this space is we've been able to scale the leads that we're delivering our outbound bankers through the Customer brain. But not only that, at the same time we've been able to increase the engagement. So I like this one because you've got growth across both the volume that you're sending through. But how much more customers are actually resonating with conversations that are about the whole of banking relationship.

And yeah, and what I love about this one at least is what you mean about that whole of relationship. Conversation is an offset account is kind of linked to your mortgage. And so if you've got a mortgage and you've got savings and an offset account, it actually counteracts the interest that you're paying against your home loan. So it's a really great benefit for a customer. And the conversation for the banker is so easy.

Hey, I see that you've got money sitting in an account that's not linked to your home loan. How can I transfer that for you? It's going to help save you money. And what we found is a lot of customers are like, oh, but wait a minute. I have transaction accounts in three other banks. I'm going to move those over to you so I can get the full value from my relationship. So it's a really nice conversation for the bankers to have. And that's really what we try to do in that channel.

Make it as simple and easy for them and really bring through that. As you said, that relationship banking. Cool. And the last one I mentioned before that we're doing a lot around milestones. Um, and what we've done in the milestone. Milestone. I didn't think that was that hard to say. What we've done in the milestone space are two different types of actions. So we've got an action associated with, um, NAB celebrating the anniversary of a home loan. Customers home loan with us. So hey, Thanks.

Another year with us on your home loan. Thank you very much. But then what we've also been doing is we've been celebrating milestones for customers on their home loan. So say, hey, you're 25% paid down, your 50% pay down your 75% pay down. Both actions work really well. They're among our top performing actions. However, what's really interesting is the action associated with the well done.

Your 25, 50, 75% pay down, which is about something a customer has done, outperforms the other, which is about the relationship with NAB by 50%. It's kind of common sense. It's about something the customer's done, and we're celebrating something that that the customer has done.

And I think that's just really interesting as we think about how we grow out, our portfolio of actions as well, knowing that when you're building something about customer and their behavior and celebrating it, it works really well. Yeah. This is the next stage of Happy Birthday. We're moved on from Happy Birthday.

For those of you who know me and we're moving into milestones, and I got one recently and it was around my home loan and I have an offset account, and it showed me how much I had saved over the year. And it was amazing. And you know, I'm a sucker for confetti. So I said, it said like play animation.

And I played it and it had all the confetti on my screen, like, how do you measure that? That to me is really around just naturally celebrating customers and having a little bit of fun with it, particularly in your digital ecosystem. And like Lisa said, when it's talking about me, it's going to outperform because I love celebrating myself. I don't know what that says about me, Lisa. I think we might leave that there.

Um, so, um, I think just to to tie this all together, I think what we're really starting to observe now with the scale that we're starting or that we have built is the customer experience for NAB's customers is really starting to shift. So when I was prepping for for this presentation, I actually went back and I had a look at what would a customer communication experience look like pre implementation of NAB's Customer brain. What does it look like now.

And so if we go pre brain what we did we did a lot of campaigns usually very email heavy. You might get a banker call once every six months. And we communicated to you at very regular and set intervals. Um what we're seeing now I think is a much more fluid communication approach where we're actually reacting to what customers are doing, and they're getting a lot more, and they're getting a lot more connectivity between those sales service and engagement messages.

So it really is a big shift for us that we can actually start to see between pre and post. Excellent. So that was all we were going to cover today. We'd like to open the floor to questions. Please come up to the mic if you do. If not thank you very much for hearing our story and good luck on your, decisioning journey. The brave first person. Hello, my name is Rakesh. I'm from citizens Bank. We are in the same journey. So I just wanted to understand. You mentioned an interesting point.

You said like, you know, almost around 60% if I got the number right. What? We focus on service messages, right? And the rest of it kind of is divided elsewhere on the sales and other side. How do you deal with your business when the marketing teams, when they come to you asking that we need to push these messages, we want to get more drive, more sales. It's the. Number. One question. Number one question I get asked. Um, there's a there's a few ways to solve that.

If you're truly data driven in building out your contact strategies and you're looking, the first thing that we do is we look in our data to try to find the best sales opportunities. Right. And the best sales opportunities aren't a volume game. It's very particular. You can see a customer who is doing something and you're responding to it. So that's first and foremost.

We hoover up all the trigger based communications and we go back to the business and we say, we're going to build you a series of not one campaign once a quarter to get 100,000 credit card mailers. We're going to build you a series of always on actions, which means you're going to get the sales throughout the whole year, and it's going to be relevant. And actually the ROI on it is going to be three times five times the ROI of a normal spray campaign. So that's the first conversation.

So you don't ignore their need because they do have a need. And customers do have needs for products. And we want to make sure that if a customer wants to buy a product for us, we're not saying no. Right. So then when you have that conversation, the second part of the conversation is then okay, but we actually don't have enough to say to our customers about sales because the sales lifecycle is so low for a customer.

So if you want to keep your customer engaged, and we all know how much more costly it is to acquire new customers than it is to retain your own, you can have a retention conversation with them, and a retention conversation isn't waiting until a customer is about to leave you. A retention conversation is engaging them throughout their life cycle. So two parts, two part answer to that. Make sure you meet the needs of the sales, but in a more targeted and direct way.

Improve the ROI and then turn the retention conversation into an engagement conversation. Thank you. All right. So good question. Of course you can ask a question. Yes. You've obviously done CDH many times. So you've learned lots of lessons and you've put them to use. Can you put them to use every time. What have you guys done in the past year If you repeated the year again, you wouldn't. What's the. What's the latest and biggest lesson learned? What's that one? Oh that is absolutely the one.

Yeah, yeah. Thank you. Lisa just reminded me of the one. Um. I'll start. And, Lisa, feel free to add in. Uh, we we talk about optimizing. Right. And everybody's marketers love to optimize, and we love to optimize. What we noticed is that we're over optimizing, and we're now in this scope of tinkering. So when I look at the number of optimizations I've had in the brain in the past year, um, Lisa's target shouldn't be 400.

It should probably be 600, because we're doing way too much tinkering, and we should be investing in building new. You've really got to let the engine learn, and you've got to give it a little bit of time to that. So I think we're a little bit overzealous on trying to put something in and then optimize and optimize. So that would be my probably my biggest lesson over the past year.

I think that's where we're starting to play around more to with the adaptive model cutoffs and actually then just kind of letting things go a little bit and actually using those to guide, obviously whether or not our customer gets chosen for that. So, uh, and just working through with the marketers around, hey, we want to take a different approach on this, and this is what we're thinking of doing. And they've been really receptive to it.

Um, as opposed to that, hey, we want to, you know, change 100,000 to 80,000 and trying to sort of stop some of those conversations happening. Hi. Hello. Um, you mentioned you get test and learn questions a lot, and then you also mentioned allowing CDH to just do its thing, put a bunch of treatments in there and just seeing how they perform. How do you balance that test and learn versus letting CDH do its thing? I think Test and Learn looks different than what it used to look like.

I think that was probably my point. Like test and learn used to be very much get it in, figure out which one wins. Switch it off. So Test and learn for me is much more around, um, creative variants different ways in. Because if you've spent all this time building up a really sophisticated decisioning engine and you make a decision based on data, but then you speak to every single customer in exactly the same way, you're kind of missing a trick on the personalization scale.

So I think, um, that's where I'm focusing. My testing effort is on different ways in to get, um, that resonate with different customer segments. Does that make sense? Thanks. Thank you very much. I am just starting this journey, so I have many questions and especially about outbound marketing. You mentioned that you have gone through. You now have like a fluid journey, but for outbound, at some point you need to decide when you want to communicate to the client.

Can you talk about that approach? How do you decide when to communicate and when not? You talk a little bit about trigger. A lot of it is around trigger based. So what we really try to do is pull out the if things are really trigger based. So I can see as an example, the one that we shared last year was we could see customers on our website downloading what's called like an extended transaction history. So, you know, 12 months of your transactions key signal for you that your customer is looking at all the transactions as they're about to shop around for their home loan. For me, huge signal of intent, really trigger based, really timely. We do not wait for sending out that message. That is high priority outbound trigger based. So you kind of those are the where we really try to make the outbound message based on what the customer is doing and the customer intent.

That being said, you'll obviously always for email have email times and thresholds that you want to send things out. We still send newsletters. They still work really well in business banking. Business customers like long dwell time. They like rich content articles. Those are important. We try to cut it up more into snippets and make it more relevant. But that's always going to be an outbound email and sometimes that's okay as well.

They get used to it on a regular cycle and they want to almost subscribe to that. So outbound marketing is still a really critical part of the whole ecosystem, but it's one part of the ecosystem. It's not the whole. Thanks. Hey. Uh oh. I'm supposed to be asking you questions. Exactly. Um, those are great examples of, um, content and experiences that you put together.

I'm wondering what kind of, um, help do you give the business to ideate those variants or the initial ideas in the first place to use data? Do you how do you come up with the ideas? Have you seen Pega Blueprint? No, I'm just kidding. Make sure you let Alan know that one. Um, no. Actually, Pega Blueprint is really helpful, by the way. Um, and we did. We used it with our white label business at one point, and they were blown away around all the different variants.

Um, but I also use good old fashioned analytics. Right. Look at your book. Look at your portfolio. Who are your customers? What are they doing? My data scientists, well, they do love building propensity models. They're also looking at unstructured data web chat, call center. Um, they're looking at all of that customer intent and trying to figure out where there's hotspots, where there's complaints, where there's issues that we can help our customers with. So definitely analytics. Um, yeah.

Blueprint is amazing when you're doing like a blank sheet of paper ideation session. And then what we would do is pull down and do some analytics off the back of it to try to apply it to our book.

A lot of it's about spending time with the business and just understanding what the problems are that they're trying to solve, and then stepping that into, okay, well, how do we solve that from a how can we take analytics as a way to sort of help help solve that and structure up what we might want to do? Yeah, that's a great point. A lot of listening to the business and just actually kind of going, hey, this is what your business problem is.

Did you know that the brain could actually help you solve this by, you know, this? So that's a really good point, because sometimes, as we all know, you get a brief and it's very specific with doing something. When we get something like that through, we go back to the business and we go, what are you trying to achieve? Because there might be a better way of doing it. Andrew Birmingham from Miter Australia in last year's presentation.

It really felt like a lot of the benefits you were talking about were benefits to the bank, whereas this year it feels much more like the customer benefits, much more sort of front and center. I'm just wondering if that's a sort of deliberate strategy, or if it's just a function of the evolution or the maturity. It's definitely a deliberate part of our strategy and a deliberate part of our strategy from day one. I think we're probably full. Um, what's it called? You fall accustomed to trying to share the metrics that most people are interested in. I'm surprised you didn't say so. Can you talk to me about your revenue figures? I can, but I'm not going to. So I think we do often default to those figures, because that's how you win hearts and minds in the business. And to be fair, that's what we were doing year one and two. We were winning hearts and minds in the business, but it has given us that right to play in the other space.

Um, we have always kept our 60 over 40, but we probably just didn't shout about it enough. So thank you for that feedback. Well, thank you very much, everyone. Really appreciate you coming to see us in. All the best on your journey. Thanks everyone..