PegaWorld iNspire 2023: AmRisc's Journey: Automating the Insurance Submission Intake and Clearance Process Using Pega's AI/NLP and OCR Abilities

A big step in our journey automating a completely manual process. We achieved what other vendors said was nearly impossible by introducing Pega automation capabilities to the end-to-end submission clearance process for AmRisc, a Truist Insurance Holdings company and the market leader in underwriting catastrophe and specialty insurance for commercial properties.

The solution consumes emails from submission mailboxes and extracts data from the email and its attachments using AI/ML, NLP, and OCR. It creates cases, and augments data via integrations from internal systems, makes numerous decisions based on business rules, and determines team assignment and clearance status. It streamlines the account creation in our proprietary systems, uploads attachments to repositories, notifies producers of the clearance status, and handles exceptions via smart routing.

Transcript:

- Hey everybody, good afternoon. Thanks for joining us. My name is Shelley Davis. I'm the Chief Information Officer at AmRisc, We're part of Truist Insurance Holdings company, and I'm here to talk to you today about a project that's near and dear to my heart. Pega is something I was not aware of until a couple of years ago when the bank found a little extra money in one of our capital projects and knew we had a need for automation of some sort. And so at that point I partnered with our friends at Cognizant who ultimately built the solution for us. I have to apologize, I was supposed to co-present with someone and had a death in the family, so he's unable to join, so I hope I can answer any of your technical questions, but I can at least explain the journey we've been on and the benefits, some of the challenges we'll talk about as well with that project. So just a little intro to AmRisc, AmRisc leads the market in underwriting catastrophe and specialty insurance for commercial property. So we are basically a group of underwriters, typically writing properties, anything over seven and a half million dollars. So they get really, really large, lots of data, if you can imagine, a lot of manual work there as well, taking in submissions, which we'll talk about. AmRisc was ultimately established in 2000 as part of Truist Insurance Holdings, our parent company is Truist... Excuse me, Truist Financial Corporation which obviously is one of the largest financial groups in the nation. We are one of the first MGAs to develop a multi-model approach to risk underwriting. So I'm not sure if... If not everybody's in the insurance sector here, I'll explain some of these terms. But when you're dealing with properties this large, you've got a lot of data. Everything from the building materials used, the year of roof, distance from coast, anything that is required for catastrophe modeling. Essentially we're primarily looking at providing coverage in storm ridden areas, fire, flood, quake, things like that. So we need a lot of data in order to understand the level of risk that insuring a property would take. And so AmRisc really set the tone with their multi-model approach, which is where this submission process comes in as well. Most of their success is attributed to their employees. They're a small group considering how profitable, it's about 250 employees. They write about two and a half billion dollars in premium a year, so a lot of work, a lot of data going in there. We're gonna talk today about submissions. So again, for those not familiar with insurance, the submission is the first part of their operational workflow. It is where our producers, so we sit in the middle between the producers and our carriers, doing the underwriting there. We receive about 80 to a 100,000 submissions a year. So the submission is essentially coming from a producer, which could be a broker or an agent. We work in both wholesale and retail insurance, and it's their request for a quote. So out of those 80 to a 100,000, we probably end up only quoting about half of those, and we only end up binding about 25 to 30%. So if you can imagine that submission process is really where people are weeding out which of these emails will we essentially end up creating a quote for? And then how many of those will end up binding? So that's a little bit of overview about AmRisc, jump into what the problem statement was. So trying to find automation for the submission process was something that even some of the big firms, big consulting firms told us could not be done years ago. And it took me a while to figure out why they were saying it couldn't be done, and technology, I think almost anything can be done. And it turns out after about spending about a year on the project myself working with the business, the biggest complexity was understanding the business rules. The business really struggled to articulate how a system could identify the business rules. How can we determine when something needs to be stopped at a human for intervention? What makes it belong to this team? So to give you a little history of that, a little more color around the submissions themselves, they have a triage team. This consists of people like, even the receptionist, a group of clerical employees offshore, that sit there and go through an inbox that's strictly for submissions. So when those producers send those emails, it comes into one of three different inboxes and outlook by distribution channel. So I've shown here the hierarchy of AmRisc. AmRisc has four distribution channels currently. We've got Waypoint Wholesale, Chronos Retail, AmRisc Online and AmRisc Specialty. So those are all AmRisc employees, but the work is divvied up based on whether it's wholesale or retail, and then based on different things like geographic location, the perils that are being covered in insurance. So this chart here or this list, I use to kind of convey the complexity of just identifying which team gets this, because the submission is sent to a general inbox. So trying to understand where does the system take it from there, what data points allow us to understand who to give this to. Before Pega came around, that triage team would spend about five to 10 minutes per email. Because if you imagine an inbox, it's not just submissions coming in there, you've got spam coming in there, you have general inquiries, loss run emails, just a whole slew of things that they still have to go through and figure out, is this just a question? What is it a question about? Which team do I send this to? So in their Outlook inbox, under the primary inbox they have each one of these teams. So under Waypoint you can see they've got a division called general Property. They've got five different teams just under that one division. And so on this screen you'll see those are all the different divisions and teams, and humans are having to figure out from an email, who does this go to? And what does that email look like? It's very unstructured data, they're not in the same format. Some of them will have data points we need in the subject line of the email, some it's in the body. There's always attachments and the attachments are critical. They'll have all of the locations for a property. So there's a Excel file we usually ask for called an SOV. One property could have over a thousand locations on that, and each of those locations have 200 plus data points. And then an ACCORD file being a more structured file that would be attached too. So these clerical employees were having to open these manually and then would have a cheat sheet of all the business rules to figure out who does this go to? Now let me move that email over to that group. Then from there, the underwriting analysts come in and pick that email up out of their team inbox. They read through the same thing, make sure this is who it belongs to. They determine whether or not they can even provide a quote. If they can't, they send messages back to the producer. If they can, then they hand key all of that information into our proprietary underwriting system. So you can imagine the inefficiencies of these people that are very valuable to the company. We could be using them in a lot of other ways than just going through email and sorting files like that. So in addition to the triage team, the underwriting analysts I mentioned, they spend on average about 30 minutes per submission just doing this. And this is only a small part of their job. They have a lot of other responsibilities with AmRisc. So you can imagine this takes a lot of their time, then taking all those attachments, filing them manually. As I mentioned, the submissions contain various attachments. We also get PDF files, word documents, and in talking with the business, the first requirement they had was, I want you to extract all the data from everything. And I looked at them and I said, "We can do that, but where do we put that data?" I think that was the biggest disconnect was, we'll extract anything you want, but what do we do with it? And so that took a long time to get to that conversation and understand what we're looking for in those attachments. So the different sources of data that are required to determine clearance status. So part of the submission process, what we're trying to do is determine if the submission is clear to receive a quote or not clear. There could be numerous reasons it would not be clear. If it's a renewal for insurance, we save that for the incumbent, but other producers are going to send us requests to try and cover that property. So we have to make decisions, typically done by a human, to try and see, are you the incumbent? Where is this account? They look in the body, like I said, the body of the email, the subject line, and just anything else in there to determine which one of these groups does this go to. And that's the very first step in the workflow is, who does this belong to? After that, decisions have to be made to see is this a duplicate email we already received? Is this a multiple, meaning did the same producer accidentally send us this more than once? We don't wanna process it as a brand new submission. Is this something that... A producer that we are not allowed to do business with? So we get a lot of emails from them as well. We don't want to send them a quote if they're not somebody that's appointed with us. So quite a few decisions in that workflow. So that's when, once we realized that all the manual work was happening, the business opportunity presented itself, and we named this system the submission automation system or SAS, and it's not very creative at all, but that is what the users voted on. So that's what it's called. We refer to it as SAS, which is a custom application sitting on the Pega platform. Some of the objectives here, introducing automation into this extremely manual process was first and foremost objective. They wanted to free those resources up to be used on other projects, other tasks. Business grows very fast, even though we're in a very hard market right now, AmRisc does not slow down. So definitely wanting to utilize those resources as efficiently as we can. Also implementing business rules to validate things, it's very easy with all that manual process to have human error, you know, to look at something and not realize that's not an appointed producer. You're trying to provide a quote to them, you know, this went to the wrong team, you're wasting time by sending this to the wrong people. So really using business rules and validation to streamline that process. And then ultimately also is, utilizing API integration to our proprietary underwriting system. So just because Pega can take that email in, look at it and make decisions, the next step after that, once we know it's clear or not clear, somebody's hand keying that into our proprietary systems, why can't we call an API and push that in there? Why can't we call another API to the system we use called ImageRight that stores all their documentation. So that's what we did as well. And the goals there kind of go hand in hand with those objectives. Increasing the efficiency in what we call the submission triage process, which is the first half of the process that says who does this belong to? And can I clear this or not? The clearance, once we know it's clear, then we have automated emails that go out to the producers. Again, saving time from a human manually having to type these emails each time. It also provided an opportunity to track submissions and report on them. So if you think of everything going into an Outlook inbox, that's extremely difficult to track and report to senior management on how many submissions do we get a year? We know how many accounts we create in the system, how many do we get from producers that aren't appointed to do business with us? We don't necessarily enter those in our system. What about, you know, competitors to those producers? How many submissions for the same exact account do we get? We couldn't track any of that. And so Pega has made that possible, which has really helped. Big goal was automating the end to end process. And that was probably the hardest thing to explain to the business was, I know you'd like it to be a hundred percent success on day one, but it's only as good as the data we receive. And if you can imagine, I'm sure as your systems do as well, the data's not always clean. People misspell the street names, they put abbreviations in there. And so being prepared for all of that data, we knew we would not be at a hundred percent on day one, but our goal is to get there as close as we can. And then also to identify additional revenue opportunities. In that inbox, because it's such a manual, time consuming process, we were receiving, we found out, receiving requests from producers that we might want to do business with. But because of time and lack of resources, we couldn't entertain all of those submissions. So this gave a way to say, "Hey, this is not an appointed producer, but is it somebody we want to talk about and maybe appoint them and start doing business with them?" So those were some of our goals. I know this screenshot's a little hard to see. I can try and point a little bit here, this kind of diagonal line, this is what I think of as the happy case of how we ended up implementing the solution. So this would be in the diagonal if no human had to be involved, if we got the perfect data, and were able to process this all the way. The producer sends that email, the email listener's sitting in there on that submission inbox, and it's picking up 100% of the emails that come in. It's bringing them into the submission automation system, again, built on the Pega platform. And it's initially deciding what kind of email is this, is this a submission, a true submission? So we had to define those requirements of, what makes it a submission email, versus no, this is just a general question. Put that in this other work queue over here. So once we know that it's a submission and I'm not talking too much today about the the other types of emails, but we do have processes for handling those as well. So this was more around the submissions. We then want it to... If everything processes there, we have all the data. We want it to be integrated into these two systems. It first goes into what we call RiscTrack, which is our proprietary underwriting system. RiscTrack will kick off the clearance email, or it will be a not clear email if we're not gonna clear it for you. And then it stores all the documents in image, right? This middle piece here, we ended up building three user roles in a portal that we built inside of this application. And it is for the different user groups that if human intervention was needed, that's where it stops. So the very first stop there is called our marketing team. The system initially checks to validate this producer, is it an appointed producer? If not, we need to send it to somebody, because we don't wanna waste the resources going down that path of what team to assign it to. So it routes to our marketing team, they take a quick look. It may just be that the system didn't have that producer in there. We might be doing business with that company, but maybe it's a new producer and they haven't been set up in our underwriting system. So they need to stop and set them up, and then finish processing. They can push the submission through the system that way. If it's a non appointed producer, we have automated emails that come out and say, "Hey, you're not appointed, but if you'd like to talk more about that, here's how to do that." The next is this triage team. This is the group that was the... The clerical team that was really looking at what group does this belong to. So in this part of the system there, it will route to them if for example, we can't extract the data. If we get a format that we are not prepared for, which we will talk about how that got us a little bit in a few minutes. So if we get something that we can't read, if there's required data that's missing, so you've sent me a list of properties, but you forgot to put the ZIP code on all of them, something like that. It has to go to a human for them to take a look at it. We've built the screen so they're very easy to just populate what you see. They can open the attachment that they need without having to download the entire original email. And then as soon as they push the button there, provide the information, the system picks it back up and finishes processing. And at any given time in there it can go to our UA team as well. So the UA team is the one that they're ready to start underwriting this, start writing this business. And so things like, if the system thinks it might be a renewal, but it just can't tell for sure. If it finds duplicates, it finds the same account number, it finds the same policy number, it needs to go to a human. We do have business rules in there that use that process of elimination. And if it can get to just one and confidently say that is the match, then it will go through without human intervention. But that human intervention is there as the fallback of, obviously we're only as good as the data we are given, right? So that was that in a nutshell. We look at things, like I said, it's non-submission emails, the producer validation, renewal status. Underwriter assignment. So once it's assigned to a team, the underwriters on that team are given different levels of criteria, let's call it, for which accounts they can work. Some work the lower dollar amount accounts, some can work accounts over $500 million, some only work in certain counties. So all of that logic had to be built in and that's where the business really had trouble articulating, because they like to move those people around throughout the different teams and divisions. And if you're not telling the technology team or you're not telling the system that you've moved that person, the system's going to keep trying to assign them in their old group. So those were a lot of the things we faced there, trying to get the business to understand how this would work. And then final review, so this next... Or we won't talk about that just yet, sorry, I'll give you a note here. Final review is something we introduced to gain confidence with the business. One of the questions they kept asking us is how do we know it's going to be successful? So we built instead at the end point in the system before it integrates and calls the API into the other system. We stop it and we make the underwriters look at a 100% of these. That was a request by the business that at least for a few months, make them take a look at 100% of these before that email is automatically triggered back to the producer saying, "You're clear," and confirm that the system got it right. That just gave them a sense of comfort. So we built it in a way that we could toggle it on and off very easily for teams. So if one team, their rules are a lot easier, they see that it's working and they're more confident, we can flip that right back off and let the system just flow through. Other teams might want to take a longer look at that. And what ended up happening is we started building reports that I'll show you in a minute, and they were shocked that they were actually having to edit over 50% of the cases. And it came back to the systems working exactly as it was designed. You keep changing the rules on us, right? The system, if you're not telling the system, it's going to keep doing it the old way. So it's helped the business really see some inefficiencies in their own process, even just the way they do transfers of people, not cluing in the IT group that someone's moved to a different team, so we can make these updates. So using AI for the smart automation. My partner here was going to explain this screen, so I'll do my best here. So essentially everything I just explained, the producer sends the email to that inbox, the listener's sitting on that inbox or those, as I said, three different inboxes based on our distribution channels. It's using natural language processing, AI text analysis. So it's definitely looking for keywords, a long list of keywords that we're using. It handles the triage emails, the data extraction. So it's extracting data from a variety of formats of Excel files, that can get very messy, and the ACCORD files as well as the subject and body of the email. And then on top of it is that submission automation system. So as I've mentioned, it's looking at the submission creation process, it's handling filtering, the duplicate multiple logic has a variety of selected reports in there for the business to run. Just quite a bit of... It's pretty impressive going from just having Outlook open to now a system where we can see exactly what's happening, and it handles all of these decisions, really impressive. So this is just a slide deck or this Visio here, does not do this justice, it had to be condensed. I think we went through 37 different versions of a Visio flowchart, a swimlane diagram that... I enjoyed actually working on that. I like that kind of work. But it was just to document out the business rules, every time we would go sit through with the business and walk them through, okay, now this happens next, right? And then this and what if this happens? More edge cases came along. And so 37 versions over probably a year of gathering those requirements, because they just kept changing so much. So this is just a high level of everything. I just explained the different decision points that it makes, but again, high level workflow, the email comes in, it's determined, is this a regular non-submission email or a submission? If it's a submission, go down this path. If I need help making a decision, go to a triage group. If I still need help after that, go to the UA team, and every time it's making a decision, it's also calling our proprietary underwriting system because that's the source system for account numbers, producer information. And so that's where the system's finding its answers to the questions it's asking, and then if it can't answer its own question, going to the business for assistance. So it is extracting, like I said, is it a submission? Some of the issues we had with extraction as well were, we found that the system was not extracting from certain Excel files that should... They looked exactly like the format we needed. Turns out they were in the XSLB format. So using technology to convert those now to XLS, so that they can be processed. That was key there as well. It's looking for keywords. So there's a whole piece of the system up front, the team assignment piece, determining which team, who does this belong to? It uses a process of elimination. It starts with the first team that we gave it that has the, I would say the easier rules, and we built this so that it's in order of, if I'm not a match for team number one, am I a match for team number two? And it keeps going through that logic. To do that we had to use keywords. So we have a team that's called builder's risk. They always, or I would say 99% of the time, the emails they get have the word "builder's risk" in them, or they have a builder's risk application. So we're also looking at attachments where we're not extracting data from the attachment like a Word or PDF file. But we are extracting the name, the title of that attachment, and we're saying if this has the word, "builder's risk" or BR application in it, assign it to that team. They do of course have the ability to override that and flip assign to another team if needed during that final review. Let's see here. I'm gonna skip through a little bit of this. We can answer some of these questions at the end. I mentioned we were looking at SOVs and ACCORD files. So what were our results in the first month? This was something we were eager to see. How did we do? We had about a hundred users. So I mentioned that our company's about 250 employees, about a hundred of those are the people mostly on the underwriting side that are part of this process. About 15 of those were on the triage and marketing side. We processed over 8,000 emails. So that includes all of the spam and the general inquiries, all of that. We integrated within those 8,000 plus emails, there were over 50,000 documents attached to them. So we're bringing in all of those, extracting them from that email as well as integrating them into the other systems, the downstream systems. About 5,000 of those were cleared. That doesn't mean they were cleared on their own in particular, but they got to the clearance status, whether Pega was able to do that on its own or had to go to a human for additional information, they were able to clear that many in the first month. And about 800 were marked as not clear. So not clear is not a bad thing. We would definitely want to still receive those submissions, but now we can track on them. How many people send us submissions that we're not gonna clear them, we're not gonna give you a quote. And that's based on things, like I said, if it's a renewal, which is the majority of our business, we leave it for the incumbent. So sometimes it's just other producers trying to get that account. And then we also identified 12 non-appointed producers in that first month, that the business could then see, is there a revenue opportunity there? Or maybe we don't wanna do business with these folks. So that was nice as well. The value it added? Submissions were being processed in a minute so that diagonal line I showed you, if it was a hundred percent successful, that could process all the way end to end, file the documents and send the email out in one to two minutes at most is what we saw. Some were happening much faster than that. Improved customer experience. So the training in that manual process, each of the teams had their own version of their business rules, their cheat sheets that said, you know, "We're looking at this data, it has to fulfill this requirement, it has to be in this county. Only 10% of the value can be in the state of Florida in this county." So a lot of very complex rules, and for every submission they're having to go through and do that. So the feedback we got immediately from the users was, "Wow, I didn't have to look at that sheet anymore," even though we created a new one for them to help them understand how the system is working, as part of the requirements gathering, the business identified that those teams really shouldn't be working all that differently. The main difference should be the types of property they're insuring, but some of the rules should be the same. So it was a little bit of a learning curve for some of the underwriters as well, because you're changing up what they're used to. But that was a great experience for them having very simple screens, very direct questions. We made sure that for every step of the process that the system stops at a human. It's very clear what the system is asking, is this a renewal? Yes or no? And if they select yes, we require them to enter, what account number is it a renewal from? And those would be cases where the system could not automatically identify it, or maybe it found more than one account that we thought was a renewal. So it said, "You tell me which one it is." Once they input that account number, the system validates, it talks back and forth with the underwriting system. But from an underwriter experience, it was nice to just say, "Yes, this is this account number, move it along and go ahead and send that email for me." It also improved the data consistency. So a lot of this was attributed to the final review that I'll show you in just a minute, but one of the things they realized is, how many different ways the named insured, the corporation that's actually getting the insurance, how many different producers spelt the name differently, or they had the word limited spelled out at the end or LTD or variations like that. And so it really gave them something to extract and say, "It wasn't just the underwriter keying this incorrectly before, this is what we're getting, this many variations." So it helped them clean up their data a lot. Producer ID success rate, 95% or over 95%. We were able to match those producers. The small few that we weren't, were either non-appointed producers or as I mentioned, maybe they were newer producers with a producing company that we already work with, that just had not been set up in our system. SOV read success rate, this was critical, was over or about 70%. We were shocked initially that it was even that high because of the variation of Excel files we're getting, AmRisc has a standard, what we call a standard SOV, we put it on our website, we send it to all our producers, and they don't all use it, or they take it, it's got formulas and fixed data in there, and they will take that and rearrange it and then suddenly, you know, Pega's trying to extract the data where it's been told they'll change the column headings or delete something. So we were really impressed that the read rate was that good. Even though it didn't mean that when we read that 70%, that we had all the data we needed. So the system was built to identify, "Okay, I got 70% of the information, but there's still something missing here, so I need to go to a human for help there." Hundred percent success rate on the data extraction from PDFs using OCR technology. So that was key on the ACOORD files that we were receiving, that had a lot of the information we needed. And then attachment uploads a hundred percent. So that was extremely good news that we got there. That was an integration we had not used before, because again, it was always done manually by a human, so that was good news. So this here is a little bit more on that solution we put in there for gaining confidence with the business. As I mentioned, during testing, they love to see that they, you know, pushed a button or we would have them drop emails. We had them test by taking their real production emails, putting it in a test inbox for us, dropping it, watching the system pick it up and see the entire flow end to end. They thought that was just so cool that it could go all the way to our RiscTrack system and send the email out without them doing anything. But then they started asking questions, well, how do I know all those data points are correct? I need to see more. So this is where we put the, what we call pending final review status in there. And what it does is it queues it up for them. So as soon as the system makes that decision of, I'm clear or not clear, whether it had to be touched by a human or not, it puts it in that final review status. They go into another queue and they can quickly look at it. If they want to update any of the data, if they see that the system did get something wrong or didn't extract something fully, they can edit that and we are capturing all the fields that they are editing, so that we could run metrics like this and say, "How much are you editing and what fields are you editing? Now why are you editing it? Is the system not reading it correctly? Do you have a new business rule we don't understand or didn't know about? What's going on here?" And so we could immediately see that a signed underwriter was the top data point that they were modifying. And the reason is, they were moving people around and they did not have their criteria set up. So the system was working as expected, but the criteria that they wanted the system to use had not been updated because we hadn't been notified. So as soon as we did that, we could run this again and show that that number dropped. The named insured was also the big eye-opener for everyone, is that they didn't realize that the proprietary system has a character limit of 50 characters for named insured. So all these years that they've been doing this manually, the underwriter has been changing the name or shortening the name of the insured to fit into that system. So not only did were we able to help that, we were also able to fix that in the underwriting system to say, "Not sure who built that years ago that would put a 50 character limit on a corporate name, we can lengthen that and now you can pass that fully." But in the meantime, we gave them the option of, cut it down to 50 so that it can go through, otherwise it won't integrate. So that was good to see. Producers as well, they have the option to update the producer. So a lot of times these emails are sent by the producer's admin or their technical analyst, something like that. And so the email is coming from someone other than the producer. The system is not really looking in the body of the email for a name, it's looking for an email address. So what we've asked our producers to do is, or their assistants, is include the producer that you want in there, include their email address. Because if their name is John Smith, we probably have a John Smith across many of our producers and we wouldn't know which one it is. So we give them the option to change that if the system... If they didn't include that email address. And so we now know what company it is, but we're gonna pick as the producer the person that sent that email. This gives the underwriters a chance to say, "No, it's really this other person." Well it also opens up a door to say, "Oops, I selected the wrong John Smith, and it's actually a different producer company now." So we're monitoring to say, you should never be telling a different producer, so different client,, that they're clear or not clear. Picking the wrong one there. So this really opened their eyes to that. So like I said, we built this with the ability to turn on and off for different teams as they got comfortable. I will say so far, they like this so much, because they're learning more about it, that they have not wanted us to turn this off. And I failed to mention we went live with this in August of last year. So they've enjoyed that final review. Once they're in there, if everything looks good and they don't wanna change anything, all they do is click a button and it pushes it right on, calls the integration to those downstream systems and takes it from there. And then when they're ready to start writing that business, they go into our operational system to complete that. So what were the challenges? Data extraction, hands down, was the biggest challenge because garbage in, garbage out, right? The email bodies, even the subject lines are extremely inconsistent. We see that a lot of people put the named insured and the effective date in the subject line, that's great. But then they also add an ACCORD file and we extract a date from that. Which one is correct if they don't match? A lot of times they'll send the email or copy and paste if it's a renewal from last year's submission. So now the year is wrong and I'm putting this in the system and the system says you can't quote something that's a... You know, that wanted to be effective last year. So all of those are things we had to program for and to account for, to alert the business when necessary, to keep in line with the rules of the proprietary underwriting system to make sure it's not violating any of that. So definitely looking at the different patterns in emails, we've made a lot of enhancements in the beginning to say, "Okay, most of them are putting in the body of the email, the named insured and sometimes they just call it insured. So let's use both keywords." Or we found that effective date, sometimes they use it, sometimes they use policy period, and it has a start and end date. It's the first date that you wanna pick, let the system calculate the end date. So a lot of rules like that. The differences in SOVs that I mentioned, the variation on the amount of, this is a screenshot here of just a small piece of one of the SOVs that we have, and we're relying on probably about 10 columns in there of data that we really need to be able to make decisions. The rest of that 250 plus, it gets sent to our underwriting system, but it's not necessary for this process for the submission process. And we find consistently that they leave the county out. So one of the challenges that we had was really around user adoption with that. The business got so excited that we were saying we can extract the data, that they were hoping that that meant nobody had to touch anything. And we found out that about in the beginning, about 80% of the submissions were stopping at triage, because we would extract data, but it's missing required fields, and county was almost always one of them. So we started looking at options like, well if they're putting the ZIP code in there, can't we call another service and find out what county it is instead of stopping it with a human to look at? Those are some of the changes we implemented to improve the accuracy and stop sending so much to the triage team. And so we continue, I think that's the one area, data extraction, the different... We coded for over 70 variations of this standard. We are always going to receive SOVs that are in no format whatsoever that we've seen. So the business is aware those would have to stop, but we're constantly looking for ways to enhance that process, and that's why I'm excited to be at this conference as well, is to hopefully network and see how other people solved for some of those data extraction issues. That's really been our biggest pain point. But otherwise, very happy with the success of the system, very happy to have learned about Pega, and are looking for ways to use this in other parts of our business, like claims and different areas, so. And I think that is the end of my session or my slide. I can open it up to questions if anybody has any? Oh! Or applause. Thank you. Appreciate that.

- [Audience Member] I have a question. When you would extract the data out of the spreadsheet, did you have anything that would help the user kind of do that stare and compare? Like if it... How do they know that it was inaccurate? And if it was inaccurate, did they just bring the spreadsheet up in a window and then type in the system what the right thing was? Or did you have... Build a UI that kind of helped with that and said, "Here's the value and here's where I found it in the spreadsheet, and here's the next field and here's where it was in the spreadsheet." And did you do that in Pega or-

- Right.

- How did you make them efficient?

- We did, yeah, I know, that's a great question, thank you. It's interesting, because that's one area we struggled with is, how much do we show them on the screen. We don't wanna overwhelm them, but some of these SOVs have 800 to a thousand locations. So if we show you all that on one screen, are you gonna sit there and line by line look at them. Now on the screen that they have when they're looking at the submission, there is a link right in the attachment section that they can open the original Excel. So they can open that. We display for them only the top five locations. And the reason is, now in the back end of Pega, we store all of it, but we show them those top five, because for team assignment rules, we only grab the top five to compare it for renewals. So when we talk to the other system, we're saying, "Okay..." And I say top five, it's top five by total insurable value. So we sort that list and say, "Give me the top five highest value locations," and now when I call the operations system and I wanna find out if it's a renewal, I grab five of those, and we're looking for any match of that address. So luckily they don't have to look at all 800 of those locations to really see eye for eye that each one of those things was correct. And most of them are trained also to look for the key aspects that would make it go to this team. So for clearance purposes, they don't need all 800 of those locations, they just need to know which team does it go to, and can we clear the account. And they can tell that by looking at just the five. From what we've found so far, we haven't had any feedback. Now when it goes into that operation system, they need all 800 of those, so they can also see it there as well. So yes, on the screen is where we built that, but a smaller version of it.

- [Audience Member 2] Can you talk a little bit with your producers when you saw consistent bad data coming in, did you have any kind of process or improvement around reaching back out to them and educating.

- Yes, we actually... So the triage team, when things go to them, when submissions go to them, because we can't figure out who this belongs to, sometimes they find that it's because the producer didn't even put the SOV on there, so we can't tell you where it goes until we get more data. So one thing we built with Pega were different closure or resolution codes that the triage team could take. They can, instead of helping the submission go further in the clearance process, they could essentially close it out and select things like, this is an incomplete submission. And it had check boxes they can put on there to say you're missing the SOV or you left off the named insured, and that email would be automatically generated to the producer letting them know, please resubmit it, we can't take this the way it is. What we then did was run reports on who are our producers that are consistently doing this so that on the business side, the marketing folks can go back to them, and try and work that out and say, "Look, we wanna give you this account first." You know, part of insurance too is on new accounts, it's about first in. Whichever producer meets all the qualifications, brings us all the data, that one will be the one clear. So if you're the producer that really wants this and maybe we have a good relationship with you, if you send us bad data, unfortunately the system already processed the one that got it and we can't give you this account. So there's a lot of competitive nature in that, and so this helped go back to those producers and say, "If you want it to be cleared quickly, you know, you have to send it right at midnight at this time and make sure all the data's there." So the reporting side of it and capturing the way that they're closing those, instead of just saying it's clear or not clear, they have that ability to close it for different reasons.

- [Amit] Lovely. This is Amit. Sorry.

- [Audience Member 3] Yeah, in your end-to-end journey, you guys are using an LP tool from Pega, right? I think, did you find out how accuracy NLP is providing... Pega NLP is providing, and is that just starting model you evaluated with some other product that you started using with the first stage.

- Shivaji, you might be able to shed light. Shivaji works with Truist as well on the... And very familiar with Pega. So on capturing the success rate of natural language processing. Yeah. The other thing we use to give us confidence on the technology side, because when the business calls and says, "This doesn't work, this should have been cleared or this should have not been cleared," the first thing I do is I say, "Well, show me the data." And I wanna back through the whole process of what happened. So the Cognizant team did a great job of building out an audit trail that not only we can see, but the end user can see, and we were very specific on the messages in there. So they can see why the system chose what it did. It pulls out, not every piece of data that we extract, but the key pieces of data. So if it wants to know what occupancy codes, that's what we use to determine the teams as well. It will say, this is what they put in the system and this is not one of your occupancy codes. So instead of them having to go back through that entire SOV, they can look at key points like that in the audit log, and that's been, like I said, extremely helpful from a production support standpoint as well. Yeah, sorry.

- Thank you. Good presentation. Very quick question. Well, two questions. How long the overall journey took, right from the start to the implementation? Second, you talk a lot about schedule of values, which is very relevant to property, but did you go and implement the solution only for a specific line of business or did you go wider and started looking into casualty or financial lines or anything else that?

- Sure, no, thank you. That was a great question. So how long it took, it took us just under two years. And I would say that, do I think it would take that long to do it again, no. I think a lot of lessons were learned. Literally the first year was, even though we started developing some proof of concepts with it, with the cognizant team as myself and... Oops, sorry, myself and my team were learning about what Pega can do. It was the requirements gathering process, because we went and sat with every one of those teams, and then with executive management at AmRisc as well, to make sure that we were showing them all the differences. I mean it was really... They were all doing things so completely different. So that exercise and none of those 37 iterations of that Visio, that took about a year. Once the requirements were set, I believe the majority of the development was done in, I'd say under nine months. And I think we could, now that we have an understanding of this and the process, I think it could be done quicker. And we did roll it out to everyone except there's one distribution channel that was on that, that's a little interesting. It's called AmRisc Online. That is the only group that doesn't handle submissions. Instead of a submission, the producers actually go into our underwriting portal. They have a producer facing screen they can get into, and they create the account themselves, and it's routed to an underwriter to see if it can be quoted or not. So we knew though that some of those producers sometimes forget, or it's the same producer that does business with both our AmRisc Online group and our Waypoint wholesale group, that they will send things to that inbox. And so we had a lot of logic, that was probably the second most complex part of the system, was identifying, does this qualify for that team and if so, let's make sure, because occasionally they would have accounts that, maybe when the renewal came in, that AmRisc Online Group would say, "No, Waypoint can have it." So we had to look at the whole history of the... Or the system has to look at that. And then what we would do at that point, if we realized it was for that AmRisc Online Group, the system would close automatically the submission, not integrate it to the other proprietary systems and send an email to the producer saying, you know, "You've submitted this through the wrong method, please go to this website and handle your submission there."

- [Audience Member 4] Lovely. Thank you.

- [Audience Member 5] Hi, I have two questions, one probably for my clarification, did you guys use email listener, or did you guys use an email bot? Like Pega has an email bot functionality as well?

- I've always called it a listener, but-

- It was an email listener.

- Just the email listener.

- [Audience Member 6] Email listener, but what it does is, the listener reads the email, extracts everything from it using NLP, and then there are RPA capabilities which extracts the data from the Excel spreadsheet as well as from the attachments.

- Okay.

- Yeah, so it is a combination of both.

- [Audience Member 5] Okay, but Pega also has an email bot separate functionality-

- Yeah.

- [Audience Member 5] Wherein the email itself will be making it as- Thanks for clarification. Did you guys have any issues handling large attachments which come in the emails?

- Large attachments?

- Yeah.

- Yes, attachments were a huge issue for us, and AmRisc doesn't like to get rid of any attachments. In fact, in our proprietary system we have over 20 years of data and we're trying to migrate it, we're rewriting that system. And I just wanna cry a little bit. It's a lot of data, but on the Pega side, our development team was saying, "This database is growing, it's insane, we can't do this." So we've had to come up with some archival strategies, because we know those same attachments go into the underwriting system as well as ImageRight. So we do on a, I believe, it's a six month cadence, something like that, that we archive those attachments.

- Thank you.

- Hi. I'm curious, when you guys were looking at the underwriter final review, have you considered or looked into having that tuning process be done directly by the underwriters? Meaning like they determine whether or not the entity extracted is right, and the bot learns from that real time? Or is it just the reporting on the back end and then you adjust from there?

- Right now we're reporting on the backend and then adjusting. We're going back to the business to make sure, is this what you really intended? There is some tuning going on as well once we learn that, but I don't believe it's automated, and I'd love to learn more about those options if they are there.

- [Audience Member 7] And maybe that's what the previous person was talking about, that might be the difference between email bot and just the email listener.

- Okay.

- I'm not sure, maybe a Pega person in here could share that, but-

- Absolutely I'd love to learn more, thank you.

- [Audience Member 7] Second question is, was there an opportunity to, on the front end of this process, create some more structured input through like a web form or something before? Like why was it even coming in in an email channel in the first place?

- Exactly, that was my very first question to the business, because I'm of the technology mind that, why would you take this if we could build a screen that... Because there's not so much data, even if they could add an attachment to it, let them key in some of the data, they're not gonna key in 800, 2000 locations and we already have an extraction tool in our underwriting platform that lets them upload that and extracts the locations out of it. So couldn't we use something like that? The answer was, it's very hard to change our producers. So this is an insurance industry expectation that when a producer is sending us a submission, they're usually sending that same submission to maybe five or six of our other competitors. It's kinda like if you shop for car insurance, maybe you're, you know, sending out, getting quotes from everybody, and to tell them that they have to go to AmRisc's website and do this form and then, you know, would the other MGAs like us have another form? So what they do is they create one email and blast it out. And there's also, as I mentioned, the first in rule, they have jobs set up to send those emails directly at midnight because there's a limit on how many days early before that policy is supposed to start, that they can send us the submission. If it's sent too early, which we have that logic too. The system will close it and say, "You're five days too early, you can't submit this." So the producers feel like they have a lot more control when it's in an email, and our business has really wanted to help change that. But that adoption, I think with our producers is what's stopping them.

- Sure. Thank you.

- Sure.

- [Audience Member 8] Hi, I saw in one slide you had OCR, hundred percent accuracy. I was wondering is it with the digital documents or the attachments, or do you have any handwritten documents also? Because it's a hundred percent, so I was wondering if hand written also is a hundred percent can be readable with this tool.

- Right now it's the ACCORD files that we're using. So those are structured, so that was a lot more... That was a lot easier to do, right? I think the business, as I said earlier, they would love us to extract all data from any type of file.

- Okay.

- But since it's not all structured, it's very, very difficult.

- Okay.

- So yeah, but those are... We haven't had any issues on those extractions. It's always been in the Excel files.

- [Audience Member 8] Okay, and the second issue was, you had this NLP built on the emails reading that, so do we have to handle the phishing attacks? Some other emails too, we have to handle it differently? Or the tool capable of handling those phishing emails with a, you know?

- Shivaji, do you know how the-

- [Shivaji] To be honest, I'll have to check on that.

- Okay.

- Yeah, I don't think we built out anything specifically over there for that.

- Yeah.

- [Shivaji] I think since it's a Truist internal mailbox, I believe anything that is junk or anything would automatically go to spam, because we were always reading into the inbox. I don't think we built out anything specifically just to handle spams or anything.

- Right.

- Thank you.

- Yeah, we extract from everything, so. And I'm happy to get you those questions. You guys can find me on LinkedIn if you'd like to connect and I can get you more answers to those questions as well.

- [Audience Member 9] I apologize, I missed the very beginning and I also didn't catch the response on the LOBs, but in terms of submissions, do loss runs play a role in what's submitted to you guys, and how are you... Or are you even addressing loss runs right now or is that a future state?

- So we're not addressing them as part of the submission process. They will attach them to those emails occasionally. So like a loss run history report, something like that. And we will extract that. We don't read those files yet, but we will push those into the underwriting system so that when the underwriter sits down to write the business, they'll have that there for them and they can enter them in the system. We also, if we get an email that is purely loss runs, so it's not really a submission, but someone has separately sent us the loss runs, that automatically gets closed, that Pega case, and it routes forwards the email to our loss runs team that then handles it from there. But I could see using Pega to expand that capability.

- [Audience Member 9] Because that data is valuable in the underwriting.

- Very valuable in insurance. Yeah, how many losses have you had on this you know, account's gonna change your risk rating.

- [Audience Member 9] One last question, supplemental ACCORD forms or supplemental forms from the agent standpoint or producer standpoint? Those are, I'm assuming all ad hoc, 'cause you can't really build to that standard, right?

- Right, and you know, they gave us a few variations of the ACCORD files, a few different versions there that they wanted us to code to. And then the SOVs, which are really AmRisc proprietary, but we coded to that, and technically we could code to anything, they just haven't asked, because the biggest issue is they know they want that data. They don't know where they want it to go. Right now they just have it as attachments that they can look at when they need to. They don't use it to make automated decisions. So I think as they grow in that process, they may have us do that.

- Thank you.

- Sure.

- [Audience Member 10] Hi, one question, do you have any monitoring in place for the emails which are incoming, so you can review if all the emails get read from the emails list now?

- I'm sorry, could you ask that again?

- [Audience Member 10] Do you have any monitoring in place to see what comes in the email inbox and what is creating a case?

- No, and that is we... Right now it's humans monitoring it, because that is one thing we saw that when one of the servers was down, certain... There was like an hour gap and luckily it was in the middle of the night, but about two or three submissions came in that we were able to recognize did not get picked up as read, and didn't get picked up by the listener, and it was a few weeks after that was submitted. So it could be a big risk if it was something very valuable there. I think it was just regular emails, it turned out to be, but I would love to learn more if anybody has a solution for that. We've been talking with our messaging team and that's the hardest part is getting them to tell us what's sitting in that inbox. We know what the listener picked up but how can we confirm if anything was left over there?

- [Audience Member 10] All right, thank you.

- Good to know.

- [Audience Member 6] Just a quick clarification, I know there was a question regarding email listener bot, I just got a clarification. It was an email bot that we are using, not a listener. Just wanted to set that straight.

- Great, thank you. Well, awesome, well thank you so much, I appreciate it, and I hope you guys have a great rest of your day. Thank you.