How Scotiabank built an ethical and engaged AI culture

What does it take to produce an award-winning AI initiative? Scotiabank, officially the Bank of Nova Scotia and one of Canada’s largest banks, recently won two AI awards at the same event. DataIQ has given The bank received the award for the most innovative use of AI for its chatbot and recognized its overall data and AI ethics program as the best responsible AI program, calling it a “pioneering initiative in the financial sector.”

We think it would be useful to describe how an innovative and responsible AI use case came to be, and how a particular AI use case reflects the culture in which it is developed. Latest Scotiabank article in 2021but a lot has changed since then, and not just the advent of generative AI.

How to Create an Effective Chatbot

The app that won the DataIQ Innovation Award powers a chatbot for Scotiabank’s contact center. You may know that AI-powered chatbots are commonplace at major banks. What makes one chatbot better than another is the quality of the underlying knowledge it contains and the quality of the AI ​​models that serve it to customers. Scotiabank is tackling both of these issues.

Grace Lee, the bank’s head of data and analytics, said she was particularly proud of the contact center’s participatory effort to improve knowledge quality. The center took ownership of its knowledge base and organized it efficiently to ensure that every document fed into the chatbot was clear, unique and up-to-date. recognized the creative application of the tool to auxiliary AI models that improve and maintain chatbot training. This AI-for-AI strategy has allowed Lee’s team to automate important parts of the bot training process, such as identifying optimal new training topics, saving thousands of hours of manual work. These efficiencies have led to a better product: since its introduction in late 2022, the chatbot’s accuracy level has increased from 35% to 90%.

More than 40% of customer questions via chat are answered without human intervention. When customers decide to speak to a human agent, they expect the agent to be knowledgeable about what they discussed with the chatbot. To deliver this capability, the bank developed a quick summary feature using large language models. The agent receives a short text summary of the conversation, including the customer’s intent and the requested action. Providing the summary reduces the overall time it takes for the agent to get up to speed by 60-70%.

Contact center staff weren’t the only ones working on the chatbot. Other groups included Lee’s organization responsible for customer insights, data and analytics, digital products and design, and software engineering. Lee told us that the chatbot’s participatory and collaborative development is indicative of a cultural shift within the bank.

Generative AI tools like ChatGPT have made AI more accessible, and people are excited and engaged enough about what AI can now do that they are more willing to make improvements to the unstructured data used to power it. This is consistent with the findings of our most recent survey of large organizations’ data environments regarding State of Data and AI in Large Enterprises in 2024:For the first time, the number of respondents saying their organization had a data and analytics culture in place doubled (from 21% to 43%) in one year. We concluded that generative AI was the most likely cause, and Lee’s comments seem to support that hypothesis.

A new data domain

When we wrote about Scotiabank three years ago, the focus was on creating trusted and reusable data sets, or RADs. As the bank has moved its data to the cloud, the focus is now on consolidating RADs into a cloud-hosted enterprise data model that the entire bank can access: This creates a single version of the truth and simplifies data management, governance and use. Structured data of this type will always be important for banks and most other organizations.

What’s new at Scotiabank is the attempt to manage unstructured data, as the contact center did with its customer questions and answers. This type of data is the fuel for generative AI, but most organizations haven’t really started to manage it effectively yet. Data Industry Leaders Survey By the end of 2023, 93% agreed that a new data strategy was essential to the success of generative AI, but 57% had taken no steps toward a new approach.

But Scotiabank’s Lee is taking action. “Despite the fact that knowledge management is an ongoing and daunting exercise, with many documents and policies related to many products and services across multiple geographies,” she sees knowledge, information and document management as part of her data mission. “We will undoubtedly discover many duplications and challenges in our knowledge base,” she said, citing the contact centre’s discovery of multiple versions of the same bank policy, often in the form of hard copy printouts.

The bank has begun tackling the problem with a variety of initiatives. Given the growing interest in generative AI experiments and the tight connection between the knowledge of the business and that of every part of it, Lee said she expects the business side of the bank to increasingly take charge of unstructured data quality. In addition to the contact center, Scotiabank also cleaned up the knowledge base for the payments business. It was a smaller suite of products than the contact center needed to handle, and Lee said her group was able to quickly get the business team in that group up to speed on how to manage its own content. She expects her group to do this gradually in other parts of the bank.

Integrated ethics

As we discussed in a previous column on UnileverWe argue that organizations need to integrate ethics-driven thinking into the process of creating AI solutions from the beginning. Scotiabank is taking this approach not only for generative AI, but also for for all types of AI and analyticsand the use of data in the bank more generally. In addition to having an AI risk management policy, Scotiabank has a data ethics policy and a data ethics team to advance it. The policy is now part of the bank’s code of conduct, to which all employees must attest their acceptance each year. The approach to data ethics has also won an award price for this work at the Qorus-Accenture Banking Innovation Awards (OK, a bronze medal, but it was among more than 680 banking applications).

To identify ethical issues related to AI at an early stage of use case development, Scotiabank worked with Deloitte Canada to develop Ethics Assistantan application that assesses the ethical impact of an AI use case before it is fully deployed. Running the wizard is the first step in all new AI and machine learning projects at the bank. If it reveals ethical issues, the use case is at an early enough stage to change the design.

The bank also developed a mandatory data ethics training program for all members of the customer insights, data and analytics team or those doing advanced analytics work elsewhere in the bank. All of these ethical pillars combined helped set Scotiabank apart from other financial organizations and earn the DataIQ Responsible AI Award.

When we profiled Scotiabank in our 2021 column, we described a bank that was playing catch-up to its competitors. It now appears to be ahead in many ways. The breadth of the organization’s activities and involvement in AI bodes well for its future.