James Boyd-Wallis spoke with Joe Cappai, AI Governance Consultant at digital technology firm Kainos, about the business and societal benefits of responsible AI, how Kainos embeds responsible practices and principles, and what more we can do to make responsible AI the norm.
A catalyst of adoption
Responsible AI is the design and development of AI systems that operate with trust and integrity, deployed in an ethical and human-centred way, in line with standards and regulation, according to Kainos. More broadly, it often includes considering the societal impact of the technology.
However, why is it important?
Joe argues that responsible AI is important for several reasons. First, responsible AI acts as a “catalyst for AI adoption”. Research shows that around 80% of proof of concepts fail. “This is often because organisations and individuals do not trust the technology to make decisions”, explains Joe.
“My view is that if you have responsible AI baked into AI systems from the get-go, you have the processes and procedures in place to build trustworthy technology and can scale a lot faster”.
In this context, trust is defined as a technical issue. As Joe explains, “Are the outputs accurate, unbiased, and have they been evaluated on things that they should have been evaluated against and not protected characteristics, for instance?”
Second, responsible AI should also ask whether AI systems are ethical. “While we know we can build an AI system to do a certain task, we must always ask whether it is the right thing to build that system?”, asks Joe. In some cases, the more ethical choice may mean not designing or developing an AI tool.
Finally, responsible AI is important to ensure accountability. “You can’t hold a machine responsible. A person needs to put their hand up and own the decisions. So, AI systems need accountability processes and chains and flows of control. The extent of control “will depend on the context”, explains Joe.
How responsible AI benefits business and society
Put simply, “more responsible AI is more accurate AI”, says Joe. This means AI systems shouldn’t use proxy data or measures.
For example, in regulated lending there are clear limits on what data can inform a pricing decision. Income can be used. So, in this case, the AI system must use income and other acceptable data points to set the price.
“But if the system starts using protected characteristics such as race, this creates a biased and unethical outcome for the individual. It also creates significant legal, financial and reputational risks for the financial institution,” explains Joe.
“So, if organisations build responsible systems from the ground up, the outputs will be more accurate and will create better outcomes for both the organisation and society.”
As a result, he says, “we are seeing organisations embed responsible AI principles and practices as a competitive advantage. Those organisations with safe, ethical, fair and sustainable systems are more likely to attract and retain more clients”.
Moreover, responsible AI does not just benefit individuals and organisations. It can also benefit society. Responsible AI drives responsible innovation, which ensures that new inventions maximise positive outcomes.
However, “according to government data, a large portion of the UK population is uncertain about the future of AI and what it means for society”, says Joe. “As technologists, we have a responsibility to have a positive impact on society. We should build fairer, more equitable systems that people can trust.”
Responsible AI in practice
For Kainos, its approach to responsible AI begins with a discussion with clients focused on three key areas.
First, the firm explains how responsible AI lays the foundation for greater adoption and AI maturity. Second, it draws on AI assurance, or technical guardrails and standards, to build legally compliant systems. Third, the firm discusses what sovereignty and control mean for its clients.
“Here, the conversation will depend on the use case. For instance, a mission-critical national security use case will have very different sovereignty and control requirements compared to a customer-facing chatbot on a retailer’s website”, explains Joe.
This is supported by responsible AI principles and practices that include trust, ethics and integrity, built on a foundation of AI governance. Across these pillars is a human-centred approach where Kainos works to prioritise the human over the technology in any system.
While Kainos takes a robust and responsible approach to AI development, Joe argues that society and government can do more.
“Building more responsible AI systems requires greater awareness and education about AI and its consequences. It also means normalising asking the tough questions. Just because something is innovative, does not mean it is good”.
Joe suggests regulation has a role. Financial regulation ensures that banks must educate customers about their products. “Something similar could apply to AI and technology”.
AI does not present a completely novel challenge. We can “learn from other industries and apply similar principles”, suggests Joe.
However, responsible AI must also move beyond principles. “What we need now is action”, argues Joe.
“Responsible AI must move from an advisory function to actual decision-making authority. It needs to live in contracts rather than culture, because culture is the first thing cut in a downturn.”