How can we trust AI if we don’t know how it works?

This blog first appeared here on the Responsible City website, an initiative led by the City of London Corporation, as part of a series of interviews for the Business of Trust programme led by The Lord Mayor of London. 

Q&A series with Tracey Groves, founder and director of Intelligent Ethics on ethical leadership:

I know that there are safeguards, controls, checks and balances in place to assess and monitor safety and security, with governance, standards and regulation to enforce and maintain standards.

So, the question is, don’t we need a similar framework for AI? I believe we will have to learn how to trust AI enough.  I don’t need to know how the system works in detail, but I do need to have enough confidence and a belief that the decisions that are being made by the system are intelligible, fair and subject to the appropriate monitoring and checks.

I do recognise that there is concern about the ‘black box’ that AI is using more and more to make decisions, and that there will be an inherent level of mistrust in a system that we don’t understand. Therefore, my questions are, how we can learn to trust it enough? What do we need to do to ensure that there is sufficient intelligibility and transparency?  Will I ever completely understand how decisions are made and do I need to?

When critical decisions are made, for example, when a mortgage offer is accepted or declined, or a medical diagnosis is delivered using advanced technologies, how can I be confident of the fairness, accuracy and integrity of the outcome, even though I do not get to see the inner workings?

Ethical principles, supported by a robust and effective governance framework that is real and measurable, must be hard-wired into the design of AI right from the start if we are to learn to trust AI and the key outcomes of intelligent systems, enough.

READ ON INTELLIGENT ETHICS WEBSITE