Corporate AI Governance and the Path to Explainable AI

Artificial Intelligence has been making great strides in the last few years and it quickly arose to global prominence with the launch of OpenAI's ChatGPT. It has been reported that ChatGPT is the fastest growing consumer software application in history as it gained over 100 million users in just two months.

Concerns with AI governance
However, this breakthrough has also raised questions and concerns around how AI models like ChatGPT are monitored, governed and regulated. How do we make sure these tools don’t breach our privacy, access data they’re not supposed to or deliver low-quality, erroneous results for both our enterprises and our customers? How do we ensure they are not biased against race, age, religion, gender and so forth?

As Henry Kissinger said in a recent interview, “We cannot review all the knowledge that the machine has acquired. We are giving it that knowledge. But this will be one of the big debates. I am now trying to do what I did with respect to nuclear weapons, to call attention to the importance of the impact of this evolution.”

Kissinger is not alone in his concern. In an open letter co-signed by Elon Musk, Steve Wozniak and thousands of experts in this area, advocates are demanding a pause in AI research for at least six months until a set of safety protocols are implemented.

The White House’s Blueprint for an AI Bill of Rights
On May 4th of 2023, U.S. President Joe Biden met with the CEOs of leading AI companies including Google, Microsoft, OpenAI and Anthropic emphasizing the importance of ensuring the safety of AI products before deployment. The White House had published a so-called blueprint for an AI bill of rights last October which outlined five guiding principles intended to guide the responsible design, use and deployment of AI:
• Protection from faulty or unsafe AI systems
• Protection from any discriminatory AI or algorithms
• The right to data privacy and security when dealing with AI
• The right to know when you are interacting with an AI system
• The ability to opt-out of experiences that employ AI

The need for corporate AI governance
As AI rapidly gains momentum, you want to make sure your organization doesn’t become a tale of misfortune discussed in the global news cycle due to gender bias, a privacy breach or some other calamity influenced by AI. It is time to take steps to put more control and governance around your AI initiatives and turn that black box AI into a glass box.

So, what is AI governance? AI governance is a framework or a set of processes that outlines and guides the use of AI in an organization or society. The goal of AI governance is to ensure that AI is ethical, transparent, responsible, fair and compliant with legal and regulatory standards. It also involves managing the risks, quality and accountability of AI systems and their outcomes. AI governance can help organizations and stakeholders trust and benefit from AI-powered automation and decision-making.

A new corporate AI governance solution that enables explainable AI
At Quest, we are excited to introduce the first data intelligence solution in the data catalog market that enables data governance around AI models: erwin Data Marketplace. It ensures high-value, trusted AI models are easily found by the teams needing to put them to use and delivers the curation and governance capabilities to develop and manage explainable AI within your organization.

With the help of erwin Data Marketplace, you can ensure you employ AI models that are trained using the right datasets with certain quality. You can ensure training datasets do not contain any sensitive information which may cause bias. You can monitor the quality of datasets and catch data drifts on time to ensure you retrain or refrain your models as needed. It helps the adaptation of your AI models in your business by building that trust and transparency.

Figure 1: an AI model profile in erwin Data Marketplace.

With erwin Data Marketplace, you have one central location to curate all of the information associated with the AI model. Definition, description, business purpose, model type, when the model was last trained, who is responsible for its governance, and more are all documented and readily available for explanation and anticipated future regulatory compliance. Just as with the enterprise datasets governed within erwin Data Marketplace, automated data value scoring relative to the model and user ratings and reviews help to quickly steer potential users to best-fitting, high-value models to consider for use. Built-in task and workflow capabilities route access requests to appropriate parties and establish a full audit trail surrounding access approvals, denials and associated actions.

With the help erwin Data Marketplace provides, your organization can begin to take steps now towards responsible and explainable AI practices – ensuring AI is used in your best interests and is compliant with any AI regulations that may be introduced.

Visit the erwin Data Marketplace product webpage for more information on AI governance capabilities, and how you can make it easy for all across your organization to shop, share and compare the governed enterprise data and AI models that will maximize the business impact of your data.

You can also see erwin Data Marketplace firsthand in this recent 30-minute erwin Data Marketplace webcast or by viewing this 7-minute quick tour.

Yetkin Ozkucur is Data Intelligence Practice Lead at Quest Software.