Every company should have an AI policy in place, if not a full AI program, that identifies the company’s philosophy, obligations and who is accountable to whom.
Companies with a governance gap lack the discipline to identify who owns, validates, monitors and is accountable for the AI model when it impacts customers or final outcomes, Woods said.
“You can’t just create and stand up a policy. You actually have to live it,” said Woods.
Who is accountable for owning your AI outcomes?
Leadership should be able to name the person or people responsible for AI outcomes. Note: This shouldn’t be a small group within IT, but executives or oher another point person prepared to speak to the board about what they’re seeing.
What human oversight is there to ensure decisions are made correctly?
There should always be a human in the middle of a high-impact decision. This oversight should ensure your AI isn't drifting off course, introducing bias or making decisions that don’t make sense. The human in the middle also verifies that decisions look right.
Oversight roles can vary depending on a company's AI journey. Early on and without a budget for a dedicated role, this could fall to executive leadership, with each leader responsible for AI oversight on their respective teams.
Organizations with more mature AI growth programs might have a dedicated AI role, such as a chief risk officer or a chief privacy officer. This person is also responsible for ensuring that the company’s AI philosophy guides how AI is actually being implemented.
How are you documenting, monitoring and providing oversight for AI use? Do you have audit trails? Ask to see documentation, policies, audit trails and the outcomes.
If a regulator comes into your business or you have to provide proof that AI is working as intended, can you explain it? Can you document it?
Your company should test the models to ensure that potential negative outcomes, such as inaccuracies, bias and security concerns, are corrected and do not seep into actual decisions that affect your business or your customers.
Teams should not only test these models to guard against problems but also provide clear documentation and audit trails of the outcomes of those audits and what actions were taken to correct errors. Woods recommends saving those documents indefinitely for historical reference.
When is human review required, and when do humans have the ability to override a decision?