The NIST AI Risk Management Framework

As AI becomes more integrated into our everyday lives, ensuring its responsible development and use is vital. AI governance provides the necessary structures and processes to achieve this. This article will focus on the NIST (National Institute of Standards and Technology) AI Risk Management Framework (AI RMF), which provides a structure for managing AI.

Here's a breakdown of key NIST AI RMF resources:

• NIST AI Risk Management Framework (AI RMF): This comprehensive framework provides a structured approach to identifying, assessing, and managing risks associated with AI systems throughout their lifecycle. The AI RMF outlines core functions (Govern, Map, Measure, Manage) and provides detailed categories and subcategories with suggested actions. Organizations can use this structure as a template to build their own risk management processes, documentation, and governance structures. The AI RMF helps you think through all the critical aspects of responsible AI implementation.  
• NIST AI RMF Playbook: This is a companion resource to the AI RMF and offers suggested actions, references, and related guidance to achieve the outcomes laid out in the AI RMF Core. The Playbook is organized according to the four core functions and their subcategories, providing practical ideas that organizations can adapt and incorporate into their own workflows and documentation. You can download the Playbook in PDF, CSV, Excel, and JSON formats, making it easier to integrate the suggestions into your internal systems.  
• AI RMF Profiles: NIST encourages the development of use-case specific "Profiles" based on the AI RMF. These profiles are implementations of the AI RMF functions, categories, and subcategories tailored to a specific setting or application. While NIST provides some examples of use cases, the concept of a profile serves as a template for how organizations can customize the AI RMF to their unique contexts, risk tolerance, and resources.  

AI Governance in Healthcare Diagnostics

To truly appreciate the significance of AI governance, let's consider an example in healthcare. Hospitals are now implementing an AI-powered systems to analyze medical images, such as X-rays and MRIs, to assist radiologists in detecting potential anomalies like tumors.

The hospital would leverage the NIST AI RMF to systematically address the risks associated with this diagnostic tool.

• Govern: They would establish clear roles and responsibilities for overseeing the AI system, including medical professionals and IT specialists. Policies would be put in place regarding data privacy (especially concerning sensitive patient information under regulations like HIPAA), algorithm bias detection, and the process for validating the AI's accuracy.
• Map: The hospital would map the AI system's lifecycle, from data acquisition and training to deployment and ongoing monitoring. This would involve understanding the data sources, the AI model's architecture, and the potential points of failure or bias.
• Measure: Performance metrics would be defined to evaluate the AI's accuracy, sensitivity, and specificity in identifying anomalies. Regular audits and testing would be conducted to ensure the system performs as expected and doesn't introduce new risks over time. They would also measure the impact of AI on the radiologists' workflow and patient outcomes.
• Manage: If risks are identified, for instance a higher rate of false positives for a particular demographic, the hospital would have established protocols to mitigate these risks. This might involve retraining the model with more diverse data, adjusting the AI's confidence thresholds, or implementing a human-in-the-loop system where a radiologist always reviews the AI's findings before a diagnosis is made.

NIST provides the "how-to" and the structural elements for responsible AI. It's up to individual organizations and sectors to take these resources and adapt them into their own specific templates, policies, procedures, and documentation. Ascendient Learning’s live, hands-on Responsible AI with the NIST AI Risk Management Framework covers all of this and much more and can be customized for your organization’s AI governance needs.