Amazon SageMaker Model Cards

Amazon SageMaker Model Cards offer a structured, centralized way to record important details about machine learning models throughout their life cycle for governance and reporting. They help teams keep track of details like the model’s purpose, how it was trained, its performance, and any risks to be aware of—especially helpful in industries that require detailed documentation. Think of it as a “profile” for each model, capturing essential details, performance data, and recommendations to help stakeholders understand its strengths, limitations, and suitability for specific tasks.

What does a SageMaker Model Card store?

Image SageMaker Model Cards as an instruction manual that provides detailed documentation of a machine learning (ML) model. It helps developers and users understand how to operate and maintain the model and provides a clear understanding of the model’s purpose, behavior, and requirements. This information of the model is stored in the Model card JSON scema. Let’s look at the information that is stored in the Model Cards in detail:

Intended use of a model

Sagemaker Model Cards store the intended uses of the model to help users and other developers provide the information needed to deploy and train the model. Generally, the following is stored to provide intended use:

• The general purpose of the model that describes the use cases of the model.

• Type and format of data used to train the model.

• Scenarios in which the model is intended to be used and the scenarios where the model is not a good fit.

• Any assumptions asked during the development of the model.

Risk rating

The risk rating in Sagemaker Model Cards communicates the potential risk associated with the deployment and the use of the machine learning model. It provides a standardized way of documenting the level of risk the model poses and allows organizations to ensure proper governance and ethical considerations are in place. This risk rating can be unknown, low, medium, or high.

Evaluation metrics

The Model card JSON schema also stores the evaluation matrix for a mode to provide detailed insights into its performance. These matrices help understand the ML model's performance against specific benchmarks.

Benefits of SageMaker Model Cards

Amazon SageMaker Model Cards provide a streamlined way to document, assess, and share your model details with internal and external stakeholders.

• Centralized documentation: They help organize key information, like a model’s purpose, performance, risk levels, and ethical considerations, supporting transparency across the model life cycle.

• Support for responsible AI: By including sections for intended use and risk ratings, Model Cards promote responsible AI practices, guiding oversight based on model impact.

• Integration with AWS ecosystem: They seamlessly integrate with tools like SageMaker Clarify for automated evaluation metrics, aiding regulatory compliance.

• Version control: Each model card version is immutable, recording changes over time. This ensures transparency and traceability of model updates, supporting version management and audit trails.

• Easy sharing: They can be exported as PDFs, providing a lasting record that’s useful for sharing insights with stakeholders and meeting audit requirements.

How SageMaker Model Cards work

Amazon SageMaker Model Cards provide an easy way to track and share key information about ML models. Following are general steps to show how it works: