Question · NIST AI RMF · MEASURE
Which NIST AI RMF MEASURE subcategories can a third party evidence?
MEASURE uses quantitative, qualitative or mixed methods to analyze, benchmark and monitor AI risks, through 4 categories and 22 subcategories.
01 / The answer
The short answer
A third party can most readily supply evidence for the testing-focused subcategories: MEASURE 2.1 (documented test sets, metrics and TEVV tools), 2.3 (performance under deployment-like conditions), 2.4 (production monitoring), 2.5 (validity, reliability and generalizability limits) and 2.13 (effectiveness of the TEVV process). MEASURE 1.3 also calls for independent assessors or internal experts outside the development team. Subcategories that depend on internal context or judgment, such as 2.2 human-subject evaluations, 2.10 privacy, 2.11 fairness and bias, and 2.12 environmental impact, usually also need the organization's own records.
02 / The rule
What NIST AI RMF · MEASURE requires
MEASURE uses quantitative, qualitative or mixed methods to analyze, benchmark and monitor AI risks, through 4 categories and 22 subcategories. It is where documented test sets, metrics, deployment-like performance testing, production monitoring and evaluation of the measurement process itself are expected.
- 01Select metrics for the most significant mapped risks and regularly reassess metric appropriateness (MEASURE 1.1, 1.2).
- 02Document test sets, metrics and TEVV tools; measure performance under conditions similar to deployment (MEASURE 2.1, 2.3).
- 03Monitor functionality and behavior of the AI system in production (MEASURE 2.4).
- 04Demonstrate validity and reliability, document generalizability limits, and evaluate safety, security, explainability, privacy and fairness (MEASURE 2.5–2.11).
03 / Records
Records to keep
- 01TEVV plan with documented test sets, metrics and tools
- 02Evaluation reports under deployment-like conditions
- 03Production monitoring dashboards or logs with drift and incident records
- 04Review of measurement effectiveness and metric updates
Optional evidence · where Planisphere fits
Keep a record anyone can check
When your team meets a duty, Planisphere can seal the event it reports as a receipt.
A disclosure shown, a review signed, a model change approved: the event your system reports.
The event becomes a sealed receipt, dated and attributed to whoever is signed in.
Paste the receipt into Verify. No account or key.
A record supports your compliance evidence process. It is not a legal finding or a certification. Systems that call Planisphere's gate API can also hold or refuse an action before it runs.
SOURCES
Go back to the source
National Institute of Standards and Technology
National Institute of Standards and Technology
NIST Trustworthy and Responsible AI Resource Center
RELATED
Related: NIST AI RMF · MEASURE: the full guide · What ongoing-monitoring evidence does M-25-21 expect? · Does NIST AI RMF require independent / third-party evaluation?