United States (federal) · voluntary framework
NIST AI RMF · MEASURE: what it asks, and the records to keep
MEASURE uses quantitative, qualitative or mixed methods to analyze, benchmark and monitor AI risks, through 4 categories and 22 subcategories. It is where documented test sets, metrics, deployment-like performance testing, production monitoring and evaluation of the measurement process itself are expected.
01 / What it asks
What NIST AI RMF · MEASURE asks
Organizations adopting the AI RMF; MEASURE is done by TEVV, engineering and risk teams for each AI system, before and after deployment.
- 01Select metrics for the most significant mapped risks and regularly reassess metric appropriateness (MEASURE 1.1, 1.2).
- 02Document test sets, metrics and TEVV tools; measure performance under conditions similar to deployment (MEASURE 2.1, 2.3).
- 03Monitor functionality and behavior of the AI system in production (MEASURE 2.4).
- 04Demonstrate validity and reliability, document generalizability limits, and evaluate safety, security, explainability, privacy and fairness (MEASURE 2.5–2.11).
- 05Evaluate and document effectiveness of the TEVV metrics and processes (MEASURE 2.13).
- 06Track existing and emergent risks over time and gather feedback on measurement efficacy (MEASURE 3, 4).
02 / AI work
Where it touches AI work
AI-specific. It covers test, evaluation, verification and validation of AI models and systems, including monitoring of deployed behavior.
03 / Dates
The dates that matter
NIST releases AI RMF 1.0 (NIST AI 100-1)
Generative AI Profile (NIST AI 600-1) released
AI Action Plan directs NIST to revise the AI RMF
Concept note for critical infrastructure AI RMF profile
04 / Enforcement
Who enforces it
None; voluntary.
05 / Records
Records an organisation keeps
What an auditor or an inquiry usually asks to see. Your obligations depend on your role and your facts.
- 01TEVV plan with documented test sets, metrics and tools
- 02Evaluation reports under deployment-like conditions
- 03Production monitoring dashboards or logs with drift and incident records
- 04Review of measurement effectiveness and metric updates
Optional evidence · where Planisphere fits
Keep a record anyone can check
When your team meets a duty, Planisphere can seal the event it reports as a receipt.
A disclosure shown, a review signed, a model change approved: the event your system reports.
The event becomes a sealed receipt, dated and attributed to whoever is signed in.
Paste the receipt into Verify. No account or key.
A record supports your compliance evidence process. It is not a legal finding or a certification. Systems that call Planisphere's gate API can also hold or refuse an action before it runs.
QUESTIONS
The questions that come next
What ongoing-monitoring evidence does M-25-21 expect?+
M-25-21 requires agencies to monitor high-impact AI on an ongoing basis, through testing and periodic human review where feasible, to find adverse effects on performance and security, including possible privacy, civil rights or civil liberties violations. Monitoring must be designed to detect unforeseen circumstances, changes to the system after deployment, and changes in the context of use or data. Agencies must apply mitigations, keep system and use documentation, and where possible make the evaluation traceable. Repeatable test results compared across runs are useful evidence. This lines up with AI RMF MEASURE 2.4, though the memo does not cite the RMF. Read the full answer
Which NIST AI RMF MEASURE subcategories can a third party evidence?+
A third party can most readily supply evidence for the testing-focused subcategories: MEASURE 2.1 (documented test sets, metrics and TEVV tools), 2.3 (performance under deployment-like conditions), 2.4 (production monitoring), 2.5 (validity, reliability and generalizability limits) and 2.13 (effectiveness of the TEVV process). MEASURE 1.3 also calls for independent assessors or internal experts outside the development team. Subcategories that depend on internal context or judgment, such as 2.2 human-subject evaluations, 2.10 privacy, 2.11 fairness and bias, and 2.12 environmental impact, usually also need the organization's own records. Read the full answer
Does NIST AI RMF require independent / third-party evaluation?+
No. The AI RMF is voluntary and does not mandate a third-party evaluator. MEASURE 1.3 says internal experts who were not front-line developers and/or independent assessors should take part in regular assessments, so internal independence can meet it. Other rules do require independence: OMB memo M-25-21 requires an independent review of each high-impact AI impact assessment by someone in the agency who was not involved in development, and New York City's Local Law 144 requires an independent bias audit of automated employment decision tools before use. Read the full answer
SOURCES
Go back to the source
National Institute of Standards and Technology
National Institute of Standards and Technology
NIST Trustworthy and Responsible AI Resource Center
CHANGE LOG
What changed, and when
Rebuilt from primary sources; facts checked on this date.
Related: NIST AI RMF · MEASURE for defense AI workflows · NIST AI RMF · OMB M-25-21/22 · DoW RAI S&IP · EO 14179 · FIPS 140-3 · SHA-256 / FIPS 180-4