Model card

Mitchell et al. proposed model cards at FAT* 2019 with nine sections: model details (developer, date, version, type, licence, citation), intended use (primary uses and users, out-of-scope uses), factors (the groups, instrumentation and environments across which performance may vary), metrics (and why they were chosen, including decision thresholds and how uncertainty is estimated), evaluation data, training data, quantitative analyses, ethical considerations, and caveats and recommendations. The key technical idea is disaggregated evaluation: reporting metrics separately for each relevant factor and their intersections (for example skin type × gender in their face-analysis example), because a single aggregate accuracy hides subgroups where the model fails. Datasheets for Datasets (Gebru et al., 2018) is the companion practice for data.

On Hugging Face, the model card is the repository's README.md: a YAML metadata header (licence, language, library, pipeline tag, datasets, base_model, and a model-index block with structured evaluation results) followed by free-text Markdown. The base_model field lets the hub build lineage graphs for fine-tunes, adapters and quantisations, which is useful for supply-chain review. Frontier labs additionally publish system cards, which describe a deployed system - the model plus safety mitigations, policies and the results of red teaming and dangerous-capability evaluations - rather than the model weights alone.

Regulation has turned parts of the card into obligations without prescribing the format. For general-purpose AI models, EU AI Act Art. 53(1)(b) and Annex XII require information for downstream providers, including intended tasks and acceptable-use policy, architecture and number of parameters, input and output modalities and formats, licence, and information on training data; Annex XI lists the fuller technical documentation owed to authorities. The General-Purpose AI Code of Practice (2025) supplies a Model Documentation Form covering these items. For high-risk systems, Art. 13 instructions for use and Art. 11 technical documentation (Annex IV) cover similar ground at system level. NIST AI RMF MAP and MEASURE functions expect comparable documentation of context, limitations and test results.

Common weaknesses: cards written once at release and never updated for new versions; benchmark results that are self-reported, measured under undisclosed settings or inflated by test-set contamination; missing disaggregated results, often because demographic labels are unavailable; vague out-of-scope sections; and silence on the training data, frequently for legal reasons. A deployer should therefore treat a model card as a supplier claim to verify, re-run evaluations on its own data and languages - Danish performance, for instance, is often weaker than headline English benchmarks suggest - and record the result in its own documentation.