Model Audit

The purpose is to test whether a model is used within the grounds, purpose, and domain of validity that can be documented. The audit may be applied to mathematical, statistical, physical, economic, legal, administrative, linguistic, and machine-learning models, but its questions and evidential requirements must be adapted to the relevant field.

Model Audit concerns the relation between model and object. It does not decide in advance whether the model is true, false, realistic, useful, or misleading. A model may be highly useful without being a complete account of its object, and an idealised model may carry genuine knowledge where its limits are visible and correspondence is tested.

Model Audit becomes imprecise where model, data, representation, calculation, simulation, and actuality are treated as interchangeable. The distinctions below are functional: the same artefact may perform several roles, but those roles must then be documented separately.

That a model incorporates relations, parameters, or dynamic rules does not make it empty. But ontological content does not follow from symbols or calculation alone. Assignment to actuality requires a visible correspondence relation, traceable evidence, and testing against relevant alternatives.

The mandate must identify the particular model, version, and use to be tested. It is not sufficient to state that the task is to ‘audit the model’ or ‘expose model capture.’

The same model may be defensible for one purpose and indefensible for another. The mandate must therefore bound objective, population, time period, decision context, and the kind of conclusion actually being assessed.

The model should be traceable from object through data selection, definitions, measurement, filtering, preprocessing, assumptions, structure, and parameters to output and final claim. Traceability does not require every stage to be simple, but it does require each stage to be identifiable and open to testing.

Inherited coordinate systems, defaults, training data, classifications, and software libraries may carry assumptions not visible in the final product. They must be registered where relevant to the finding.

• Record the source of observations, measurements, training data, or register data.
• Document definitions, units, coordinate systems, and selection criteria.
• Distinguish fixed parameters from estimated, calibrated, and learned parameters.
• Record transformations, filtering, normalisation, aggregation, and missingness.
• Identify software version, rounding, numerical methods, and known implementation deviations.
• Show how output is translated into text, figure, category, decision, or ontological claim.
• Register stages that are unavailable for inspection, reproduction, or independent testing.

The questions should open the model to testing without building in that it must be false or capturing. Answers must be tied to documentation, tests, and concrete use cases.

Internal consistency shows that the model coheres under its own rules, but not by itself that it corresponds to actuality. External correspondence is strengthened where the model meets material not merely used to construct or calibrate it.

Predictive strength is relevant evidence but must be assessed together with purpose, baseline, alternatives, uncertainty, data leakage, and whether the prediction was specified before the outcome was known. Operational success may provide strong support, but it does not automatically confirm every interpretation of the model’s parts.

Model capture is not another name for model use, mathematics, idealisation, or professional agreement. The risk pattern becomes relevant where the model gains governing priority over its object, determines what may count as relevant grounds, and protects that priority against corrective observation or testing.

An F6 finding may be recorded where several concrete indications point in this direction but decisive conditions for F7 remain undocumented. F7 requires documentation of the complete relation among model, substitution, governing operation, resistance to correction, and consequence.

Model Audit uses the common finding categories F1–F7. The finding must state which model, output, use, and claim were assessed, which validation exists, and which uncertainty or limitation remains.

A finding should distinguish error in data, implementation, model structure, parameter, interpretation, and use. That a model is bounded or idealised is not itself an error where the limitation is relevant, open, and kept within the domain of validity.

Correction must answer to the bounded finding. Its purpose is not to remove models or demand model-free practice, but to restore precise use, visible uncertainty, relevant counter-testing, and traceable assignment to actuality.

A model may be corrected through data, definition, structure, parameter, implementation, documentation, interface, decision rule, or validity boundary. The measure must address the stage to which the finding actually belongs.

• Correct errors in data, units, code, parameters, or transformations and document the effect.
• Clearly separate model output, professional interpretation, and final decision.
• Expose assumptions, uncertainty, data coverage, and domain of validity at the point of use.
• Add independent validation, competing models, sensitivity analysis, or human review where risk requires it.
• Bound or suspend extrapolation without documented grounds.
• Change the decision rule where model effect exceeds what validation supports.
• Permit affected parties to provide relevant individual or case-specific grounds not captured by the model.
• Define responsibility, deadline, review, rollback, and closure criteria.

Model Audit may itself become model-capturing where the auditor’s own categories, competing model, or preferred ontology is made incorrigible. The audit must therefore test its own mandate, evidence selection, and language under the same requirements.

The method can expose traceability failures, inferential leaps, and validity problems, but cannot by itself settle every professional question. Claims about statistical identifiability, physical mechanism, software safety, or regulatory conformity may require specialist competence and separate standards.

A Model Audit must be documented in the common audit record or an equivalent traceable format. The record should show mandate, model version, model chain, data sources, assumptions, tests, alternatives, findings, contradiction, correction, and planned review.

Review must examine whether correction actually improved correspondence, robustness, intelligibility, and decision quality, and whether it reduced risk without creating a new model or manual practice with the same incorrigible function.