README

modelcardr audits predictions from a model that has already run. It does not retrain the model and does not need the fitted model object.

Install

Open modelcardr.Rproj in RStudio and choose Build > Install and Restart, or install the built source package:

install.packages("modelcardr.tar.gz", repos = NULL, type = "source")
library(modelcardr)

Binary classification

audit <- audit_predictions(
  truth = actual_class,
  probability = positive_class_probability,
  positive = "yes",
  bootstrap = 500
)
render_audit(audit, "binary-audit.html")

The audit includes classification metrics, bootstrap intervals, confusion matrix, threshold trade-offs, ROC and precision-recall curves, probability calibration, optional subgroup checks, and optional probability drift.

Multiclass classification

audit <- audit_predictions(
  truth = actual_class,
  probability = class_probability_matrix,
  task = "multiclass_classification",
  bootstrap = 500
)
render_audit(audit, "multiclass-audit.html")

The audit includes overall metrics, macro and weighted F1, minimum class recall, multiclass log loss and Brier score, one-vs-rest AUC, confusion matrix, class-level metrics, top-label calibration, and subgroup comparisons.

Regression

audit <- audit_predictions(
  truth = actual_value,
  estimate = predicted_value,
  task = "regression",
  bootstrap = 500
)
render_audit(audit, "regression-audit.html")

The audit includes MAE, RMSE, MSE, R-squared, bias, MAPE, SMAPE, correlation, bootstrap intervals, actual-versus-predicted diagnostics, residual analysis, and subgroup error comparisons.

Automatic task detection

task = "auto" is the default. Character and factor outcomes are treated as classification. Numeric truth and numeric estimates are treated as regression. For numeric class codes, set task explicitly.

Pass/fail requirements

rules <- audit_requirements(
  min_accuracy = 0.80,
  min_macro_f1 = 0.75,
  max_expected_calibration_error = 0.05
)

audit <- audit_predictions(
  truth = actual_class,
  probability = class_probability_matrix,
  task = "multiclass_classification",
  requirements = rules
)

audit$overall_status

Regression rules include max_rmse, max_mae, min_r_squared, max_absolute_bias, max_mape, and max_smape.