mfrmr Visual Diagnostics

This vignette is a compact map of the main base-R diagnostics in mfrmr. It is organized around four practical questions:

All examples use packaged data and preset = "publication" so the same code is suitable for manuscript-oriented graphics.

If you are selecting figures for a report, use reporting_checklist() before or alongside this vignette. Its "Visual Displays" rows now mirror the public plotting family shown here.

Minimal setup

library(mfrmr)

toy <- load_mfrmr_data("example_core")

fit <- fit_mfrm(
  toy,
  person = "Person",
  facets = c("Rater", "Criterion"),
  score = "Score",
  method = "JML",
  model = "RSM",
  maxit = 20
)
#> Warning: Optimizer did not fully converge (code = 1, status = iteration_limit).
#> Optimizer reached the iteration limit before the terminal gradient became small
#> enough for review-only acceptance. Consider increasing maxit (current: 20) or
#> relaxing reltol (current: 1e-06).

diag <- diagnose_mfrm(fit, residual_pca = "none")
checklist <- reporting_checklist(fit, diagnostics = diag)
subset(
  checklist$checklist,
  Section == "Visual Displays",
  c("Item", "Available", "NextAction")
)
#>                                   Item Available
#> 21                          Wright map      TRUE
#> 22                QC / facet dashboard      TRUE
#> 23                Residual PCA visuals     FALSE
#> 24 Connectivity / design-matrix visual      TRUE
#> 25  Inter-rater / displacement visuals      TRUE
#> 26             Strict marginal visuals     FALSE
#> 27                  Bias / DIF visuals     FALSE
#> 28      Precision / information curves     FALSE
#> 29                Fit/category visuals      TRUE
#>                                                                                                                                NextAction
#> 21                                               Include a Wright map when the manuscript benefits from a shared-scale targeting display.
#> 22                              Use the dashboard as a first-pass triage view, then move to the specific follow-up plot behind each flag.
#> 23                                                  Run residual PCA if you want scree/loadings visuals for residual-structure follow-up.
#> 24                                                                Use the design-matrix view to support linkage and comparability claims.
#> 25                                                Use displacement and inter-rater views to localize QC issues after dashboard screening.
#> 26 For MML reporting runs, call diagnose_mfrm(..., diagnostic_mode = "both") to enable strict marginal follow-up visuals where supported.
#> 27                                                                 Run bias or DIF screening before discussing interaction-level visuals.
#> 28                                                         Resolve convergence before using information or precision curves in reporting.
#> 29                                                 Use category curves and fit visuals as local descriptive follow-up after QC screening.

1. Targeting and scale structure

Use the Wright map first when you want one shared logit view of persons, facet levels, and step thresholds.

plot(fit, type = "wright", preset = "publication", show_ci = TRUE)

Interpretation:

Compare person density on the left to facet and step locations on the right.
Large gaps suggest weaker targeting in that logit region.
Wide overlap in confidence whiskers means neighboring levels are not cleanly separated.

Next, use the pathway map when you want to see how expected scores progress across theta.

plot(fit, type = "pathway", preset = "publication")

Interpretation:

Steeper rises indicate stronger score progression.
Dominant-category strips show where each category is most likely to govern the score.
Flat or compressed regions suggest weaker category separation.

2. Local response and level issues

Unexpected-response screening is useful for case-level review.

plot_unexpected(
  fit,
  diagnostics = diag,
  abs_z_min = 1.5,
  prob_max = 0.4,
  plot_type = "scatter",
  preset = "publication"
)

Interpretation:

Upper corners combine large residual mismatch with low model probability.
Repeated appearances of the same persons or levels are more informative than a single extreme point.

Displacement focuses on level movement rather than individual responses.

plot_displacement(
  fit,
  diagnostics = diag,
  anchored_only = FALSE,
  plot_type = "lollipop",
  preset = "publication"
)

Interpretation:

Large absolute displacement indicates stronger tension between observed data and current calibration.
For anchored runs, this is especially useful as an anchor-robustness screen.

Strict marginal follow-up

When you need the package’s latent-integrated follow-up path, switch to MML and request diagnostic_mode = "both" so the legacy and strict branches stay visible side by side.

fit_strict <- fit_mfrm(
  toy,
  person = "Person",
  facets = c("Rater", "Criterion"),
  score = "Score",
  method = "MML",
  model = "RSM",
  quad_points = 7,
  maxit = 40
)

diag_strict <- diagnose_mfrm(
  fit_strict,
  residual_pca = "none",
  diagnostic_mode = "both"
)

strict_checklist <- reporting_checklist(fit_strict, diagnostics = diag_strict)
subset(
  strict_checklist$checklist,
  Section == "Visual Displays" &
    Item %in% c("QC / facet dashboard", "Strict marginal visuals"),
  c("Item", "Available", "NextAction")
)
#>                       Item Available
#> 22    QC / facet dashboard      TRUE
#> 26 Strict marginal visuals      TRUE
#>                                                                                                                       NextAction
#> 22                     Use the dashboard as a first-pass triage view, then move to the specific follow-up plot behind each flag.
#> 26 Treat strict marginal plots as exploratory corroboration screens, then corroborate with design review and legacy diagnostics.

plot_marginal_fit(
  diag_strict,
  top_n = 12,
  preset = "publication"
)

Interpretation:

Treat strict marginal plots as exploratory corroboration screens, not as standalone inferential tests.
Use the checklist rows to confirm that the current run actually supports the strict branch before routing figures into a report.
When pairwise follow-up is needed, continue with plot_marginal_pairwise(diag_strict, preset = "publication").

3. Linking and coverage

When the design may be incomplete or spread across subsets, inspect the coverage matrix before interpreting cross-subset contrasts.

sc <- subset_connectivity_report(fit, diagnostics = diag)
plot(sc, type = "design_matrix", preset = "publication")

Interpretation:

Sparse rows or columns indicate weak subset coverage.
Facets with low overlap are weaker anchors for cross-subset comparisons.

If you are working across administrations, follow up with anchor-drift plots:

drift <- detect_anchor_drift(current_fit, baseline = baseline_anchors)
plot_anchor_drift(drift, type = "heatmap", preset = "publication")

4. Residual structure and interaction screens

Residual PCA is a follow-up layer after the main fit screen.

diag_pca <- diagnose_mfrm(fit, residual_pca = "both", pca_max_factors = 4)
pca <- analyze_residual_pca(diag_pca, mode = "both")
plot_residual_pca(pca, mode = "overall", plot_type = "scree", preset = "publication")

Interpretation:

Early components with noticeably larger eigenvalues deserve follow-up.
Scree review should usually be paired with loading review for the component of interest.

For interaction screening, use the packaged bias example.

bias_df <- load_mfrmr_data("example_bias")

fit_bias <- fit_mfrm(
  bias_df,
  person = "Person",
  facets = c("Rater", "Criterion"),
  score = "Score",
  method = "MML",
  model = "RSM",
  quad_points = 7
)

diag_bias <- diagnose_mfrm(fit_bias, residual_pca = "none")
bias <- estimate_bias(fit_bias, diag_bias, facet_a = "Rater", facet_b = "Criterion")

plot_bias_interaction(
  bias,
  plot = "facet_profile",
  preset = "publication"
)

Interpretation:

Facet profiles are useful for seeing whether a small number of levels drives most flagged interaction cells.
Treat these plots as screening evidence; confirm with the corresponding tables and narrative reports.

Recommended sequence

For a compact visual workflow:

reporting_checklist() when you want the package to route which figures are already supported.
plot_qc_dashboard() for one-page triage.
plot_unexpected(), plot_displacement(), plot_marginal_fit(), and plot_interrater_agreement() for local follow-up.
plot(fit, type = "wright") and plot(fit, type = "pathway") for targeting and scale interpretation.
plot_residual_pca(), plot_bias_interaction(), and plot_information() for deeper structural review.

mfrmr Visual Diagnostics

Minimal setup

1. Targeting and scale structure

2. Local response and level issues

Strict marginal follow-up

3. Linking and coverage

4. Residual structure and interaction screens

Recommended sequence

Related help