Title:	Spectrally Deconfounded Models
Version:	1.0.13
Description:	Screen for and analyze non-linear sparse direct effects in the presence of unobserved confounding using the spectral deconfounding techniques (Ćevid, Bühlmann, and Meinshausen (2020)<jmlr.org/papers/v21/19-545.html>, Guo, Ćevid, and Bühlmann (2022) <doi:10.1214/21-AOS2152>). These methods have been shown to be a good estimate for the true direct effect if we observe many covariates, e.g., high-dimensional settings, and we have fairly dense confounding. Even if the assumptions are violated, it seems like there is not much to lose, and the deconfounded models will, in general, estimate a function closer to the true one than classical least squares optimization. 'SDModels' provides functions SDAM() for Spectrally Deconfounded Additive Models (Scheidegger, Guo, and Bühlmann (2025) <doi:10.1145/3711116>) and SDForest() for Spectrally Deconfounded Random Forests (Ulmer, Scheidegger, and Bühlmann (2025) <doi:10.48550/arXiv.2502.03969>).
License:	GPL-3
Imports:	data.tree, DiagrammeR, doParallel, future.apply, future, ggplot2, GPUmatrix, gridExtra, locatexec, parallel, pbapply, Rdpack, tidyr, fda, grplasso, rlang
Suggests:	plotly, datasets, rpart, knitr, rmarkdown, ranger, HDclassif, qpdf, igraph, testthat (≥ 3.0.0)
RdMacros:	Rdpack
Encoding:	UTF-8
RoxygenNote:	7.3.2
VignetteBuilder:	knitr
URL:	https://www.markus-ulmer.ch/SDModels/
BugReports:	https://github.com/markusul/SDModels/issues
Config/testthat/edition:	3
NeedsCompilation:	no
Packaged:	2025-06-05 15:28:56 UTC; maulmer
Author:	Markus Ulmer [aut, cre, cph], Cyrill Scheidegger [aut]
Maintainer:	Markus Ulmer <markus.ulmer@stat.math.ethz.ch>
Repository:	CRAN
Date/Publication:	2025-06-05 16:10:05 UTC

Spectrally Deconfounded Additive Models

Description

Estimate high-dimensional additive models using spectral deconfounding (Scheidegger et al. 2025). The covariates are expanded into B-spline basis functions. A spectral transformation is used to remove bias arising from hidden confounding and a group lasso objective is minimized to enforce component-wise sparsity. Optimal number of basis functions per component and sparsity penalty are chosen by cross validation.

Usage

SDAM(
  formula = NULL,
  data = NULL,
  x = NULL,
  y = NULL,
  Q_type = "trim",
  trim_quantile = 0.5,
  q_hat = 0,
  nfolds = 5,
  cv_method = "1se",
  n_K = 4,
  n_lambda1 = 10,
  n_lambda2 = 20,
  Q_scale = TRUE,
  ind_lin = NULL,
  mc.cores = 1,
  verbose = TRUE,
  notRegularized = NULL
)

Arguments

Value

An object of class 'SDAM' containing the following elements:

Author(s)

Cyrill Scheidegger

References

Guo Z, Ćevid D, Bühlmann P (2022). “Doubly debiased lasso: High-dimensional inference under hidden confounding.” The Annals of Statistics, 50(3). ISSN 0090-5364, doi:10.1214/21-AOS2152.

Paul D, Bair E, Hastie T, Tibshirani R (2008). ““Preconditioning” for feature selection and regression in high-dimensional problems.” The Annals of Statistics, 36(4). ISSN 0090-5364, doi:10.1214/009053607000000578.

Scheidegger C, Guo Z, Bühlmann P (2025). “Spectral Deconfounding for High-Dimensional Sparse Additive Models.” ACM / IMS J. Data Sci.. doi:10.1145/3711116.

Ćevid D, Bühlmann P, Meinshausen N (2020). “Spectral Deconfounding via Perturbed Sparse Linear Models.” J. Mach. Learn. Res., 21(1). ISSN 1532-4435, http://jmlr.org/papers/v21/19-545.html.

See Also

get_Q, predict.SDAM, varImp.SDAM, predict_individual_fj, partDependence

Examples

set.seed(1)
X <- matrix(rnorm(10 * 5), ncol = 5)
Y <- sin(X[, 1]) -  X[, 2] + rnorm(10)
model <- SDAM(x = X, y = Y, Q_type = "trim", trim_quantile = 0.5, nfold = 2, n_K = 1)

# if we know that the first covariate one is relevant, we can also choose to not regularize it
model <- SDAM(x = X, y = Y, Q_type = "trim", trim_quantile = 0.5, nfold = 2, 
              n_K = 1, notRegularized = c(1))


set.seed(22)
library(HDclassif)
data(wine)
names(wine) <- c("class", "alcohol", "malicAcid", "ash", "alcalinityAsh", "magnesium", 
                 "totPhenols", "flavanoids", "nonFlavPhenols", "proanthocyanins", 
                 "colIntens", "hue", "OD", "proline")
wine <- log(wine)

# estimate model
# do not use class in the model and restrict proline to be linear 
model <- SDAM(alcohol ~ . - class, wine, ind_lin = "proline")

# extract variable importance
varImp(model)

# most important variable
mostImp <- names(which.max(varImp(model)))
mostImp

# predict for individual Xj
x <- seq(min(wine[, mostImp]), max(wine[, mostImp]), length.out = 100)
predJ <- predict_individual_fj(object = model, j = mostImp, x = x)

plot(x, predJ, 
     xlab = paste0("log ", mostImp), ylab = "log alcohol")

# partial dependece
plot(partDependence(model, mostImp))

# predict 
predict(model, newdata = wine[42, ])

## alternative function call
mod_none <- SDAM(x = as.matrix(wine[1:10, -c(1, 2)]), y = wine$alcohol[1:10], 
                 Q_type = "no_deconfounding", nfolds = 2, n_K = 4, 
                 n_lambda1 = 4, n_lambda2 = 8)

Spectrally Deconfounded Random Forests

Description

Estimate regression Random Forest using spectral deconfounding. The spectrally deconfounded Random Forest (SDForest) combines SDTrees in the same way, as in the original Random Forest (Breiman 2001). The idea is to combine multiple regression trees into an ensemble in order to decrease variance and get a smooth function. Ensembles work best if the different models are independent of each other. To decorrelate the regression trees as much as possible from each other, we have two mechanisms. The first one is bagging (Breiman 1996), where we train each regression tree on an independent bootstrap sample of the observations, e.g., we draw a random sample of size n with replacement from the observations. The second mechanic to decrease the correlation is that only a random subset of the covariates is available for each split. Before each split, we sample \text{mtry} \leq p from all the covariates and choose the one that reduces the loss the most only from those.

\widehat{f(X)} = \frac{1}{N_{tree}} \sum_{t = 1}^{N_{tree}} SDTree_t(X)

Usage

SDForest(
  formula = NULL,
  data = NULL,
  x = NULL,
  y = NULL,
  nTree = 100,
  cp = 0,
  min_sample = 5,
  mtry = NULL,
  mc.cores = 1,
  Q_type = "trim",
  trim_quantile = 0.5,
  q_hat = 0,
  Qf = NULL,
  A = NULL,
  gamma = 7,
  max_size = NULL,
  gpu = FALSE,
  return_data = TRUE,
  mem_size = 1e+07,
  leave_out_ind = NULL,
  envs = NULL,
  nTree_leave_out = NULL,
  nTree_env = NULL,
  max_candidates = 100,
  Q_scale = TRUE,
  verbose = TRUE,
  predictors = NULL
)

Arguments

Value

Object of class SDForest containing:

If return_data is TRUE the following are also returned:

If envs is provided the following are also returned:

Author(s)

Markus Ulmer

References

Breiman L (1996). “Bagging predictors.” Machine Learning, 24(2), 123–140. ISSN 0885-6125, doi:10.1007/BF00058655.

Breiman L (2001). “Random Forests.” Machine Learning, 45(1), 5–32. ISSN 08856125, doi:10.1023/A:1010933404324.

Guo Z, Ćevid D, Bühlmann P (2022). “Doubly debiased lasso: High-dimensional inference under hidden confounding.” The Annals of Statistics, 50(3). ISSN 0090-5364, doi:10.1214/21-AOS2152.

Paul D, Bair E, Hastie T, Tibshirani R (2008). ““Preconditioning” for feature selection and regression in high-dimensional problems.” The Annals of Statistics, 36(4). ISSN 0090-5364, doi:10.1214/009053607000000578.

Ćevid D, Bühlmann P, Meinshausen N (2020). “Spectral Deconfounding via Perturbed Sparse Linear Models.” J. Mach. Learn. Res., 21(1). ISSN 1532-4435, http://jmlr.org/papers/v21/19-545.html.

See Also

get_Q, get_W, SDTree, simulate_data_nonlinear, regPath, stabilitySelection, prune, partDependence

Examples

set.seed(1)
n <- 50
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + rnorm(n)
model <- SDForest(x = X, y = y, Q_type = 'no_deconfounding', nTree = 5, cp = 0.5)
predict(model, newdata = data.frame(X))

###### subset of predictors
# if we know, that only the first covariate has an effect on y,
# we can estimate only its effect and use the others just for deconfounding
model <- SDForest(x = X, y = y, cp = 0.5, nTree = 5, predictors = c(1))


set.seed(42)
# simulation of confounded data
sim_data <- simulate_data_nonlinear(q = 2, p = 150, n = 100, m = 2)
X <- sim_data$X
Y <- sim_data$Y
train_data <- data.frame(X, Y)
# causal parents of y
sim_data$j

# comparison to classical random forest
fit_ranger <- ranger::ranger(Y ~ ., train_data, importance = 'impurity')

fit <- SDForest(x = X, y = Y, nTree = 100, Q_type = 'pca', q_hat = 2)
fit <- SDForest(Y ~ ., nTree = 100, train_data)
fit

# we can plot the fit to see whether the number of trees is high enough
# if the performance stabilizes, we have enough trees otherwise one can fit
# more and add them
plot(fit)

# a few more might be helpfull
fit2 <- SDForest(Y ~ ., nTree = 50, train_data) 
fit <- mergeForest(fit, fit2)

# comparison of variable importance
imp_ranger <- fit_ranger$variable.importance
imp_sdf <- fit$var_importance
imp_col <- rep('black', length(imp_ranger))
imp_col[sim_data$j] <- 'red'

plot(imp_ranger, imp_sdf, col = imp_col, pch = 20,
     xlab = 'ranger', ylab = 'SDForest', 
     main = 'Variable Importance')

# check regularization path of variable importance
path <- regPath(fit)
# out of bag error for different regularization
plotOOB(path)
plot(path)

# detection of causal parent using stability selection
stablePath <- stabilitySelection(fit)
plot(stablePath)

# pruning of forest according to optimal out-of-bag performance
fit <- prune(fit, cp = path$cp_min)

# partial functional dependence of y on the most important covariate
most_imp <- which.max(fit$var_importance)
dep <- partDependence(fit, most_imp)
plot(dep, n_examples = 100)

Spectrally Deconfounded Tree

Description

Estimates a regression tree using spectral deconfounding. A regression tree is part of the function class of step functions f(X) = \sum_{m = 1}^M 1_{\{X \in R_m\}} c_m, where (R_m) with m = 1, \ldots, M are regions dividing the space of \mathbb{R}^p into M rectangular parts. Each region has response level c_m \in \mathbb{R}. For the training data, we can write the step function as f(\mathbf{X}) = \mathcal{P} c where \mathcal{P} \in \{0, 1\}^{n \times M} is an indicator matrix encoding to which region an observation belongs and c \in \mathbb{R}^M is a vector containing the levels corresponding to the different regions. This function then minimizes

(\hat{\mathcal{P}}, \hat{c}) = \text{argmin}_{\mathcal{P}' \in \{0, 1\}^{n \times M}, c' \in \mathbb{R}^ {M}} \frac{||Q(\mathbf{Y} - \mathcal{P'} c')||_2^2}{n}

We find \hat{\mathcal{P}} by using the tree structure and repeated splitting of the leaves, similar to the original cart algorithm (Breiman et al. 2017). Since comparing all possibilities for \mathcal{P} is impossible, we let a tree grow greedily. Given the current tree, we iterate over all leaves and all possible splits. We choose the one that reduces the spectral loss the most and estimate after each split all the leave estimates \hat{c} = \text{argmin}_{c' \in \mathbb{R}^M} \frac{||Q\mathbf{Y} - Q\mathcal{P} c'||_2^2}{n} which is just a linear regression problem. This is repeated until the loss decreases less than a minimum loss decrease after a split. The minimum loss decrease equals a cost-complexity parameter cp times the initial loss when only an overall mean is estimated. The cost-complexity parameter cp controls the complexity of a regression tree and acts as a regularization parameter.

Usage

SDTree(
  formula = NULL,
  data = NULL,
  x = NULL,
  y = NULL,
  max_leaves = NULL,
  cp = 0.01,
  min_sample = 5,
  mtry = NULL,
  fast = TRUE,
  Q_type = "trim",
  trim_quantile = 0.5,
  q_hat = 0,
  Qf = NULL,
  A = NULL,
  gamma = 0.5,
  gpu = FALSE,
  mem_size = 1e+07,
  max_candidates = 100,
  Q_scale = TRUE,
  predictors = NULL
)

Arguments

Value

Object of class SDTree containing

Author(s)

Markus Ulmer

References

Breiman L, Friedman JH, Olshen RA, Stone CJ (2017). Classification And Regression Trees. Routledge. ISBN 9781315139470, doi:10.1201/9781315139470.

Glur C (2023). “data.tree: General Purpose Hierarchical Data Structure.” https://CRAN.R-project.org/package=data.tree.

Guo Z, Ćevid D, Bühlmann P (2022). “Doubly debiased lasso: High-dimensional inference under hidden confounding.” The Annals of Statistics, 50(3). ISSN 0090-5364, doi:10.1214/21-AOS2152.

Paul D, Bair E, Hastie T, Tibshirani R (2008). ““Preconditioning” for feature selection and regression in high-dimensional problems.” The Annals of Statistics, 36(4). ISSN 0090-5364, doi:10.1214/009053607000000578.

Ćevid D, Bühlmann P, Meinshausen N (2020). “Spectral Deconfounding via Perturbed Sparse Linear Models.” J. Mach. Learn. Res., 21(1). ISSN 1532-4435, http://jmlr.org/papers/v21/19-545.html.

See Also

simulate_data_nonlinear, regPath.SDTree, prune.SDTree, partDependence

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + rnorm(n)
model <- SDTree(x = X, y = y, cp = 0.5)

###### subset of predictors
# if we know, that only the first covariate has an effect on y,
# we can estimate only its effect and use the others just for deconfounding
model <- SDTree(x = X, y = y, cp = 0.5, predictors = c(1))


set.seed(42)
# simulation of confounded data
sim_data <- simulate_data_step(q = 2, p = 15, n = 100, m = 2)
X <- sim_data$X
Y <- sim_data$Y
train_data <- data.frame(X, Y)
# causal parents of y
sim_data$j

tree_plain_cv <- cvSDTree(Y ~ ., train_data, Q_type = "no_deconfounding")
tree_plain <- SDTree(Y ~ ., train_data, Q_type = "no_deconfounding", cp = 0)

tree_causal_cv <- cvSDTree(Y ~ ., train_data)
tree_causal <- SDTree(y = Y, x = X, cp = 0)

# check regularization path of variable importance
path <- regPath(tree_causal)
plot(path)

tree_plain <- prune(tree_plain, cp = tree_plain_cv$cp_min)
tree_causal <- prune(tree_causal, cp = tree_causal_cv$cp_min)
plot(tree_causal)
plot(tree_plain)

Copy a forest

Description

Returns a copy of the forest object. Might be useful if you want to keep the original forest in comparison to the pruned forest.

Usage

## S3 method for class 'SDForest'
copy(object, ...)

Arguments

Value

A copy of the SDForest object

Author(s)

Markus Ulmer

See Also

prune

Examples

set.seed(1)
X <- matrix(rnorm(10 * 20), nrow = 10)
Y <- rnorm(10)
fit <- SDForest(x = X, y = Y, nTree = 2, cp = 0.5)
fit2 <- copy(fit)

Copy a tree

Description

Returns a copy of the tree object. Might be useful if you want to keep the original tree in comparison to the pruned tree.

Usage

## S3 method for class 'SDTree'
copy(object, ...)

Arguments

Value

A copy of the SDTree object

Author(s)

Markus Ulmer

See Also

prune

Examples

set.seed(1)
X <- matrix(rnorm(10 * 20), nrow = 10)
Y <- rnorm(10)
fit <- SDTree(x = X, y = Y, cp = 0.5)
fit2 <- copy(fit)

Cross-validation for the SDTree

Description

Estimates the optimal complexity parameter for the SDTree using cross-validation. The transformations are estimated for each training set and validation set separately to ensure independence of the validation set.

Usage

cvSDTree(
  formula = NULL,
  data = NULL,
  x = NULL,
  y = NULL,
  max_leaves = NULL,
  cp = 0,
  min_sample = 5,
  mtry = NULL,
  fast = TRUE,
  Q_type = "trim",
  trim_quantile = 0.5,
  q_hat = 0,
  Qf = NULL,
  A = NULL,
  gamma = 0.5,
  gpu = FALSE,
  mem_size = 1e+07,
  max_candidates = 100,
  nfolds = 3,
  cp_seq = NULL,
  mc.cores = 1,
  Q_scale = TRUE
)

Arguments

Value

A list containing

Author(s)

Markus Ulmer

References

Guo Z, Ćevid D, Bühlmann P (2022). “Doubly debiased lasso: High-dimensional inference under hidden confounding.” The Annals of Statistics, 50(3). ISSN 0090-5364, doi:10.1214/21-AOS2152.

Paul D, Bair E, Hastie T, Tibshirani R (2008). ““Preconditioning” for feature selection and regression in high-dimensional problems.” The Annals of Statistics, 36(4). ISSN 0090-5364, doi:10.1214/009053607000000578.

Ćevid D, Bühlmann P, Meinshausen N (2020). “Spectral Deconfounding via Perturbed Sparse Linear Models.” J. Mach. Learn. Res., 21(1). ISSN 1532-4435, http://jmlr.org/papers/v21/19-545.html.

See Also

SDTree prune.SDTree regPath.SDTree

Examples

set.seed(1)
n <- 50
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + rnorm(n, 0, 5)
cp <- cvSDTree(x = X, y = y, Q_type = 'no_deconfounding')
cp

Function of x on a fourier basis

Description

Function of x on a fourier basis with a subset of covariates having a causal effect on Y using the parameters beta. The function is given by:

f(x_i) = \sum_{j = 1}^p 1_{j \in js} \sum_{k = 1}^K (\beta_{j, k}^{(1)} \cos(0.2 k x_j) + \beta_{j, k}^{(2)} \sin(0.2 k x_j))

Usage

f_four(x, beta, js)

Arguments

Value

the value of the function f(x)

Author(s)

Markus Ulmer

See Also

simulate_data_nonlinear

Examples

set.seed(42)
# simulation of confounded data
sim_data <- simulate_data_nonlinear(q = 2, p = 150, n = 100, m = 2)
X <- sim_data$X
j <- sim_data$j[1]
apply(X, 1, function(x) f_four(x, sim_data$beta, j))

SDForest fromList method

Description

Converts the trees in an SDForest object from class list to class Node (Glur 2023).

Usage

## S3 method for class 'SDForest'
fromList(object, ...)

Arguments

Value

an SDForest object with the trees in Node format

Author(s)

Markus Ulmer

References

Glur C (2023). “data.tree: General Purpose Hierarchical Data Structure.” https://CRAN.R-project.org/package=data.tree.

See Also

fromList fromList.SDTree

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + rnorm(n)
model <- SDForest(x = X, y = y, Q_type = 'no_deconfounding', cp = 0.5, nTree = 2)
fromList(toList(model))

SDTree fromList method

Description

Converts the tree in an SDTree object from class list to class Node (Glur 2023).

Usage

## S3 method for class 'SDTree'
fromList(object, ...)

Arguments

Value

an SDTree object with the tree in Node format

Author(s)

Markus Ulmer

References

Glur C (2023). “data.tree: General Purpose Hierarchical Data Structure.” https://CRAN.R-project.org/package=data.tree.

See Also

toList

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + rnorm(n)
model <- SDTree(x = X, y = y, Q_type = 'no_deconfounding', cp = 0.5)
fromList(toList(model))

Estimation of spectral transformation

Description

Estimates the spectral transformation Q for spectral deconfounding by shrinking the leading singular values of the covariates.

Usage

get_Q(X, type, trim_quantile = 0.5, q_hat = 0, gpu = FALSE, scaling = TRUE)

Arguments

Value

Q of class matrix, the spectral transformation matrix.

Author(s)

Markus Ulmer

References

Guo Z, Ćevid D, Bühlmann P (2022). “Doubly debiased lasso: High-dimensional inference under hidden confounding.” The Annals of Statistics, 50(3). ISSN 0090-5364, doi:10.1214/21-AOS2152.

Paul D, Bair E, Hastie T, Tibshirani R (2008). ““Preconditioning” for feature selection and regression in high-dimensional problems.” The Annals of Statistics, 36(4). ISSN 0090-5364, doi:10.1214/009053607000000578.

Ćevid D, Bühlmann P, Meinshausen N (2020). “Spectral Deconfounding via Perturbed Sparse Linear Models.” J. Mach. Learn. Res., 21(1). ISSN 1532-4435, http://jmlr.org/papers/v21/19-545.html.

Examples

set.seed(1)
X <- matrix(rnorm(50 * 20), nrow = 50)
Q_trim <- get_Q(X, 'trim')
Q_pca <- get_Q(X, 'pca', q_hat = 5)
Q_plain <- get_Q(X, 'no_deconfounding')

Estimation of anchor transformation

Description

Estimates the anchor transformation for the Anchor-Objective. The anchor transformation is W = I-(1-\sqrt{\gamma}))\Pi_A, where \Pi_A = A(A^TA)^{-1}A^T. For \gamma = 1 this is just the identity. For \gamma = 0 this corresponds to residuals after orthogonal projecting onto A. For large \gamma this is close to the orthogonal projection onto A, scaled by \gamma. The estimator \text{argmin}_f ||W(Y - f(X))||^2 corresponds to the Anchor-Regression Estimator (Rothenhäusler et al. 2021), (Bühlmann 2020).

Usage

get_W(A, gamma, intercept = FALSE, gpu = FALSE)

Arguments

Value

W of class matrix, the anchor transformation matrix.

Author(s)

Markus Ulmer

References

Bühlmann P (2020). “Invariance, Causality and Robustness.” Statistical Science, 35(3). ISSN 0883-4237, doi:10.1214/19-STS721.

Rothenhäusler D, Meinshausen N, Bühlmann P, Peters J (2021). “Anchor Regression: Heterogeneous Data Meet Causality.” Journal of the Royal Statistical Society Series B: Statistical Methodology, 83(2), 215–246. ISSN 1369-7412, doi:10.1111/rssb.12398.

Examples

set.seed(1)
n <- 50
X <- matrix(rnorm(n * 1), nrow = n)
Y <- 3 * X + rnorm(n)
W <- get_W(X, gamma = 0)
resid <- W %*% Y

Get the sequence of complexity parameters of an SDForest

Description

This function extracts the sequence of complexity parameters of an SDForest that result in changes of the SDForest if pruned. Only cp values that differ in the first three digits after the decimal point are returned.

Usage

## S3 method for class 'SDForest'
get_cp_seq(object, ...)

Arguments

Value

A sequence of complexity parameters

Author(s)

Markus Ulmer

See Also

regPath stabilitySelection get_cp_seq.SDTree

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + rnorm(n)
model <- SDForest(x = X, y = y, Q_type = 'no_deconfounding', cp = 0, nTree = 2)
get_cp_seq(model)

Get the sequence of complexity parameters of an SDTree

Description

This function extracts the sequence of complexity parameters of an SDTree that result in changes of the tree structure if pruned. Only cp values that differ in the first three digits after the decimal point are returned.

Usage

## S3 method for class 'SDTree'
get_cp_seq(object, ...)

Arguments

Value

A sequence of complexity parameters

Author(s)

Markus Ulmer

See Also

regPath stabilitySelection

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + rnorm(n)
model <- SDTree(x = X, y = y, Q_type = 'no_deconfounding', cp = 0)
get_cp_seq(model)

Merge two forests

Description

This function merges two forests. The trees are combined and the variable importance is calculated as a weighted average of the two forests. If the forests are trained on the same data, the predictions and oob_predictions are combined as well.

Usage

mergeForest(fit1, fit2)

Arguments

Value

merged SDForest object set.seed(1) n <- 50 X <- matrix(rnorm(n * 5), nrow = n) y <- sign(X[, 1]) * 3 + rnorm(n) fit1 <- SDForest(x = X, y = y, Q_type = 'no_deconfounding', nTree = 5, cp = 0.5) fit2 <- SDForest(x = X, y = y, nTree = 5, cp = 0.5) mergeForest(fit1, fit2)

Author(s)

Markus Ulmer

Partial dependence

Description

This function calculates the partial dependence of a model on a single variable. For that predictions are made for all observations in the dataset while varying the value of the variable of interest. The overall partial effect is the average of all predictions. (Friedman 2001)

Usage

partDependence(object, j, X = NULL, subSample = NULL, mc.cores = 1)

Arguments

Value

An object of class partDependence containing

Author(s)

Markus Ulmer

References

Friedman JH (2001). “Greedy Function Approximation: A Gradient Boosting Machine.” The Annals of Statistics, 29(5), 1189–1232. ISSN 00905364, http://www.jstor.org/stable/2699986.

See Also

SDForest, SDTree

Examples

set.seed(1)
x <- rnorm(100)
y <- sign(x) * 3 + rnorm(100)
model <- SDTree(x = x, y = y, Q_type = 'no_deconfounding')
pd <- partDependence(model, 1, X = x, subSample = 10)
plot(pd)

Plot performance of SDForest against number of trees

Description

This plot helps to analyze whether enough trees were used. If the loss does not stabilize one can fit another SDForest and merge the two.

Usage

## S3 method for class 'SDForest'
plot(x, ...)

Arguments

Value

A ggplot object

Author(s)

Markus Ulmer

See Also

SDForest set.seed(1) n <- 10 X <- matrix(rnorm(n * 5), nrow = n) y <- sign(X[, 1]) * 3 + rnorm(n) model <- SDForest(x = X, y = y, Q_type = 'no_deconfounding', cp = 0.5, nTree = 500) plot(model)

Plot SDTree

Description

Plot the SDTree.

Usage

## S3 method for class 'SDTree'
plot(x, ...)

Arguments

Value

graph object from DiagrammeR::render_graph()

Author(s)

Markus Ulmer

See Also

SDTree set.seed(1) n <- 10 X <- matrix(rnorm(n * 5), nrow = n) y <- sign(X[, 1]) * 3 + rnorm(n) model <- SDTree(x = X, y = y, Q_type = 'no_deconfounding', cp = 0.5) plot(model)

Plot partial dependence

Description

This function plots the partial dependence of a model on a single variable.

Usage

## S3 method for class 'partDependence'
plot(x, n_examples = 19, ...)

Arguments

Value

A ggplot object.

Author(s)

Markus Ulmer

See Also

partDependence set.seed(1) x <- rnorm(10) y <- sign(x) * 3 + rnorm(10) model <- SDTree(x = x, y = y, Q_type = 'no_deconfounding', cp = 0.5) pd <- partDependence(model, 1, X = x) plot(pd)

Visualize the paths of an SDTree or SDForest

Description

This function visualizes the variable importance of an SDTree or SDForest for different complexity parameters. Both the regularization path and the stability selection path can be visualized.

Usage

## S3 method for class 'paths'
plot(x, plotly = FALSE, selection = NULL, sqrt_scale = FALSE, ...)

Arguments

Value

A ggplot object with the variable importance for different regularization. If the path object includes a cp_min value, a black dashed line is added to indicate the out-of-bag optimal variable selection.

Author(s)

Markus Ulmer

See Also

regPath stabilitySelection

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + sign(X[, 2]) + rnorm(n)
model <- SDTree(x = X, y = y, Q_type = 'no_deconfounding', cp = 0.5)
paths <- regPath(model)
plot(paths)

plot(paths, plotly = TRUE)

Visualize the out-of-bag performance of an SDForest

Description

This function visualizes the out-of-bag performance of an SDForest for different complexity parameters. Can be used to choose the optimal complexity parameter.

Usage

plotOOB(object, sqrt_scale = FALSE)

Arguments

Value

A ggplot object

Author(s)

Markus Ulmer

See Also

regPath.SDForest

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + sign(X[, 2]) + rnorm(n)
model <- SDForest(x = X, y = y, Q_type = 'no_deconfounding', cp = 0.5)
paths <- regPath(model)
plotOOB(paths)

Predictions for SDAM

Description

Predicts the response for new data using a fitted SDAM.

Usage

## S3 method for class 'SDAM'
predict(object, newdata, ...)

Arguments

Value

A vector of predictions for the new data.

Author(s)

Cyrill Scheidegger

See Also

SDAM

Examples

set.seed(1)
X <- matrix(rnorm(10 * 5), ncol = 5)
Y <- sin(X[, 1]) -  X[, 2] + rnorm(10)
model <- SDAM(x = X, y = Y, Q_type = "trim", trim_quantile = 0.5, nfold = 2, n_K = 1)
predict(model, newdata = data.frame(X))

Predictions for the SDForest

Description

Predicts the response for new data using a fitted SDForest.

Usage

## S3 method for class 'SDForest'
predict(object, newdata, mc.cores = 1, ...)

Arguments

Value

A vector of predictions for the new data.

Author(s)

Markus Ulmer

See Also

SDForest

Examples

set.seed(1)
n <- 50
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + rnorm(n)
model <- SDForest(x = X, y = y, Q_type = 'no_deconfounding', nTree = 5, cp = 0.5)
predict(model, newdata = data.frame(X))

Predictions for the SDTree

Description

Predicts the response for new data using a fitted SDTree.

Usage

## S3 method for class 'SDTree'
predict(object, newdata, ...)

Arguments

Value

A vector of predictions for the new data.

Author(s)

Markus Ulmer

See Also

SDTree

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + rnorm(n)
model <- SDTree(x = X, y = y, Q_type = 'no_deconfounding', cp = 0.5)
predict(model, newdata = data.frame(X))

Out-of-bag predictions for the SDForest

Description

Predicts the response for the training data using only the trees in the SDForest that were not trained on the observation.

Usage

predictOOB(object, X = NULL)

Arguments

Value

A vector of out-of-bag predictions for the training data. #' set.seed(1) n <- 50 X <- matrix(rnorm(n * 5), nrow = n) y <- sign(X[, 1]) * 3 + rnorm(n) model <- SDForest(x = X, y = y, Q_type = 'no_deconfounding', nTree = 5, cp = 0.5) predictOOB(model)

Author(s)

Markus Ulmer

See Also

SDForest prune.SDForest plotOOB

Predictions of individual component functions for SDAM

Description

Predicts the contribution of an individual component j using a fitted SDAM.

Usage

predict_individual_fj(object, j, x = NULL)

Arguments

Value

A vector of predictions for fj evaluated at Xjnew.

Author(s)

Cyrill Scheidegger

See Also

SDAM

Examples

set.seed(1)
X <- matrix(rnorm(10 * 5), ncol = 5)
Y <- sin(X[, 1]) -  X[, 2] + rnorm(10)
model <- SDAM(x = X, y = Y, Q_type = "trim", trim_quantile = 0.5, nfold = 2, n_K = 1)
predict_individual_fj(model, j = 1, seq(-2, 2, length.out = 100))

Print SDAM

Description

Print number of covariates and number of active covariates for SDAM object.

Usage

## S3 method for class 'SDAM'
print(x, ...)

Arguments

Value

No return value, called for side effects

Author(s)

Cyrill Scheidegger

See Also

SDAM

Examples

set.seed(1)
X <- matrix(rnorm(10 * 5), ncol = 5)
Y <- sin(X[, 1]) -  X[, 2] + rnorm(10)
model <- SDAM(x = X, y = Y, Q_type = "trim", trim_quantile = 0.5, nfold = 2, n_K = 1)
print(model)

Print SDForest

Description

Print contents of the SDForest.

Usage

## S3 method for class 'SDForest'
print(x, ...)

Arguments

Value

No return value, called for side effects

Author(s)

Markus Ulmer

See Also

SDForest

Examples

set.seed(1)
n <- 50
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + rnorm(n)
model <- SDForest(x = X, y = y, Q_type = 'no_deconfounding', nTree = 5, cp = 0.5)
print(model)

Print a SDTree

Description

Print contents of the SDTree.

Usage

## S3 method for class 'SDTree'
print(x, ...)

Arguments

Value

No return value, called for side effects

Author(s)

Markus Ulmer

See Also

SDTree

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + rnorm(n)
model <- SDTree(x = X, y = y, Q_type = 'no_deconfounding', cp = 0.5)
print(model)

Print partDependence

Description

Print contents of the partDependence.

Usage

## S3 method for class 'partDependence'
print(x, ...)

Arguments

Value

No return value, called for side effects

Author(s)

Markus Ulmer

See Also

partDependence, plot.partDependence

Examples

set.seed(1)
x <- rnorm(10)
y <- sign(x) * 3 + rnorm(10)
model <- SDTree(x = x, y = y, Q_type = 'no_deconfounding', cp = 0.5)
pd <- partDependence(model, 1, X = x)
print(pd)

Prune an SDForest

Description

Prunes all trees in the forest and re-calculates the out-of-bag predictions and performance measures. The training data is needed to calculate the out-of-bag statistics. Note that the forest is pruned in place. If you intend to keep the original forest, make a copy of it before pruning.

Usage

## S3 method for class 'SDForest'
prune(object, cp, X = NULL, Y = NULL, Q = NULL, pred = TRUE, ...)

Arguments

Value

A pruned SDForest object

Author(s)

Markus Ulmer

See Also

copy prune.SDTree regPath

Examples

set.seed(1)
X <- matrix(rnorm(10 * 20), nrow = 10)
Y <- rnorm(10)
fit <- SDForest(x = X, y = Y, nTree = 2)
pruned_fit <- prune(copy(fit), 0.2)

Prune an SDTree

Description

Removes all nodes that did not improve the loss by more than cp times the initial loss. Either by themselves or by one of their successors. Note that the tree is pruned in place. If you intend to keep the original tree, make a copy of it before pruning.

Usage

## S3 method for class 'SDTree'
prune(object, cp, ...)

Arguments

Value

A pruned SDTree object

Author(s)

Markus Ulmer

See Also

copy

Examples

set.seed(1)
X <- matrix(rnorm(10 * 20), nrow = 10)
Y <- rnorm(10)
tree <- SDTree(x = X, y = Y)
pruned_tree <- prune(copy(tree), 0.2)
tree
pruned_tree

Calculate the regularization path of an SDForest

Description

This function calculates the variable importance of an SDForest and the out-of-bag performance for different complexity parameters.

Usage

## S3 method for class 'SDForest'
regPath(object, cp_seq = NULL, X = NULL, Y = NULL, Q = NULL, copy = TRUE, ...)

Arguments

Value

An object of class paths containing

Author(s)

Markus Ulmer

See Also

plot.paths plotOOB regPath.SDTree prune get_cp_seq SDForest

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + sign(X[, 2]) + rnorm(n)
model <- SDForest(x = X, y = y, Q_type = 'no_deconfounding', cp = 0.5)
paths <- regPath(model)
plotOOB(paths)
plot(paths)

plot(paths, plotly = TRUE)

Calculate the regularization path of an SDTree

Description

This function calculates the variable importance of an SDTree for different complexity parameters.

Usage

## S3 method for class 'SDTree'
regPath(object, cp_seq = NULL, ...)

Arguments

Value

An object of class paths containing

Author(s)

Markus Ulmer

See Also

plot.paths prune get_cp_seq SDTree

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + sign(X[, 2]) + rnorm(n)
model <- SDTree(x = X, y = y, Q_type = 'no_deconfounding', cp = 0.5)
paths <- regPath(model)
plot(paths)

plot(paths, plotly = TRUE)

Simulate data with linear confounding and non-linear causal effect

Description

Simulation of data from a confounded non-linear model. The data generating process is given by:

Y = f(X) + \delta^T H + \nu

X = \Gamma^T H + E

where f(X) is a random function on the fourier basis with a subset of size m covariates X_j having a causal effect on Y.

f(x_i) = \sum_{j = 1}^p 1_{j \in js} \sum_{k = 1}^K (\beta_{j, k}^{(1)} \cos(0.2 k x_j) + \beta_{j, k}^{(2)} \sin(0.2 k x_j))

E, \nu are random error terms and H \in \mathbb{R}^{n \times q} is a matrix of random confounding covariates. \Gamma \in \mathbb{R}^{q \times p} and \delta \in \mathbb{R}^{q} are random coefficient vectors. For the simulation, all the above parameters are drawn from a standard normal distribution, except for \nu which is drawn from a normal distribution with standard deviation 0.1. The parameters \beta are drawn from a uniform distribution between -1 and 1.

Usage

simulate_data_nonlinear(q, p, n, m, K = 2, eff = NULL, fixEff = FALSE)

Arguments

Value

a list containing the simulated data:

Author(s)

Markus Ulmer

See Also

f_four

Examples

set.seed(42)
# simulation of confounded data
sim_data <- simulate_data_nonlinear(q = 2, p = 150, n = 100, m = 2)
X <- sim_data$X
Y <- sim_data$Y

Simulate data with linear confounding and causal effect following a step-function

Description

Simulation of data from a confounded non-linear model. Where the non-linear function is a random regression tree. The data generating process is given by:

Y = f(X) + \delta^T H + \nu

X = \Gamma^T H + E

where f(X) is a random regression tree with m random splits of the data. Resulting in a random step-function with m+1 levels, i.e. leaf-levels.

f(x_i) = \sum_{k = 1}^K 1_{\{x_i \in R_k\}} c_k

E, \nu are random error terms and H \in \mathbb{R}^{n \times q} is a matrix of random confounding covariates. \Gamma \in \mathbb{R}^{q \times p} and \delta \in \mathbb{R}^{q} are random coefficient vectors. For the simulation, all the above parameters are drawn from a standard normal distribution, except for \delta which is drawn from a normal distribution with standard deviation 10. The leaf levels c_k are drawn from a uniform distribution between -50 and 50.

Usage

simulate_data_step(q, p, n, m, make_tree = FALSE)

Arguments

Value

a list containing the simulated data:

Author(s)

Markus Ulmer

References

Glur C (2023). “data.tree: General Purpose Hierarchical Data Structure.” https://CRAN.R-project.org/package=data.tree.

See Also

simulate_data_nonlinear

Examples

set.seed(42)
# simulation of confounded data
sim_data <- simulate_data_step(q = 2, p = 15, n = 100, m = 2)
X <- sim_data$X
Y <- sim_data$Y

Calculate the stability selection of an SDForest

Description

This function calculates the stability selection of an SDForest (Meinshausen and Bühlmann 2010). Stability selection is calculated as the fraction of trees in the forest that select a variable for a split at each complexity parameter.

Usage

## S3 method for class 'SDForest'
stabilitySelection(object, cp_seq = NULL, ...)

Arguments

Value

An object of class paths containing

Author(s)

Markus Ulmer

References

Meinshausen N, Bühlmann P (2010). “Stability Selection.” Journal of the Royal Statistical Society Series B: Statistical Methodology, 72(4), 417–473. ISSN 1369-7412, doi:10.1111/j.1467-9868.2010.00740.x.

See Also

plot.paths regPath prune get_cp_seq SDForest

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + sign(X[, 2]) + rnorm(n)
model <- SDForest(x = X, y = y, Q_type = 'no_deconfounding', nTree = 2, cp = 0.5)
paths <- stabilitySelection(model)
plot(paths)

plot(paths, plotly = TRUE)

SDForest toList method

Description

Converts the trees in an SDForest object from class Node (Glur 2023) to class list. This makes it substantially easier to save the forest to disk.

Usage

## S3 method for class 'SDForest'
toList(object, ...)

Arguments

Value

an SDForest object with the trees in list format

Author(s)

Markus Ulmer

References

Glur C (2023). “data.tree: General Purpose Hierarchical Data Structure.” https://CRAN.R-project.org/package=data.tree.

See Also

fromList toList.SDTree

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + rnorm(n)
model <- SDForest(x = X, y = y, Q_type = 'no_deconfounding', cp = 0.5, nTree = 2)
toList(model)

SDTree toList method

Description

Converts the tree in an SDTree object from class Node (Glur 2023) to class list. This makes it substantially easier to save the tree to disk.

Usage

## S3 method for class 'SDTree'
toList(object, ...)

Arguments

Value

an SDTree object with the tree in list format

Author(s)

Markus Ulmer

References

Glur C (2023). “data.tree: General Purpose Hierarchical Data Structure.” https://CRAN.R-project.org/package=data.tree.

See Also

fromList

Examples

set.seed(1)
n <- 10
X <- matrix(rnorm(n * 5), nrow = n)
y <- sign(X[, 1]) * 3 + rnorm(n)
model <- SDTree(x = X, y = y, Q_type = 'no_deconfounding', cp = 0.5)
toList(model)

Extract Variable importance for SDAM

Description

This function extracts the variable importance of an SDAM. The variable importance is calculated as the empirical squared L2 norm of fj. The measure is not standardized.

Usage

## S3 method for class 'SDAM'
varImp(object)

Arguments

Value

A vector of variable importance

Author(s)

Cyrill Scheidegger

See Also

SDAM

Examples

set.seed(1)
X <- matrix(rnorm(10 * 5), ncol = 5)
Y <- sin(X[, 1]) -  X[, 2] + rnorm(10)
model <- SDAM(x = X, y = Y, Q_type = "trim", trim_quantile = 0.5, nfold = 2)
varImp(model)

Extract variable importance of an SDForest

Description

This function extracts the variable importance of an SDForest. The variable importance is calculated as the sum of the decrease in the loss function resulting from all splits that use a covariate for each tree. The mean of the variable importance of all trees results in the variable importance for the forest.

Usage

## S3 method for class 'SDForest'
varImp(object)

Arguments

Value

A named vector of variable importance

Author(s)

Markus Ulmer

See Also

varImp.SDTree SDForest

Examples

data(iris)
fit <- SDForest(Sepal.Length ~ Sepal.Width + Petal.Length + Petal.Width, 
                 iris, nTree = 10, cp = 0.5)
varImp(fit)

Extract variable importance of an SDTree

Description

This function extracts the variable importance of an SDTree. The variable importance is calculated as the sum of the decrease in the loss function resulting from all splits that use this covariate.

Usage

## S3 method for class 'SDTree'
varImp(object)

Arguments

Value

A named vector of variable importance

Author(s)

Markus Ulmer

See Also

varImp.SDForest SDTree

Examples

data(iris)
tree <- SDTree(Sepal.Length ~ Sepal.Width + Petal.Length + Petal.Width, iris, cp = 0.5)
varImp(tree)