Run Additional Optimization Iterations

Description

Use this function to continue optimization of a bayesOpt object.

Usage

addIterations(
  optObj,
  iters.n = 1,
  iters.k = 1,
  otherHalting = list(timeLimit = Inf, minUtility = 0),
  bounds = optObj$bounds,
  acq = optObj$optPars$acq,
  kappa = optObj$optPars$kappa,
  eps = optObj$optPars$eps,
  gsPoints = optObj$optPars$gsPoints,
  convThresh = optObj$optPars$convThresh,
  acqThresh = optObj$optPars$acqThresh,
  errorHandling = "stop",
  saveFile = optObj$saveFile,
  parallel = FALSE,
  plotProgress = FALSE,
  verbose = 1,
  ...
)

Arguments

Details

By default, this function uses the original parameters used to create optObj, however the parameters (including the bounds) can be customized. If new bounds are used which cause some of the prior runs to fall outside of the bounds, these samples are removed from the optimization procedure, but will remain in scoreSummary. FUN should return the same elements and accept the same inputs as the original, or this function may fail.

Value

An object of class bayesOpt having run additional iterations.

Examples

scoringFunction <- function(x) {
  a <- exp(-(2-x)^2)*1.5
  b <- exp(-(4-x)^2)*2
  c <- exp(-(6-x)^2)*1
  return(list(Score = a+b+c))
}

bounds <- list(x = c(0,8))

Results <- bayesOpt(
    FUN = scoringFunction
  , bounds = bounds
  , initPoints = 3
  , iters.n = 1
  , gsPoints = 10
)
Results <- addIterations(Results,iters.n=1)

Bayesian Optimization with Gaussian Processes

Description

Maximizes a user defined function within a set of bounds. After the function is sampled a pre-determined number of times, a Gaussian process is fit to the results. An acquisition function is then maximized to determine the most likely location of the global maximum of the user defined function. This process is repeated for a set number of iterations.

Usage

bayesOpt(
  FUN,
  bounds,
  saveFile = NULL,
  initGrid,
  initPoints = 4,
  iters.n = 3,
  iters.k = 1,
  otherHalting = list(timeLimit = Inf, minUtility = 0),
  acq = "ucb",
  kappa = 2.576,
  eps = 0,
  parallel = FALSE,
  gsPoints = pmax(100, length(bounds)^3),
  convThresh = 1e+08,
  acqThresh = 1,
  errorHandling = "stop",
  plotProgress = FALSE,
  verbose = 1,
  ...
)

Arguments

Value

An object of class bayesOpt containing information about the process.

• FUN The scoring function.

• bounds The bounds originally supplied.

• iters The total iterations that have been run.

• initPars The initialization parameters.

• optPars The optimization parameters.

• GauProList A list containing information on the Gaussian Processes used in optimization.

• scoreSummary A data.table with results from the execution of FUN at different inputs. Includes information on the epoch, iteration, function inputs, score, and any other information returned by FUN.

• stopStatus Information on what caused the function to stop running. Possible explenations are time limit, minimum utility not met, errors in FUN, iters.n was reached, or the Gaussian Process encountered an error.

• elapsedTime The total time in seconds the function was executing.

Vignettes

It is highly recommended to read the GitHub for examples. There are also several vignettes available from the official CRAN Listing.

References

Jasper Snoek, Hugo Larochelle, Ryan P. Adams (2012) Practical Bayesian Optimization of Machine Learning Algorithms

Examples

# Example 1 - Optimization of a continuous single parameter function
scoringFunction <- function(x) {
  a <- exp(-(2-x)^2)*1.5
  b <- exp(-(4-x)^2)*2
  c <- exp(-(6-x)^2)*1
  return(list(Score = a+b+c))
}

bounds <- list(x = c(0,8))

Results <- bayesOpt(
    FUN = scoringFunction
  , bounds = bounds
  , initPoints = 3
  , iters.n = 2
  , gsPoints = 10
)

## Not run: 
# Example 2 - Hyperparameter Tuning in xgboost
if (requireNamespace('xgboost', quietly = TRUE)) {
  library("xgboost")

  data(agaricus.train, package = "xgboost")

  Folds <- list(
      Fold1 = as.integer(seq(1,nrow(agaricus.train$data),by = 3))
    , Fold2 = as.integer(seq(2,nrow(agaricus.train$data),by = 3))
    , Fold3 = as.integer(seq(3,nrow(agaricus.train$data),by = 3))
  )

  scoringFunction <- function(max_depth, min_child_weight, subsample) {

    dtrain <- xgb.DMatrix(agaricus.train$data,label = agaricus.train$label)

    Pars <- list(
        booster = "gbtree"
      , eta = 0.01
      , max_depth = max_depth
      , min_child_weight = min_child_weight
      , subsample = subsample
      , objective = "binary:logistic"
      , eval_metric = "auc"
    )

    xgbcv <- xgb.cv(
         params = Pars
       , data = dtrain
       , nround = 100
       , folds = Folds
       , prediction = TRUE
       , showsd = TRUE
       , early_stopping_rounds = 5
       , maximize = TRUE
       , verbose = 0
    )

    return(
      list(
          Score = max(xgbcv$evaluation_log$test_auc_mean)
        , nrounds = xgbcv$best_iteration
      )
    )
  }

  bounds <- list(
      max_depth = c(2L, 10L)
    , min_child_weight = c(1, 100)
    , subsample = c(0.25, 1)
  )

  ScoreResult <- bayesOpt(
      FUN = scoringFunction
    , bounds = bounds
    , initPoints = 3
    , iters.n = 2
    , iters.k = 1
    , acq = "ei"
    , gsPoints = 10
    , parallel = FALSE
    , verbose = 1
  )
}

## End(Not run)

Change Save File Location

Description

Use this to change the saveFile parameter in a pre-existing bayesOpt object.

Usage

changeSaveFile(optObj, saveFile = NULL)

Arguments

Value

The same optObj with the updated saveFile.

Examples

## Not run: 
scoringFunction <- function(x) {
  a <- exp(-(2-x)^2)*1.5
  b <- exp(-(4-x)^2)*2
  c <- exp(-(6-x)^2)*1
  return(list(Score = a+b+c))
}

bounds <- list(x = c(0,8))

Results <- bayesOpt(
    FUN = scoringFunction
  , bounds = bounds
  , initPoints = 3
  , iters.n = 2
  , gsPoints = 10
  , saveFile = "filepath.RDS"
)
Results <- changeSaveFile(Results,saveFile = "DifferentFile.RDS")

## End(Not run)

Get the Best Parameter Set

Description

Returns the N parameter sets which resulted in the maximum scores from FUN.

Usage

getBestPars(optObj, N = 1)

Arguments

Value

A list containing the FUN inputs which resulted in the highest returned Score. If N > 1, a data.table is returned. Each row is a result from FUN, with results ordered by descending Score.

Examples

scoringFunction <- function(x) {
  a <- exp(-(2-x)^2)*1.5
  b <- exp(-(4-x)^2)*2
  c <- exp(-(6-x)^2)*1
  return(list(Score = a+b+c))
}

bounds <- list(x = c(0,8))

Results <- bayesOpt(
    FUN = scoringFunction
  , bounds = bounds
  , initPoints = 3
  , iters.n = 2
  , gsPoints = 10
)
print(getBestPars(Results))

Get Local Optimums of acq From a bayesOpt Object

Description

Returns all local optimums of the acquisition function, no matter the utility.

Usage

getLocalOptimums(
  optObj,
  bounds = optObj$bounds,
  acq = optObj$optPars$acq,
  kappa = optObj$optPars$kappa,
  eps = optObj$optPars$eps,
  convThresh = optObj$optPars$convThresh,
  gsPoints = optObj$optPars$gsPoints,
  parallel = FALSE,
  verbose = 1
)

Arguments

Details

gsPoints points in the parameter space are randomly initialized, and the L-BFGS-B method is used to find the closest local optimum to each point. dbscan is then used to cluster points together which converged to the same optimum - only unique optimums are returned.

Value

A data table of local optimums, including the utility (gpUtility), the utility relative to the max utility (relUtility), and the steps taken in the L-BFGS-B method (gradCount).

Examples

scoringFunction <- function(x) {
  a <- exp(-(2-x)^2)*1.5
  b <- exp(-(4-x)^2)*2
  c <- exp(-(6-x)^2)*1
  return(list(Score = a+b+c))
}

bounds <- list(x = c(0,8))

Results <- bayesOpt(
    FUN = scoringFunction
  , bounds = bounds
  , initPoints = 3
  , iters.n = 2
  , gsPoints = 10
)
print(getLocalOptimums(Results))

Plot a bayesOpt object

Description

Returns 2 stacked plots - the top shows the results from FUN at each iteration. The bottom shows the utility from each point before the search took place.

Usage

## S3 method for class 'bayesOpt'
plot(x, ...)

Arguments

Value

an object of class ggarrange from the ggpubr package.

Examples

scoringFunction <- function(x) {
  a <- exp(-(2-x)^2)*1.5
  b <- exp(-(4-x)^2)*2
  c <- exp(-(6-x)^2)*1
  return(list(Score = a+b+c))
}

bounds <- list(x = c(0,8))

Results <- bayesOpt(
    FUN = scoringFunction
  , bounds = bounds
  , initPoints = 3
  , iters.n = 2
  , gsPoints = 10
)
# This plot will also show in real time with parameter plotProgress = TRUE in bayesOpt()
plot(Results)

Print a bayesOpt object

Description

Print a bayesOpt object

Usage

## S3 method for class 'bayesOpt'
print(x, ...)

Arguments

Value

NULL

Update Gaussian Processes in a bayesOpt Object

Description

To save time, Gaussian processes are not updated after the last iteration in addIterations(). The user can do this manually, using this function if they wish. This is not necessary to continue optimization using addIterations.

Usage

updateGP(optObj, bounds = optObj$bounds, verbose = 1, ...)

Arguments

Value

An object of class bayesOpt with updated Gaussian processes.

Examples

# Create initial object
scoringFunction <- function(x) {
  a <- exp(-(2-x)^2)*1.5
  b <- exp(-(4-x)^2)*2
  c <- exp(-(6-x)^2)*1
  return(list(Score = a+b+c))
}

bounds <- list(x = c(0,8))

Results <- bayesOpt(
    FUN = scoringFunction
  , bounds = bounds
  , initPoints = 3
  , iters.n = 2
  , gsPoints = 10
)

# At this point, the Gaussian Process has not been updated
# with the most recent results. We can update it manually:
Results <- updateGP(Results)