Alternating Optimization

This function performs alternating optimization of the function f.

Usage

ao(
  f,
  p,
  ...,
  partition = as.list(1:length(p)),
  base_optimizer = optimizeR::Optimizer$new("stats::optim"),
  iterations = 10,
  tolerance = 1e-06,
  f_partition = vector(mode = "list", length = length(partition)),
  joint_end = FALSE,
  verbose = FALSE
)

Arguments

f

A function to be optimized, returning a single numeric. The first argument of f must be a numeric of the same length as p followed by any other arguments specified by the ... argument.

p

A numeric vector, the starting parameter values for the optimization.

...

Additional arguments to be passed to f.

partition

A list of vectors of indices of p, specifying the partition of the parameter vector in the alternating optimization process. The default is as.list(1:length(p)), i.e. each parameter is optimized separately. Parameter indices can be members of multiple subsets.

base_optimizer

An Optimizer object, which can be created via Optimizer. It numerically solves the optimization problems in the partitions.

iterations

An integer, the maximum number of iterations through partitions before the alternating optimization process is terminated. Can also be Inf, in which case tolerance is responsible for the termination. The default is 10.

tolerance

A non-negative numeric. The alternating optimization terminates prematurely (i.e., before interations is reached) if the euclidean distance between the current estimate and the one from the last iteration is smaller than tolerance. The default is 1e-6.

f_partition

A list of the same length as partition. The i-th element can be a function that computes the value of f for the i-th parameter set defined in partition. The function must be of the form function(theta_part, theta_rest, ...), where

theta_part receives the parameter set for the current partition (this argument can be named differently),
theta_rest receives the remaining parameters (this argument must be named theta_rest),
... receives the additional arguments to f. Alternatively, it can be NULL, in which case f is used.

joint_end

If TRUE, the parameter set is optimized jointly after the alternating optimization process is terminated. The default is FALSE.

verbose

If TRUE, full tracing details are printed during the alternating optimization process. The default is FALSE.

Value

A list with the elements

estimate, the optimal parameter vector found,
value, the value of f at estimate,
sequence, a data.frame of the function values, estimates and computation times in the single iterations and partitions,
and seconds, the overall computation time in seconds.

Examples

# definition of the Himmelblau function
himmelblau <- function(x) (x[1]^2 + x[2] - 11)^2 + (x[1] + x[2]^2 - 7)^2

# alternating minimization separately for x_1 and x_2
# parameter restriction: -5 <= x_1, x_2 <= 5
ao(
  f = himmelblau, p = c(0, 0), partition = list(1, 2), iterations = Inf,
  base_optimizer = optimizeR::Optimizer$new(
    which = "stats::optim", lower = -5, upper = 5, method = "L-BFGS-B"
  )
)
#> $value
#> [1] 1.940035e-12
#> 
#> $estimate
#> [1]  3.584428 -1.848126
#> 
#> $sequence
#>    iteration partition        value      seconds       p1        p2
#> 1          0        NA 1.700000e+02 0.0000000000 0.000000  0.000000
#> 2          1         1 1.327270e+01 0.0168607235 3.395691  0.000000
#> 3          1         2 1.743666e+00 0.0021073818 3.395691 -1.803183
#> 4          2         1 2.847292e-02 0.0016138554 3.581412 -1.803183
#> 5          2         2 4.687472e-04 0.0013971329 3.581412 -1.847412
#> 6          3         1 7.368063e-06 0.0024566650 3.584381 -1.847412
#> 7          3         2 1.157612e-07 0.0009458065 3.584381 -1.848115
#> 8          4         1 1.900153e-09 0.0009291172 3.584427 -1.848115
#> 9          4         2 4.221429e-11 0.0007297993 3.584427 -1.848126
#> 10         5         1 3.598278e-12 0.0007421970 3.584428 -1.848126
#> 11         5         2 1.940035e-12 0.0007402897 3.584428 -1.848126
#> 
#> $seconds
#> [1] 0.02852297
#>