Implicit Stochastic Gradient Descent Optimizer

This optimizer performs an implicit stochastic gradient descent algorithm. It is mainly used within a bamlss call.

opt_isgd(x, y, family, weights = NULL, offset = NULL,
     gammaFun = function(i) 1/(1 + i), shuffle = TRUE,
     CFun = function(beta) diag(length(beta)),
     start = NULL, i.state = 0)

Arguments

x

For function boost() the x list, as returned from function bamlss.frame, holding all model matrices and other information that is used for fitting the model.

y

The model response, as returned from function bamlss.frame.

family

A bamlss family object, see family.bamlss.

weights

Prior weights on the data, as returned from function bamlss.frame.

offset

Can be used to supply model offsets for use in fitting, returned from function bamlss.frame.

gammaFun

Function specifying the step length.

shuffle

Should the data be shuffled?

CFun

Hessian approximating function.

start

Vector of starting values.

i.state

Added to gammaFUN().

Details

tpf

Value

For function opt_isgd() a list containing the following objects:

fitted.values

A named list of the fitted values based on the last iteration of the modeled parameters of the selected distribution.

parameters

A matrix, each row corresponds to the parameter values of one iteration.

sgd.summary

The summary of the stochastic gradient descent algorithm which can be printed and plotted.

References

Toulis, P and Airoldi, EM (2015): Scalable estimation strategies based on stochastic approximations: Classical results and new insights. Statistics and Computing, 25, no. 4, 781--795. doi: 10.1007/s11222-015-9560-y

Author

Thorsten Simon

Note

Motivated by the lecture 'Regression modelling with large data sets' given by Ioannis Kosmidis in Innsbruck, January 2017.

Warning

CAUTION: Arguments weights and offset are not implemented yet!

See also

bamlss.frame, bamlss

Examples

if (FALSE) {
set.seed(111)
d <- GAMart(n = 10000)
f <- num ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b <- bamlss(f, data = d, optimizer = opt_isgd, sampler = FALSE)
plot(b, ask = F)

## loop over observations a 2nd time
b <- bamlss(f, data = d, optimizer = opt_isgd, sampler = FALSE, start = parameters(b),
            i.state = b$model.stats$optimizer$sgd.summary$i.state)
plot(b, ask = F)

## try differeent gammaFuns, e.g.,
# gammaFun <- function(i) .3/sqrt((1+i)) + 0.001

## testing some families
f2 <- bin ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b2 <- bamlss(f2, data = d, optimizer = opt_isgd, sampler = FALSE, family = "binomial")

f3 <- cens ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b3 <- bamlss(f3, data = d, optimizer = opt_isgd, sampler = FALSE, family = "cnorm")
}