Explainable ‘AI’ utilizing Gradient Boosted randomized networks Pt2 (the Lasso)

This put up is about LSBoost, an Explainable ‘AI’ algorithm which makes use of Gradient Boosted randomized networks for sample recognition. As we’ve mentioned it final week LSBoost is a cousin of GFAGBM’s LS_Boost. In LSBoost, extra particularly, the so referred to as weak learners from LS_Boost are based mostly on randomized neural networks’ parts and variants of Least Squares regression fashions.

I’ve already offered some promising examples of use of LSBoost based mostly on Ridge Regression weak learners. In mlsauce’s model 0.7.1, the Lasso can be used in its place ingredient to the weak learners. Here’s a comparability of the regression coefficients obtained through the use of mlsauce’s implementation of Ridge regression and the Lasso:

image-title-here

image-title-here

The next instance is about coaching set error vs testing set error, as a perform of the regularization parameter, each for Ridge regression and Lasso-based weak learners.

Packages and information


# 0 - Packages and information ------------------------------------------------------- library(devtools)
devtools::install_github("thierrymoudiki/mlsauce/R-package")
library(mlsauce)
library(datasets) print(abstract(datasets::mtcars)) X <- as.matrix(datasets::mtcars[, -1])
y <- as.integer(datasets::mtcars[, 1]) n <- dim(X)[1]
p <- dim(X)[2]
set.seed(21341)
train_index <- pattern(x = 1:n, dimension = flooring(0.8*n), exchange = TRUE)
test_index <- -train_index
X_train <- as.matrix(X[train_index, ])
y_train <- as.double(y[train_index])
X_test <- as.matrix(X[test_index, ])
y_test <- as.double(y[test_index]) 

LSBoost utilizing Ridge regression


# 1 - Ridge ------------------------------------------------------------------- obj <- mlsauce::LSBoostRegressor() # default h is Ridge
print(obj$get_params()) n_lambdas <- 100
lambdas <- 10**seq(from=-6, to=6, size.out = n_lambdas)
rmse_matrix <- matrix(NA, nrow = 2, ncol = n_lambdas)
rownames(rmse_matrix) <- c("coaching rmse", "testing rmse") for (j in 1:n_lambdas)
{ obj$set_params(reg_lambda = lambdas[j]) obj$match(X_train, y_train) rmse_matrix[, j] <- c(sqrt(imply((obj$predict(X_train) - y_train)**2)), sqrt(imply((obj$predict(X_test) - y_test)**2)))
} 

image-title-here

image-title-here

LSBoost utilizing the Lasso


# 2 - Lasso ------------------------------------------------------------------- obj <- mlsauce::LSBoostRegressor(solver = "lasso")
print(obj$get_params()) n_lambdas <- 100
lambdas <- 10**seq(from=-6, to=6, size.out = n_lambdas)
rmse_matrix2 <- matrix(NA, nrow = 2, ncol = n_lambdas)
rownames(rmse_matrix2) <- c("coaching rmse", "testing rmse") for (j in 1:n_lambdas)
{ obj$set_params(reg_lambda = lambdas[j]) obj$match(X_train, y_train) rmse_matrix2[, j] <- c(sqrt(imply((obj$predict(X_train) - y_train)**2)), sqrt(imply((obj$predict(X_test) - y_test)**2)))
} 

image-title-here

image-title-here

R session data


> print(session_info())
 Session data ───────────────────────────────────────────────────────────── setting worth model R model 4.0.2 (2020-06-22) os Ubuntu 16.04.6 LTS system x86_64, linux-gnu ui RStudio language (EN) collate C.UTF-8 ctype C.UTF-8 tz And many others/UTC date 2020-07-31  Packages ───────────────────────────────────────────────────────────────── bundle * model date lib supply assertthat 0.2.1 2019-03-21 [1] RSPM (R 4.0.2) backports 1.1.8 2020-06-17 [1] RSPM (R 4.0.2) callr 3.4.3 2020-03-28 [1] RSPM (R 4.0.2) cli 2.0.2 2020-02-28 [1] RSPM (R 4.0.2) crayon 1.3.4 2017-09-16 [1] RSPM (R 4.0.2) curl 4.3 2019-12-02 [1] RSPM (R 4.0.2) desc 1.2.0 2018-05-01 [1] RSPM (R 4.0.2) devtools * 2.3.1 2020-07-21 [1] RSPM (R 4.0.2) digest 0.6.25 2020-02-23 [1] RSPM (R 4.0.2) ellipsis 0.3.1 2020-05-15 [1] RSPM (R 4.0.2) fansi 0.4.1 2020-01-08 [1] RSPM (R 4.0.2) fs 1.4.2 2020-06-30 [1] RSPM (R 4.0.2) glue 1.4.1 2020-05-13 [1] RSPM (R 4.0.2) jsonlite 1.7.0 2020-06-25 [1] RSPM (R 4.0.2) lattice 0.20-41 2020-04-02 [2] CRAN (R 4.0.2) magrittr 1.5 2014-11-22 [1] RSPM (R 4.0.2) Matrix 1.2-18 2019-11-27 [2] CRAN (R 4.0.2) memoise 1.1.0 2017-04-21 [1] RSPM (R 4.0.2) mlsauce * 0.7.1 2020-07-31 [1] Github (thierrymoudiki/mlsauce@68e391a) pkgbuild 1.1.0 2020-07-13 [1] RSPM (R 4.0.2) pkgload 1.1.0 2020-05-29 [1] RSPM (R 4.0.2) prettyunits 1.1.1 2020-01-24 [1] RSPM (R 4.0.2) processx 3.4.3 2020-07-05 [1] RSPM (R 4.0.2) ps 1.3.3 2020-05-08 [1] RSPM (R 4.0.2) R6 2.4.1 2019-11-12 [1] RSPM (R 4.0.2) rappdirs 0.3.1 2016-03-28 [1] RSPM (R 4.0.2) Rcpp 1.0.5 2020-07-06 [1] RSPM (R 4.0.2) remotes 2.2.0 2020-07-21 [1] RSPM (R 4.0.2) reticulate 1.16 2020-05-27 [1] RSPM (R 4.0.2) rlang 0.4.7 2020-07-09 [1] RSPM (R 4.0.2) rprojroot 1.3-2 2018-01-03 [1] RSPM (R 4.0.2) rstudioapi 0.11 2020-02-07 [1] RSPM (R 4.0.2) sessioninfo 1.1.1 2018-11-05 [1] RSPM (R 4.0.2) testthat 2.3.2 2020-03-02 [1] RSPM (R 4.0.2) usethis * 1.6.1 2020-04-29 [1] RSPM (R 4.0.2) withr 2.2.0 2020-04-20 [1] RSPM (R 4.0.2) [1] /dwelling/rstudio-consumer/R/x86_64-laptop-linux-gnu-library/4.0
[2] /choose/R/4.0.2/lib/R/library 

No put up in August

Leave a Reply

Your email address will not be published. Required fields are marked *