L1 Regularization for "lightgbm" engine in tidymodels

Question

How can I insert a regularization parameter in tidymodels for boost_tree()? In the normal lightgbm package there is the tuning parameter lambda_l1. I would like to use this in tidymodels as well.

I tried to code it like this, but I am unsure if I am doing right:

lgbm_model <-
        boost_tree(
            mode = "regression",
            # mtry = 1,               
            trees = tune(),           
            min_n = tune(),           
            tree_depth = tune(),       
            learn_rate = tune(),      
            loss_reduction = tune()   
        ) %>%
        set_engine("lightgbm", lambda = 1)

Answer 1

Since lambda_l1 isn't a main argument for boost_tree(), you would indeed supply that argument to set_engine(). Supply it exactly as it is named in lightgbm::lgb.train() or it's param argument, using lambda_l1. The bonsai package, which implements support for the "lightgbm" engine with parsnip, will take care of passing that argument to the right place.

Your code would look something like this:

library(tidymodels)
library(bonsai)

lgb <-
  boost_tree(mode = "regression",) %>%
  set_engine("lightgbm", lambda_l1 = .9)

lgb_fit <- fit(lgb, mpg ~ ., mtcars)

lgb_fit
#> parsnip model object
#> 
#> LightGBM Model (1 tree)
#> Objective: regression
#> Fitted to dataset with 10 columns

To confirm that lambda_l1 was supplied as you intended, you can extract the underlying LightGBM fit and poke inside of it:

lgb_fit_engine <- extract_fit_engine(lgb_fit)
lgb_fit_engine$params$lambda_l1
#> [1] 0.9

^{Created on 2024-03-28 with reprex v2.1.0}

:)