Python LightGBM error: joblib.externals.loky.process_executor.TerminatedWorkerError {SIGSEGV(-11)}

Question

I am utilizing Microsoft’s lightgbm (lgbm) library. Whilst my lgbm script is VERY similar to my scripts for xgboost and random forests (which both work fine), I appear to consistently get the following error on both the Mac Book Pro and MacStudios (with M1 chips) when using lgbm:

joblib.externals.loky.process_executor.TerminatedWorkerError: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker.

The exit codes of the workers are {SIGSEGV(-11)}

Relevant Code:

_train_x, _val_x, _train_y, _val_y = train_test_split(_train_x, _train_y, test_size = 0.2)
    
lgbm_model = LGBMClassifier(bagging_fraction = 0.75, bagging_freq = 5, random_state=42, verbose=-1, force_col_wise=True)
    
kfoldcv = StratifiedKFold(n_splits=3, shuffle=True, random_state=7)
    
lgbm_random_search = RandomizedSearchCV(estimator = lgbm_model, param_distributions = self._param_dict, n_iter = self.num_searches, cv = kfoldcv, verbose=2, random_state=42, n_jobs=-1)
    
lgbm_random_search.fit(_train_x, _train_y)

self._CrossVal_largest_accscore = lgbm_random_search.best_score_
    
lgbm_model = LGBMClassifier(n_jobs=-1, verbose=-1, force_col_wise=True, bagging_fraction = 0.75, bagging_freq = 5, **lgbm_random_search.best_params_)

lgbm_model.fit(_train_x, _train_y, callbacks=[early_stopping(50), log_evaluation(50)], eval_set=[(_val_x,_val_y)])

NB when I simply remove the clause njobs=-1 my program just terminates when running the line:

lgbm_random_search.fit(_train_x, _train_y)

Environment:

System Software Overview:

System Version: macOS 14.0 (23A344)
Kernel Version: Darwin 23.0.0
Boot Volume: Macintosh 
HDBoot Mode: Normal
Secure Virtual Memory: Enabled
System Integrity Protection: Enabled

Hardware Overview:

  Model Name: MacBook Pro
  Model Number: MK1F3B/A
  Chip: Apple M1 Pro
  Total Number of Cores: 10 (8 performance and 2 efficiency)
  Memory: 16 GB
  System Firmware Version: 10151.1.1
  OS Loader Version: 10151.1.1
  Activation Lock Status: Enabled

Application Software

Visual Studio Code==1.72.2
python==3.10.121

Python Packages

---

anaconda-client==1.12.0
anaconda-navigator==2.4.2 
conda==23.7.2
conda-build==3.26.0
joblib==1.3.0
lightgbm==4.0.0
matplotlib==3.7.1
matplotlib-inline==0.1.6
numpy==1.23.5
pandas==2.0.3
scikit-image==0.20.0
scikit-learn==1.3.0
scipy==1.11.1
statsmodels==0.14.0
sympy==1.12
xgboost==2.0.0

I have reviewed and attempted some of the suggested solutions in the following websites to no avail (eg removing njobs argument; adding ‘pre_dispatch=2’ argument; reinstalling anaconda, lightgbm, joblib; using a lower number of estimators 30 to 300 etc):

• GridSearchCV with n_jobs=-1 is not working for Decision Tree/Random Forest classification

• How do I fix/debug this Multi-Process terminated worker error thrown in scikit learn

• TerminatedWorkerError in GridSearch

• A worker process managed by the executor was unexpectedly terminated

• How do I fix/debug this Multi-Process terminated worker error thrown in scikit learn

• TerminatedWorkerError: A worker process managed by the executor was unexpectedly terminated

• Issues with multiple jobs when using RandomizedSearchCV

Answer 1

I think I have found a solution here.

The issue appears to have been resolved by installing lightgbm using the following method:

$ sudo conda install -c conda-forge lightgbm 

rather than the usual pip install method.