I'm working on a project where I'm using DESlib library for ensemble learning in Python. I'm interested in extracting the feature importance values from the ensemble classifiers. However, I couldn't find a direct attribute or method in DESlib to obtain feature importances. DESlib (Dynamic Ensemble Selection library) is a Python library for ensemble learning. It provides various techniques for dynamic classifier and ensemble selection, allowing for adaptive combination of multiple classifiers to improve predictive performance and decision-making. DESlib supports both single- and multi-label classification problem I would appreciate any guidance on how to extract feature importances using DESlib. Specifically, I'm using KNORAU for my ensemble.
Here's the relevant portion of my code:
rng = np.random.RandomState(1234561)
pool_classifiers = BaggingClassifier(RandomForestClassifier(),
random_state=rng
)
pool_classifiers.fit(Xl, yl)
# Initialize the KNORA-U ensemble
knu = KNORAU(pool_classifiers)
# Set the value of k (region of competence size)
knu.k = 3
# Fit the KNORA-U ensemble to the training data
knu.fit(Xl, yl)
# Predict probabilities for the positive class
y_proba = knu.predict_proba(Xl_t)[:, 1]
# Compute the AUC score
auc = roc_auc_score(yl_t, y_proba)
print(auc)
# Compute the confusion matrix
cm = confusion_matrix(yl_t, y_pred)
# Plot the confusion matrix
plt.figure(figsize=(6, 6))
sns.heatmap(cm, annot=True, cmap='Blues', fmt='d')
plt.title('KNORA-U - Confusion Matrix')
plt.xlabel('Predicted Label')
plt.ylabel('True Label')
plt.show()`pool_classifiers =
The code fits a BaggingClassifier with a RandomForestClassifier as the base estimator, then applies the KNORA-U ensemble method to predict probabilities for the positive class, calculate the AUC score, and plot the confusion matrix.