Extracting feature importance with DESlib in Python

34 Views Asked by At

I'm working on a project where I'm using DESlib library for ensemble learning in Python. I'm interested in extracting the feature importance values from the ensemble classifiers. However, I couldn't find a direct attribute or method in DESlib to obtain feature importances. DESlib (Dynamic Ensemble Selection library) is a Python library for ensemble learning. It provides various techniques for dynamic classifier and ensemble selection, allowing for adaptive combination of multiple classifiers to improve predictive performance and decision-making. DESlib supports both single- and multi-label classification problem I would appreciate any guidance on how to extract feature importances using DESlib. Specifically, I'm using KNORAU for my ensemble.

Here's the relevant portion of my code:

rng = np.random.RandomState(1234561)

pool_classifiers = BaggingClassifier(RandomForestClassifier(),
                                     random_state=rng
                                     )
pool_classifiers.fit(Xl, yl)

# Initialize the KNORA-U ensemble
knu = KNORAU(pool_classifiers)

# Set the value of k (region of competence size)
knu.k = 3

# Fit the KNORA-U ensemble to the training data
knu.fit(Xl, yl)

# Predict probabilities for the positive class
y_proba = knu.predict_proba(Xl_t)[:, 1]

# Compute the AUC score
auc = roc_auc_score(yl_t, y_proba)
print(auc)

# Compute the confusion matrix
cm = confusion_matrix(yl_t, y_pred)

# Plot the confusion matrix
plt.figure(figsize=(6, 6))
sns.heatmap(cm, annot=True, cmap='Blues', fmt='d')
plt.title('KNORA-U - Confusion Matrix')
plt.xlabel('Predicted Label')
plt.ylabel('True Label')
plt.show()`pool_classifiers = 

The code fits a BaggingClassifier with a RandomForestClassifier as the base estimator, then applies the KNORA-U ensemble method to predict probabilities for the positive class, calculate the AUC score, and plot the confusion matrix.

0

There are 0 best solutions below