SVM initialize with given weights & support vectors

32 Views Asked by At

I am working with a federated learning framework (flower) where I have multiple SVM learned separately. After their training I want to aggregate the SVMs or more specific the the hyperplanes to separate the data (We can't share the data across the different SVMs).
Following thinks I checked:

  • I think I can't just use pickle and save and load the models since I still need to aggregate them.
  • I also can't just extract the weights (coef_) as stated here, since loading the updated ones back into the SVM is not possible.
  • If I use the support_vectors_ which as far as I understood are necessary to span the hyperplane, it is possible to extract them and load the modified ones back to the SVM. If I do so I still miss Information to create the aggregated SVM.

Two questions:

  • Which Information is needed for the SVM to predict/fit on the existing aggregation?
  • Or more general is there a better way to solve this problem?

Here a short snippet of what I need:

import numpy as np
from sklearn.svm import SVC

X1 = np.array([[3,4],[1,4],[2,3],[6,-1],[7,-1],[5,-3]] )
y1 = np.array([-1,-1, -1, 1, 1 , 1 ])

X2 = np.array([[1,2],[2,4],[3,5],[7,8],[8,9],[9,10]])
y2 = np.array([-1,-1,-1,1,1,1])

clf1 = SVC(C = 1e5, kernel = 'linear')
clf1.fit(X1, y1) 
clf2 = SVC(C = 1e5, kernel = 'linear')
clf2.fit(X2, y2)

# do some aggregation
clf = SVC(C = 1e5, kernel = 'linear')
clf.support_vectors_ = (clf1.support_vectors_ + clf2.support_vectors_) / 2
clf.intercept_ = (clf1.intercept_ + clf2.intercept_) / 2
clf.dual_coef_ = (clf1.dual_coef_ + clf2.dual_coef_) / 2

And since visualizations help - I want something like the black line:

import matplotlib.pyplot as plt

from sklearn import svm
from sklearn.datasets import make_blobs
from sklearn.inspection import DecisionBoundaryDisplay

plt.scatter(X1[:, 0], X1[:, 1], c=y1, s=30, cmap=plt.cm.Paired, label='X1', marker='x')
plt.scatter(X2[:, 0], X2[:, 1], c=y2, s=30, cmap=plt.cm.Paired, label='X2', marker='o')

# plot the decision function for clf, clf1 and clf2
ax = plt.gca()

DecisionBoundaryDisplay.from_estimator(
    clf1,
    np.concatenate((X1, X2)),
    plot_method="contour",
    levels=[0],
    alpha=0.7,
    ax=ax,
)
DecisionBoundaryDisplay.from_estimator(
    clf2,
    np.concatenate((X1, X2)),
    plot_method="contour",
    levels=[0],
    alpha=0.7,
    ax=ax,
)
# DecisionBoundaryDisplay.from_estimator(
#     clf,
#     np.concatenate((X1, X2)),
#     plot_method="contour",
#     levels=[0],
#     alpha=0.7,
#     ax=ax,
# )
# what I want from the aggregated SVM
x = np.linspace(0, 8)
y = 4*x - 15
plt.plot(x, y, 'k-')

plt.legend()
plt.show()

enter image description here

0

There are 0 best solutions below