How to classify facials features embedding with high accuracy(+ 90%). what adjustment can i make in svm model to classify 20+ classes

Question

I used facenet for extracting features and used svm to classify. It worked fine but after 20 classes accuracy went down to 75%. How can I optimise svm while utilising GPU.

I used this svm class model scikit learn's svm model doesn't use gpu so i used this instead

class SVM(nn.Module):
    def __init__(self):
        super(SVM, self).__init__()
        self.fc = nn.Linear(X.shape[1], len(ClassList))

    def forward(self, x):
        return self.fc(x)

but with this accuracy is very low for 20+ classes I also tried using this :

class SoftmaxUsed(nn.Module):
    def __init__(self):
        super().__init__()
        self.layers = nn.Sequential(nn.Linear(512, 1024),
                                 nn.ReLU(),
                                 nn.Dropout(0.2),
                                 nn.Linear(1024, 1024),
                                 nn.ReLU(),
                                 nn.Dropout(0.2),
                                 nn.Linear(1024, len(ClassList)),
                                 nn.LogSoftmax(dim=1))
    def forward(self, x):
        return self.layers(x)

But still the accuracy was 86% at max