Python machine learning pytorch test/train epoch results problem

Question

Created a linear neural network for 3 labels classification. I defined the train and test loops and then i ran them for 50 times printing results every 10 epochs. In each print the accuracy of the test is often the same. How can it be the same?

import torch.nn.functional as F
import torch
from torch.utils.data import DataLoader, TensorDataset
import torch.nn as nn
import torch.optim as optim
from sklearn.metrics import accuracy_score, confusion_matrix
import seaborn as sns
import matplotlib.pyplot as plt

class NeuralModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.flatten = nn.Flatten()
        self.linear_stack = nn.Sequential(
            nn.Linear(25,50),
            nn.Linear(50,50),
            nn.Linear(50, 3),
        )

    def forward(self, x):
        x = self.flatten(x)
        logits = self.linear_stack(x)
        return logits

model = NeuralModel()
print(model)

X_train_tensor = torch.Tensor(X_train_std)
y_train_tensor = torch.Tensor(y_train)
X_test_tensor = torch.Tensor(X_test_std)
y_test_tensor = torch.LongTensor(y_test)

batch_size = 32
train_dataset = TensorDataset(X_train_tensor, y_train_tensor)
train_dataloader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
test_dataset = TensorDataset(X_test_tensor, y_test_tensor)
test_dataloader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)

loss_fn = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)

epochs = 50
def train(dataloader, model, loss_fn, optimizer):
    size = len(dataloader.dataset)
    model.train()
    train_loss = 0.0
    all_preds, all_targets = [], []
    for batch, (X, y) in enumerate(dataloader):
        pred = model(X)
        loss = loss_fn(pred, y.long())
        loss.backward()
        optimizer.step()
        optimizer.zero_grad()
        train_loss += loss.item()

        _, predicted_classes = pred.max(dim=1)
        all_preds.extend(predicted_classes.numpy())
        all_targets.extend(y.numpy())

    train_loss = train_loss / len(dataloader)
    accuracy = accuracy_score(all_targets, all_preds)

    return train_loss, accuracy

def test(dataloader, model, loss_fn):
    size = len(dataloader.dataset)
    model.eval()
    test_loss = 0
    all_preds, all_targets = [], []

    with torch.no_grad():
        for X, y in dataloader:
            pred = model(X)
            test_loss += loss_fn(pred, y.long()).item()
            all_preds.extend(pred.argmax(1).numpy())
            all_targets.extend(y.numpy())

    test_loss /= len(dataloader)
    accuracy = accuracy_score(all_targets, all_preds)
    confusion_mat = confusion_matrix(all_targets, all_preds)

    return test_loss, accuracy, confusion_mat


for epoch in range(1, 51):
    train_loss, train_accuracy = train(train_dataloader, model, loss_fn, optimizer)
    print(f"Epoch {epoch}: Training Loss: {train_loss:.4f}, Accuracy: {train_accuracy:.4f}")

    test_loss, test_accuracy, confusion_mat = test(test_dataloader, model, loss_fn)
    print(f"Epoch {epoch}: Test Loss: {test_loss:.4f}, Accuracy: {test_accuracy:.4f}")
    print("Confusion Matrix:")
    print(confusion_mat)

RESULTS

NeuralModel(
  (flatten): Flatten(start_dim=1, end_dim=-1)
  (linear_stack): Sequential(
    (0): Linear(in_features=25, out_features=50, bias=True)
    (1): Linear(in_features=50, out_features=50, bias=True)
    (2): Linear(in_features=50, out_features=3, bias=True)
  )
)
Epoch 1: Training Loss: 1.0222, Accuracy: 0.6897
Epoch 1: Test Loss: 0.9916, Accuracy: 0.6460
Confusion Matrix:
[[46  1 15]
 [16  7  4]
 [ 2  2 20]]
Epoch 2: Training Loss: 0.9527, Accuracy: 0.7433
Epoch 2: Test Loss: 0.9277, Accuracy: 0.7788
Confusion Matrix:
[[58  1  3]
 [14 10  3]
 [ 2  2 20]]
Epoch 3: Training Loss: 0.8982, Accuracy: 0.8352
Epoch 3: Test Loss: 0.8713, Accuracy: 0.8584
Confusion Matrix:
[[58  1  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 4: Training Loss: 0.8444, Accuracy: 0.8506
Epoch 4: Test Loss: 0.8187, Accuracy: 0.8584
Confusion Matrix:
[[58  1  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 5: Training Loss: 0.7867, Accuracy: 0.8621
Epoch 5: Test Loss: 0.7695, Accuracy: 0.8584
Confusion Matrix:
[[58  1  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 6: Training Loss: 0.7517, Accuracy: 0.8621
Epoch 6: Test Loss: 0.7251, Accuracy: 0.8673
Confusion Matrix:
[[59  0  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 7: Training Loss: 0.6944, Accuracy: 0.8697
Epoch 7: Test Loss: 0.6845, Accuracy: 0.8673
Confusion Matrix:
[[59  0  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 8: Training Loss: 0.6599, Accuracy: 0.8697
Epoch 8: Test Loss: 0.6476, Accuracy: 0.8673
Confusion Matrix:
[[59  0  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 9: Training Loss: 0.6122, Accuracy: 0.8697
Epoch 9: Test Loss: 0.6158, Accuracy: 0.8673
Confusion Matrix:
[[59  0  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 10: Training Loss: 0.5992, Accuracy: 0.8697
Epoch 10: Test Loss: 0.5869, Accuracy: 0.8673
Confusion Matrix:
[[59  0  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 11: Training Loss: 0.5786, Accuracy: 0.8697
Epoch 11: Test Loss: 0.5629, Accuracy: 0.8673
Confusion Matrix:
[[59  0  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 12: Training Loss: 0.5644, Accuracy: 0.8697
Epoch 12: Test Loss: 0.5436, Accuracy: 0.8673
Confusion Matrix:
[[59  0  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 13: Training Loss: 0.5227, Accuracy: 0.8736
Epoch 13: Test Loss: 0.5254, Accuracy: 0.8673
Confusion Matrix:
[[59  0  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 14: Training Loss: 0.5449, Accuracy: 0.8736
Epoch 14: Test Loss: 0.5123, Accuracy: 0.8673
Confusion Matrix:
[[59  0  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 15: Training Loss: 0.5187, Accuracy: 0.8736
Epoch 15: Test Loss: 0.4999, Accuracy: 0.8673
Confusion Matrix:
[[59  0  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 16: Training Loss: 0.5244, Accuracy: 0.8736
Epoch 16: Test Loss: 0.4921, Accuracy: 0.8673
Confusion Matrix:
[[59  0  3]
 [ 5 19  3]
 [ 2  2 20]]
Epoch 17: Training Loss: 0.4657, Accuracy: 0.8812
Epoch 17: Test Loss: 0.4818, Accuracy: 0.8673
Confusion Matrix:
[[59  0  3]
 [ 4 19  4]
 [ 2  2 20]]
Epoch 18: Training Loss: 0.4686, Accuracy: 0.8812
Epoch 18: Test Loss: 0.4725, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 4 20  3]
 [ 2  2 20]]
Epoch 19: Training Loss: 0.4382, Accuracy: 0.8812
Epoch 19: Test Loss: 0.4643, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 4 20  3]
 [ 2  2 20]]
Epoch 20: Training Loss: 0.4241, Accuracy: 0.8851
Epoch 20: Test Loss: 0.4576, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 4 20  3]
 [ 2  2 20]]
Epoch 21: Training Loss: 0.4195, Accuracy: 0.8851
Epoch 21: Test Loss: 0.4520, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 4 20  3]
 [ 2  2 20]]
Epoch 22: Training Loss: 0.4247, Accuracy: 0.8889
Epoch 22: Test Loss: 0.4475, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 3 20  4]
 [ 2  2 20]]
Epoch 23: Training Loss: 0.4162, Accuracy: 0.8889
Epoch 23: Test Loss: 0.4438, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 3 20  4]
 [ 2  2 20]]
Epoch 24: Training Loss: 0.4083, Accuracy: 0.8851
Epoch 24: Test Loss: 0.4396, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 3 20  4]
 [ 2  2 20]]
Epoch 25: Training Loss: 0.4096, Accuracy: 0.8851
Epoch 25: Test Loss: 0.4356, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 3 20  4]
 [ 2  2 20]]
Epoch 26: Training Loss: 0.4106, Accuracy: 0.8927
Epoch 26: Test Loss: 0.4317, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 3 20  4]
 [ 2  2 20]]
Epoch 27: Training Loss: 0.4289, Accuracy: 0.8966
Epoch 27: Test Loss: 0.4315, Accuracy: 0.8850
Confusion Matrix:
[[59  0  3]
 [ 3 21  3]
 [ 2  2 20]]
Epoch 28: Training Loss: 0.3843, Accuracy: 0.9004
Epoch 28: Test Loss: 0.4283, Accuracy: 0.8850
Confusion Matrix:
[[59  0  3]
 [ 3 21  3]
 [ 2  2 20]]
Epoch 29: Training Loss: 0.3862, Accuracy: 0.9004
Epoch 29: Test Loss: 0.4253, Accuracy: 0.8850
Confusion Matrix:
[[59  0  3]
 [ 3 21  3]
 [ 2  2 20]]
Epoch 30: Training Loss: 0.4483, Accuracy: 0.8966
Epoch 30: Test Loss: 0.4236, Accuracy: 0.8850
Confusion Matrix:
[[59  0  3]
 [ 3 21  3]
 [ 2  2 20]]
Epoch 31: Training Loss: 0.3959, Accuracy: 0.9042
Epoch 31: Test Loss: 0.4218, Accuracy: 0.8850
Confusion Matrix:
[[59  0  3]
 [ 3 21  3]
 [ 2  2 20]]
Epoch 32: Training Loss: 0.3850, Accuracy: 0.9042
Epoch 32: Test Loss: 0.4203, Accuracy: 0.8850
Confusion Matrix:
[[59  0  3]
 [ 2 21  4]
 [ 2  2 20]]
Epoch 33: Training Loss: 0.3851, Accuracy: 0.9004
Epoch 33: Test Loss: 0.4195, Accuracy: 0.8850
Confusion Matrix:
[[59  0  3]
 [ 2 21  4]
 [ 2  2 20]]
Epoch 34: Training Loss: 0.3822, Accuracy: 0.8966
Epoch 34: Test Loss: 0.4193, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]
Epoch 35: Training Loss: 0.4585, Accuracy: 0.8966
Epoch 35: Test Loss: 0.4189, Accuracy: 0.8850
Confusion Matrix:
[[59  0  3]
 [ 2 21  4]
 [ 2  2 20]]
Epoch 36: Training Loss: 0.3650, Accuracy: 0.8966
Epoch 36: Test Loss: 0.4181, Accuracy: 0.8850
Confusion Matrix:
[[59  0  3]
 [ 2 21  4]
 [ 2  2 20]]
Epoch 37: Training Loss: 0.3869, Accuracy: 0.9042
Epoch 37: Test Loss: 0.4179, Accuracy: 0.8850
Confusion Matrix:
[[59  0  3]
 [ 2 21  4]
 [ 2  2 20]]
Epoch 38: Training Loss: 0.3831, Accuracy: 0.8966
Epoch 38: Test Loss: 0.4184, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]
Epoch 39: Training Loss: 0.3523, Accuracy: 0.8966
Epoch 39: Test Loss: 0.4178, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]
Epoch 40: Training Loss: 0.3615, Accuracy: 0.9080
Epoch 40: Test Loss: 0.4167, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]
Epoch 41: Training Loss: 0.3601, Accuracy: 0.9042
Epoch 41: Test Loss: 0.4178, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]
Epoch 42: Training Loss: 0.4075, Accuracy: 0.9080
Epoch 42: Test Loss: 0.4173, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]
Epoch 43: Training Loss: 0.3470, Accuracy: 0.9119
Epoch 43: Test Loss: 0.4170, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]
Epoch 44: Training Loss: 0.3658, Accuracy: 0.9119
Epoch 44: Test Loss: 0.4171, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]
Epoch 45: Training Loss: 0.3518, Accuracy: 0.9119
Epoch 45: Test Loss: 0.4168, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]
Epoch 46: Training Loss: 0.3445, Accuracy: 0.9119
Epoch 46: Test Loss: 0.4166, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]
Epoch 47: Training Loss: 0.3592, Accuracy: 0.9119
Epoch 47: Test Loss: 0.4189, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]
Epoch 48: Training Loss: 0.3484, Accuracy: 0.9119
Epoch 48: Test Loss: 0.4194, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]
Epoch 49: Training Loss: 0.3396, Accuracy: 0.9080
Epoch 49: Test Loss: 0.4191, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]
Epoch 50: Training Loss: 0.3463, Accuracy: 0.9119
Epoch 50: Test Loss: 0.4208, Accuracy: 0.8761
Confusion Matrix:
[[59  0  3]
 [ 2 20  5]
 [ 2  2 20]]

shouldn't the results be more varied?

Model has to be with linear activation function, cant use any method to solve the imbalance in the data.

Answer 1

If you have N data points, the accuracy resolution cannot be less than 1/N, since changing the prediction on 1 data point while leaving all the other predictions unchanged affects the accuracy by 1/N.

In your case, it seems that you have an accuracy resolution in the test set of 0.0088 - 0.0089. Since you have 113 data points in the test set (from the confusion matrix you have 59+3+4+20+3+2+2+20=113 data points), the resolution cannot be less than 1/113 = 0.0088.