CNN Model: Achieved 100% Training Accuracy but Fails to Predict Users Correctly in Testing – Dataset or Model Issue?

41 Views Asked by At

I'm currently working on a project that involves two main modules: one for creating datasets and another for evaluating a CNN model.

Dataset Module:

Live Image Capture: This module captures live images from the user. Adding Random Images: After obtaining live images, I augment the dataset by adding random images. Train-Test Split: The dataset is then split into training and testing folders. CSV File Creation: I create CSV files for labeling the data in the train and test datasets. Image Resizing: To ensure consistency, I resize the images and replace them in the same path. CNN Model Module:

Image to Arrays Conversion: In the model module, images from both the training and testing folders are converted to arrays. Model Training: The model is trained using the converted arrays. If the model's accuracy is below 0.85, I provide an option to change its architecture. Model Details: I use the Adam optimizer and accuracy as metrics for this model. Additional details about the model can be found in the code below.

    def train_test_save_cnn_model(self,arch_type=1):
        #assigns trauin_path in src_train
        src_train=self.train_d_path
        #assigns trauin_path in src_test
        src_test=self.test_d_path
        #stores the x_train,y_train data by calling images_to_input_array function with src_train path
        [x_train,y_train]=self.images_to_input_array(src_train)
        #stores the x_test,y_test data by calling images_to_input_array function with src_test path
        [x_test,y_test]=self.images_to_input_array(src_test)
        #if arch_type is 1 enters 1 st model_architecture 
        
        if arch_type==1:
            #cnn architeture
            model = Sequential()
            model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(int(self.org_img_height),int(self.org_img_width),3)))
            model.add(MaxPooling2D((2, 2)))
            model.add(Conv2D(64, (3, 3), activation='relu'))
            model.add(MaxPooling2D((2, 2)))
            model.add(Conv2D(128, (3, 3), activation='relu'))
            model.add(MaxPooling2D((2, 2)))
            model.add(Flatten())
            model.add(Dense(128, activation='relu'))
            model.add(Dropout(0.5))
            model.add(Dense(1, activation='sigmoid'))
        elif arch_type==2:
            model = Sequential()
            model.add(Conv2D(32, (3, 3), activation='relu', padding='same', name='conv_1', 
                             input_shape=(150, 150, 3)))
            model.add(MaxPooling2D((2, 2), name='maxpool_1'))
            model.add(Conv2D(64, (3, 3), activation='relu', padding='same', name='conv_2'))
            model.add(MaxPooling2D((2, 2), name='maxpool_2'))
            model.add(Conv2D(128, (3, 3), activation='relu', padding='same', name='conv_3'))
            model.add(MaxPooling2D((2, 2), name='maxpool_3'))
            model.add(Conv2D(128, (3, 3), activation='relu', padding='same', name='conv_4'))
            model.add(MaxPooling2D((2, 2), name='maxpool_4'))
            model.add(Flatten())
            model.add(Dropout(0.5))
            model.add(Dense(512, activation='relu', name='dense_1'))
            model.add(Dense(128, activation='relu', name='dense_2'))
            model.add(Dense(1, activation='sigmoid', name='output'))
            #This model architecture  reference link
            #https://towardsdatascience.com/applied-deep-learning-part-4-convolutional-neural-networks-584bc134c1e2
        
        #It initialize optimizer,loss_functin,metrics before fitting to train data.
        model.compile(optimizer='adam', loss='binary_crossentropy',metrics=['accuracy'])
        #Train_data fits to model
        model.fit(x_train, y_train, epochs=7)
       
        # Save the model architecture to a file
        model_json = model.to_json()
        with open(os.path.join(self.user_model_path,f"{self.username}_model_architecture.json"), "w") as json_file:
            json_file.write(model_json)
        
        #Saves the model weights in specific_folder
        model.save_weights(os.path.join(self.user_model_path,f"{self.username}_model_save_weights.h5"))

        model.save(os.path.join(self.user_model_path,f"{self.username}_model.h5"))
        
        

        #return the test_acc and test_loss by evaluting model
        test_loss, test_acc = model.evaluate(x_test,y_test,verbose=2)
        return [test_loss,test_acc]

I made improvements to the model, including:

Background Correction: Used Haarcascade to crop images, addressing background memorization. Data Augmentation: Added more user images for diversity. Training Outcome: Achieved 100% accuracy during training, but the model struggles to predict users accurately during testing.

Dataset details: 100 original images + 200 augmented user images + 200 random images (from Kaggle). I also added 100 images from two other user_datasets. Seeking insights on whether the issue is with the dataset or the model.

0

There are 0 best solutions below