Struggling with High Validation Loss in Variational Autoencoder (VAE) Implementation using Keras

42 Views Asked by At

I'm working on building a Variational Autoencoder (VAE) using Keras. I have 10,000 samples that I've split into training, validation, and testing sets. Here's how I've divided them:

import pandas as pd
import sys
from sklearn.model_selection import train_test_split

NumberOfTraining = 0.6
NumberOfValidation = 0.2
NumberOfTesting = 0.2

url = "/content/drive/MyDrive/aaa_01.csv"
Data = pd.read_csv(url, encoding='utf-8')
AllData = Data
AllData = AllData.sample(frac=1)
TrainData, TestData = train_test_split(AllData, train_size=0.8)

# Number of columns and rows

All = TrainData

XAll = All.iloc[:, :32]
YAll = All.iloc[:, 32:]

TrainData, ValidationData = train_test_split(TrainData, train_size=0.8)

XTrainData = TrainData.iloc[:, :32]
YTrainData = TrainData.iloc[:, 32:]

XValidationData = ValidationData.iloc[:, :32]
YValidationData = ValidationData.iloc[:, 32:]

XTestData = TestData.iloc[:, :32]
YTestData = TestData.iloc[:, 32:]

from keras.layers import Input, Dense, Flatten, Lambda
from keras.models import Model
from keras.optimizers import Adam
import time

n_inputs = 32  # input size (num of columns)
n_outputs = 16  # output size (num of columns)

X = XTrainData.values
Y = YTrainData.values

XX = XValidationData.values
YY = YValidationData.values

VX = XTestData.values
VY = YTestData.values
XAll = XAll.values
YAll = YAll.values

XY = TrainData.values
XYValid = ValidationData.values

I want to create a VAE model to understand the relationship between X and Y. Here's the code for my VAE model:

from keras.layers import Dropout
from keras.layers import Lambda
from keras import backend as K

def sampling(args):
    latent_dim = 32
    x_mean, x_log_var = args
    epsilon = K.random_normal(shape=(K.shape(x_mean)[0], latent_dim), mean=0., stddev=1.0)
    return x_mean + K.exp(0.5 * x_log_var) * epsilon

def sampling2(args):
    z_mean, z_log_var = args
    epsilon = K.random_normal(shape=(K.shape(z_mean)[0], latent_dim))
    return z_mean + K.exp(z_log_var / 2) * epsilon

# Encoder
ncol = 32
X_size = Input(shape=(32, ))

encoded1 = Dense(1024, activation='relu')(X_size)
encoded1 = Dense(512, activation='relu')(encoded1)
encoded1 = Dense(256, activation='relu')(encoded1)

encoded1 = Flatten()(encoded1)
z_mean = Dense(32)(encoded1)
z_log_var = Dense(32)(encoded1)

z = Lambda(sampling)([z_mean, z_log_var])

# Decoder
decoded1 = Dense(256, activation='relu')(z)
decoded1 = Dense(512, activation='relu')(decoded1)
decoded1 = Dense(1024, activation='relu')(decoded1)
decoded1 = Dense(16, activation='linear')(decoded1)

autoencoder = Model(inputs=[X_size], outputs=[decoded1])
autoencoder.compile(optimizer='adam', loss='mean_squared_error')
start_time = time.time()

autoencoder.fit(X, Y, epochs=50, batch_size=200, shuffle=True, validation_data=(XValidationData, YValidationData))
autoencoder.save("Laaaa")

However, the val_loss always remains high. I'm seeking advice on how to address this issue. I've attached a screenshot of the training progress

enter image description here

0

There are 0 best solutions below