Large File Recording in MySQL using C#

56 Views Asked by At

I would like to perform a recording of large files in MySQL using C#. I tried with the following code, but it writes in parts that have the size of the buffer. When I use a buffer with the size of the file I want to write, the memory that the program is using increases a lot, resulting in a 'System out of Memory' that kills my application. The code here writes to the database in several parts with the size of the buffer. I would like to write the entire file, not in parts. Below is the code used:

Sending code:

OpenFileDialog fileDialog = new OpenFileDialog();

string source;

if (fileDialog.ShowDialog() == DialogResult.OK)
{
    source = fileDialog.FileName;
}
else
{
    MessageBox.Show("To save a file, we need one first");
    return;
}

string destination = @"<file_path>";
string destinationFileName = Path.Combine(destination, "<file_name>");
int chunkSize = 7168 * 1024; // Arbitrary buffer size

using (var sourceStream = File.OpenRead(source))
{
    using (var destinationStream = File.OpenWrite(destinationFileName))
    {
        long fileSize = sourceStream.Length;
        byte[] buffer = new byte[chunkSize];
        long totalBytesRead = 0;
        int bytesRead;

        while ((bytesRead = await sourceStream.ReadAsync(buffer, 0, buffer.Length)) > 0)
        {
            await destinationStream.WriteAsync(buffer, 0, bytesRead);
            totalBytesRead += bytesRead;

            // Calculate progress
            double progress = (double)totalBytesRead / fileSize;

            // This method is the one that writes to the database
            //InsertFileIntoDatabase(connectionString, "Test", buffer);

            // Update ProgressBar
            lblGravado.Text = "Progress: " + ((int)(progress * 100)).ToString() + "%";
        }

        // Show completion message
        MessageBox.Show("Transfer completed!");
    }
}

Code that performs writing in MySQL:

MySqlConnection connection = new MySqlConnection(connectionString);

try
{
    connection.Open();

    using (MySqlCommand command = new MySqlCommand("INSERT INTO files_table (file_name, file_content) VALUES (@name, @content)", connection))
    {
        command.Parameters.AddWithValue("@name", fileName);
        command.Parameters.AddWithValue("@content", fileContent);
        command.ExecuteNonQuery();
    }

    connection.Close();
    connection.Dispose();
}
catch (Exception ex)
{
    MessageBox.Show("Error: " + ex.Message);
    return;
}
finally
{
    connection.Close();
    connection.Dispose();
}

If you can help me with this problem, I would be grateful!

I would like you to solve this problem I am facing.

1

There are 1 best solutions below

1
Steve On

I cannot comment yet, but how about another approach for this?
How about you only save the file information and the path of the files?
If you have tons of files but only a small database with file information
it is easier to handle backups and migrations will also be simple.