Write large byte arrays on RAID0 virtual disk

161 Views Asked by At

I have a camera that produces 20 frames per second. I made an app in C# that grabs the frames and put them in a concurrent queue. I have another thread that writes the frames on filesystem. Each frame is an array of 134184960 bytes that I must write on filesystem. The filesystem is an array of 20 SSD in RAID0 with theoretical speed of 500 MB/s * 20 with the bottleneck of PCiExpress 8x (7,7 GB/s) The production rate is about 2,5 GB/s, but my writing speed doesn't go over 900 MB/s I am using Filestream.Write (link)

What can I do to increase my writing speed?

private ConcurrentQueue<CapturedFrame> cq = new ConcurrentQueue<CapturedFrame>();

....

try
{
    while (cq.TryDequeue(out BufferToWrite))
    {
        byte[] bufferInBytes = new byte[134184960];

        //Matrox function to extract bytes
        MIL.MbufGetColor(BufferToWrite.IDFrame, MIL.M_PLANAR, MIL.M_ALL_BANDS, bufferInBytes);
    
        using (FileStream fileStream = new FileStream(Path.Combine(AcquisitionFolder, DateTime.Now.ToString("yyyy-MM-dd-HHmmss-fffffff") + ".raw"), FileMode.Create))
        {
            fileStream.Write(bufferInBytes, 0, bufferInBytes.Length);
            fileStream.Close();
        }
    }   
}
catch (Exception ex)
{
    log.Error("Exception: " + ex.Message);
}

changed the FileStream to this with no improvement 65536 = 64K is the stripe size of the RAID0 array, I've set the lenght of the stream

using (FileStream fileStream = new FileStream(Path.Combine(AcquisitionFolder, DateTime.Now.ToString("yyyy-MM-dd-HHmmss-fffffff") + ".raw"), FileMode.Create, FileAccess.Write, FileShare.Read, 65536, FileOptions.SequentialScan))
                    {
                        fileStream.SetLength(3 * 8192 * 5460);
                        fileStream.Write(bufferInBytes, 0, bufferInBytes.Length);
                        fileStream.Close();
                    }
1

There are 1 best solutions below

0
Danilo7 On

I was able to reach the 2,5 GB/s write speed with two steps: