Error 500 HTTP Request with body data length bigger than 100MB on C# API

Question

We have developed an ASP.NET Core 2.0 Web API in C# created through a Visual Studio template.

It has been running for years and we have never had any performance issues. Until recently, we haven't had to deal with large HTTP requests, so we have improved the API configuration to allow requests larger than 2GB.

We have modified the Web.config of the application as follows:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.webServer>
    <security>
      <requestFiltering>
        <requestLimits maxAllowedContentLength="2147483647" />
      </requestFiltering>
    </security>
  </system.webServer>
  <system.web.extensions>
   <scripting>
     <webServices>
       <jsonSerialization maxJsonLength="2147483647"></jsonSerialization>
     </webServices>
   </scripting>
  </system.web.extensions>
</configuration>

With this configuration, we started to be able to receive requests larger than the 30MB that .NET web applications have by default without any problem, but, when these requests exceed approximately 100 MB, the application returns a 500 error.

The API is composed of 4 layers:

1. Security
2. Controller
3. Service
4. Repository

The issue is that I have tried to send data over 100MB and I have been able to verify that they reach layer 2 and 3 (layer 1 only manages user security) but when they are going to pass the data from layer 3 to layer 4, the application returns a 500 error.

The curious thing about this is that layers 3 and 4 are DLLs, they are not web projects where you can configure a request size limit.

The API is published in Azure and we think it's a memory problem, but, by increasing the API scaling and increasing the RAM and CPU, the same problem keeps happening.

Has anyone encountered something like this? Any possible solution?

Thank you very much in advance