Multi-Threading on C# Dataset table rows making an external API call and update table with API Call results

Question

I have a table in my SQL server/DB with close to 100,000 records. Using C# - need to loop thru the rows and make an external API call and update the results of the call into SQL table. I am new to multi-threading concept. How can I achieve this?

Here is the code I have - just sequential processing.

        public void MainProcess()
        {
            try
            {
                // Retrieve rows from table
                List<rowResult> rowResults = (List<rowResult>)GetRowsFromTable();
                foreach (var row in rowResults)
                {
                    callExternalAPI(row);
                }
            }
            catch (Exception)
            {

                throw;
            }
         }

How can I modify this to enable Multi-threading? Please help

Answer 1

One way to do this is to use Parallel.ForEach, replace your foreach with the below which calls callExternalApi directly with the rowResult:

Parallel.ForEach(rowResults, callExternalAPI);

Or if you need to do additional processing with each row you can use:

Parallel.ForEach(rowResults, row =>
{
    //Additional processing
    callExternalAPI(row);
    //Additional processing
});

You can also use ParallelOptions to set how many threads are used with MaxDegreeOfParallelism; though this should default to something sensible.

ParallelOptions po = new ParallelOptions()
{
    MaxDegreeOfParallelism = 4
};
Parallel.ForEach(rowResults, po, callExternalAPI);