ConcurrentQueue with items that have sequence value

Question

I'm using a ConcurrentQueue<Result> to store results of a certain activity that occurs in my system (ASP.NET Core 6). It works well.

But now I have a new requirement: I need a Result.Id property - a long value, similar to a sequential database primary key. When I enqueue an item, it must get the next value in the sequence, i.e. queue.Count + 1.

When I enqueue items, I don't use locks because ConcurrentQueue does that for me. But now that need Id, I must:

• lock access to Enqueue()
• determine the queue's size
• increment by one
• set the item's Id
• enqueue the item

or:

• lock access to Enqueue()
• enqueue the item
• determine the queue's new size
• update the item's Id (but it will be in an inconsistent state until then)

Can I do this without locking the Enqueue() method? Or maybe there's a better way? (Though I prefer to continue using ConcurrentQueue, if possible, as it already works well for what I have.)

Answer 1

Instead of assigning the Id when adding the items, you can assign it after you have finished adding items to the queue. You can use a Select with index, e.g.:

public class Item {
    public int Id { get; set; }
    public int Value { get; set; }
}

public static void Main()
{
    var q = new ConcurrentQueue<Item>();
    Parallel.For(0, 10, i => q.Enqueue(new Item() { Value = i + 100 }));
    var withIds = q.Select((x, i) => {
        x.Id = i;
        return x;
    });
    
    Console.WriteLine(string.Join(", ", withIds.Select(x => string.Format("{0} -> {1}", x.Id, x.Value))));
}

This leads to the following output:

0 -> 100, 1 -> 102, 2 -> 103, 3 -> 105, 4 -> 101, 5 -> 106, 6 -> 107, 7 -> 104, 8 -> 108, 9 -> 109

Depending on the size of the queue and the number of times you iterate it, you can also add a ToArray after the Select in order to avoid assigning the Ids multiple times.

Important to note that this only works if you can clearly separate the phase of adding items to the queue and processing the queue.

Answer 2

You could consider storing the IDs in Lazy<long> instances, and make these instances a part of the state of your Result type. Or you could store them side by side in the ConcurrentQueue<T> in value tuples, like this:

long idSeed = 0;
Func<long> idFactory = () => Interlocked.Increment(ref idSeed);

ConcurrentQueue<(Result, Lazy<long>)> queue = new();

//...

queue.Enqueue((new Result(), new Lazy<long>(idFactory)));

//...

if (queue.TryDequeue(out var entry))
{
    var (result, lazy) = entry;
    long id = lazy.Value;
}

The Lazy<T> class guarantees that the valueFactory will be invoked only once for each Lazy<T> instance. So even if multiple threads are racing to read the ID of a specific Result, all threads will get the same ID.