Service Fabric runtime is not reclaiming unused memory from Actor Service Instance

Question

Our app, one request creates one actor in any one partition, and for processing of one request Memory increasing 200MB-250MB during execution of actor, and after execution is finished I have deleted the actor

Task.Run(() =>
                        {
                            actorObject.ExecuteGrainAsync(requestId, jsonModel).ContinueWith(async (t) =>
                            {
                                await GrainFactory.DeleteActor(actorObject.GetActorId(), "Workflow", CancellationToken.None);
                            }); 
                        });


 IActorService myActorServiceProxy = ActorServiceProxy.Create(
                new Uri($"fabric:/APPSeConnect.WebAgent/{actorName}"), actorId);
                await myActorServiceProxy.DeleteActorAsync(actorId, cancellationToken);

There are no active actors in partition which we can check by querying the fabric, but still the memory is blocked many minutes(5min - more than 50min). Later exe memory size decreases just few MB.

According to this Document memory should be claimed by runtime in idealtimeout. My setting is

new ActorGarbageCollectionSettings(10, 2)

I am using this attribute too

[StatePersistence(StatePersistence.None)]

During parallel processing of actors it's impact is huge.

Answer 1

We have been facing similar issue as yours, where the Actor Service is not removing its claimed memory in 15 minutes.

The test case is like :

1. We request an Actor and wait for the actor to complete its execution.
2. The claimed memory is in 100s of MBs.
3. After execution is complete, the Actor is programmatically deleted using DeleteActorAsync API as mentioned by you.
4. The garbageCollection settings is also set to ActorGarbageCollectionSettings = new ActorGarbageCollectionSettings(10, 5).

But in our case the whole of the memory is getting released in at least 30 minutes time. Which makes us think probably it is because of Generations of GC collection.