Akka.net Context.System.EventStream in aws lambda

Question

I'm working on a solution hosting asp.net core application in AWS lambda. This allows each team member to own an environment during development only. It will run in docker for integration and go live. Corporate knowledge means it has to run this way. It's a small actor system that must run one job at a time sequentially.

The issue is when local, all works fine, but when hosted in AWS lambda, the process which begins with a number of specific actorRef.Tell() calls, work until the first call to: Context.System.EventStream.Publish(). There's no warning or failure, it just stops until the lambda times out. It seems like Context.System.EventStream.Subscribe(Self, typeof(SomeEvent)); just doesn't work

I have included copious logging, so I see exactly where it stops. I've also enabled all the akka.net logging:

"debug": {
    "autoreceive": "on",
    "eventStream": "on",
    "lifecycle": "on",
    "receive": "on",
    "unhandled": "on"
  }

• Is there something inherent in how Context.System.EventStream.Publish() that's different from actorRef.Tell() affected by hosting environment?
• What about aws lambda environment would cause this behaviour?
• Is there a way to configure akka.net to address any constraints in aws lambda?
• Has anyone encountered problems of this sort?

EDIT

Any aspect of akka.net that relies on EventStream doesn't work. Even logging using the Context.GetLogger. The actors only started logging when I changed from the Context based logger to the injected ILogger<T>.

_logger.Info("Message {@msg} validated successfully, processing", e.MsgId); // this is shown in the logs also
Context.System.EventStream.Publish(new ProcessHl7MessageCommand(e.Body));

Below is the lifecycle of the lambda calls, from 13:09:07 to 13:09:12 everything is gravy, and then published event goes nowhere:

2021-03-08T13:09:07.017+00:00   START RequestId: a57732ba-4d37-46c1-b184-6f4c4b3dc438 Version: $LATEST
2021-03-08T13:09:07.019+00:00   13:09:06.996 [WARN] Microsoft.AspNetCore.DataProtection.Repositories.EphemeralXmlRepository - Using an in-memory repository. Keys will not be persisted to storage.
2021-03-08T13:09:07.155+00:00   13:09:07.000 [WARN] Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager - Neither user profile nor HKLM registry available. Using an ephemeral key repository. Protected data will be unavailable when application exits.
2021-03-08T13:09:07.182+00:00   13:09:07.152 [WARN] Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager - No XML encryptor configured. Key {d261dd3f-bbc7-4ad4-a0a5-20cac3e43970} may be persisted to storage in unencrypted form.
2021-03-08T13:09:07.182+00:00   Akka config journal: akka.persistence.journal.sqlServer
2021-03-08T13:09:07.457+00:00   Akka config SnapshotStore: akka.persistence.snapshot-store.sqlServer
2021-03-08T13:09:07.457+00:00   [DEBUG][3/8/2021 1:09:07 PM][Thread 0001][EventStream] subscribing [akka://all-systems/] to channel Akka.Event.Debug
2021-03-08T13:09:07.457+00:00   [DEBUG][3/8/2021 1:09:07 PM][Thread 0001][EventStream] subscribing [akka://all-systems/] to channel Akka.Event.Info
2021-03-08T13:09:07.458+00:00   [DEBUG][3/8/2021 1:09:07 PM][Thread 0001][EventStream] subscribing [akka://all-systems/] to channel Akka.Event.Warning
2021-03-08T13:09:07.458+00:00   [DEBUG][3/8/2021 1:09:07 PM][Thread 0001][EventStream] subscribing [akka://all-systems/] to channel Akka.Event.Error
2021-03-08T13:09:07.538+00:00   [DEBUG][3/8/2021 1:09:07 PM][Thread 0001][EventStream] StandardOutLogger started
2021-03-08T13:09:07.555+00:00   [DEBUG][3/8/2021 1:09:07 PM][Thread 0012][akka://P80System/system] Started (Akka.Actor.SystemGuardianActor)
2021-03-08T13:09:07.555+00:00   [DEBUG][3/8/2021 1:09:07 PM][Thread 0001][EventStream] subscribing [akka://P80System/system/UnhandledMessageForwarder#274328165] to channel Akka.Event.UnhandledMessage
2021-03-08T13:09:07.555+00:00   [DEBUG][3/8/2021 1:09:07 PM][Thread 0001][EventStream(P80System)] StandardOutLogger being removed
2021-03-08T13:09:07.555+00:00   [DEBUG][3/8/2021 1:09:07 PM][Thread 0001][EventStream] unsubscribing [akka://all-systems/] from all channels
2021-03-08T13:09:07.555+00:00   [DEBUG][3/8/2021 1:09:07 PM][Thread 0014][akka://P80System/] Started (Akka.Actor.GuardianActor)
2021-03-08T13:09:12.616+00:00   END RequestId: a57732ba-4d37-46c1-b184-6f4c4b3dc438
2021-03-08T13:09:12.616+00:00   REPORT RequestId: a57732ba-4d37-46c1-b184-6f4c4b3dc438 Duration: 4995.25 ms Billed Duration: 4996 ms Memory Size: 1024 MB Max Memory Used: 249 MB Init Duration: 2362.63 ms
********
2021-03-08T13:10:03.196+00:00   START RequestId: 76e19c6d-3265-4ef0-abfe-82bf68f01bf4 Version: $LATEST
2021-03-08T13:10:03.216+00:00   13:10:03.216 [INFO] P80.Api.Controllers.MessagesController - Received 8142
2021-03-08T13:10:03.277+00:00   13:10:03.277 [INFO] P80.Ordering.Hl7MessageManagerActor - Initializing P80.Models.Messages.Commands.StartProcessingCommand
2021-03-08T13:10:03.278+00:00   13:10:03.278 [INFO] P80.Ordering.MessageLoaderActor - Loading Message 8142
2021-03-08T13:10:03.278+00:00   13:10:03.278 [INFO] P80.Ordering.MessageLoaderActor - Loading 8142
2021-03-08T13:10:03.741+00:00   13:10:03.740 [INFO] P80.Data.Context.MessageResourceAccess - Message loaded: { MsgId = 8142, CustomerName = A PHARMACY IN JURIEN BAY }
2021-03-08T13:10:03.798+00:00   13:10:03.798 [INFO] P80.Ordering.MessageLoaderActor - Message loaded: 8142
2021-03-08T13:10:03.860+00:00   13:10:03.859 [INFO] P80.Ordering.Hl7MessageManagerActor - Message 8142 validated successfully, processing
******* Here's where the first call to EventStream.Publish() occurs and nothing happens again until the lambda closes. 
2021-03-08T13:10:30.258+00:00   13:10:30.257 [INFO] P80.Api.Controllers.MessagesController - Message 8142 still processing after 00:00:27...
2021-03-08T13:10:30.262+00:00   END RequestId: 76e19c6d-3265-4ef0-abfe-82bf68f01bf4
2021-03-08T13:10:30.262+00:00   REPORT RequestId: 76e19c6d-3265-4ef0-abfe-82bf68f01bf4 Duration: 27063.17 ms Billed Duration: 27064 ms Memory Size: 1024 MB Max Memory Used: 285 MB