I am currently running sidekiq as a container in ECS.

What I have currently is that I upload a custom metric to cloudwatch which indicates the number of jobs in the queue (which I collect through a lambda function periodically) and then I've set a target tracking policy with a target value of 100 jobs to track the custom metric.

While the target tracking is working great to scale up and down the nr of ECS tasks, I can't make the ECS service have a minimum number of tasks to 0 since if i do that there will be jobs stuck in queue which will not be processed until there are more than 100 jobs in the queue.

The current target tracking policy is as follows:

  AutoScalingDefaultWorkerTarget:
    Type: AWS::ApplicationAutoScaling::ScalableTarget
    Properties:
      MinCapacity: 1
      MaxCapacity: 8
      ResourceId: !Join
        - "/"
        - - "service"
          - !Ref ECSCluster
          - !GetAtt DefaultWorkerService.Name
      ScalableDimension: ecs:service:DesiredCount
      ServiceNamespace: ecs
      RoleARN: 
        Fn::Sub: "arn:aws:iam::${AWS::AccountId}:role/aws-service-role/ecs.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_ECSService"
    

  AutoScalingDefaultWorkerTargetTrackingPolicy:
    Type: AWS::ApplicationAutoScaling::ScalingPolicy
    Properties:
      PolicyName: default-worker-target-tracking-scaling-policy
      PolicyType: TargetTrackingScaling
      ScalingTargetId: !Ref AutoScalingDefaultWorkerTarget
      TargetTrackingScalingPolicyConfiguration:
        TargetValue: 50
        ScaleInCooldown: 10
        ScaleOutCooldown: 150
        CustomizedMetricSpecification:
          MetricName: default-worker-nr-jobs-avg
          Namespace: App-Queues
          Statistic: Average

Is there a way to modify the target tracking policy so that even if 1 job is in the queue and the ECS service currently has 0 tasks running it will scale up?

I don't think I can use AWS Batch since i believe that won't be compatible with sidekiq since sidekiq is by nature a long running process and not something that finishes.

0

There are 0 best solutions below