unable to upload image greater than 4 gb on AWS ECR, ERROR reincarnated worker Process-1:4 after death

39 Views Asked by At

Sending .tar file from frontend to backend and in backend receives binary of tar file and once we get that file we are starting Django q and using it upload it to AWS ECR layer by layer Here is the Q-cluster configuration :

Q_CLUSTER = {
  "name": "abc",
  "workers": 8,
  "recycle": 500,
  'attempt_count': 1,
  'ack_failures': True,
  "compress": True,
  "save_limit": 250,
  "queue_limit": 500,
  "cpu_affinity": 1,
  "label": "Django Q",
  "redis": {
      "host": REDIS_HOST,
      "port": REDIS_PORT,
      "db": 0,
  },
}

So the Error getting while uploading :

[INFO] Starting pushing layer 66ffede0daa0486a61c11446bcb954ad89df6c2cb567ecd6ab97c12228ef6e8f/layer.tar
[INFO] Upload started

[INFO] Starting pushing layer 4562d7acfd41cfad11b031cdd9bcd217dcd3cd36fd0cc5aabfc8df18e3a6972b/layer.tar
[INFO] Upload started
[INFO] Starting pushing layer bacd26a46504a4282343e37dac8ac4c24ee649fe85c605547d57960822e076fe/layer.tar
[INFO] Upload started
Pushing... 100.0%  

[INFO] Starting pushing layer 29165659eb61efe28b84e580bdca126a60a356655dd62a6bed38d61a496fa6aa/layer.tar
[INFO] Upload started
[INFO] Starting pushing layer 50e432f951ede2c9c9c299f8a68e106282af2f59fdf1120ec9cc084a1005da22/layer.tar
[INFO] Upload started
Pushing... 100.0%  
[INFO] Starting pushing layer dc984c3c30a1a444dfdfcdff0686ef712412716920ad8fe619787f7735ab156e/layer.tar
[INFO] Upload started
[INFO] Starting pushing layer fe6d7993f09d1b294ae3980f370e0b9b434941bfa99c85a3c63428c0f6a1a2e7/layer.tar
[INFO] Upload started
[INFO] Starting pushing layer b561779bae7406e1a97dae43f3c0b45e5dd8a52d359aae533e670775cb0464df/layer.tar
[INFO] Upload started
[INFO] Starting pushing layer 170d1e7a28c707797b61f0f54c14f5ad2cff239855a823e3c2442ef09adf0ab5/layer.tar
[INFO] Upload started
[INFO] Starting pushing layer f521d5101bf2325bb2e210dbd9c86c1608ab083c4f2d819cf8bc205810492474/layer.tar
[INFO] Upload started
[INFO] Starting pushing layer 2908c3588dbe82e5eaac1f06a0b3a5d60b7d689d489c28d3c95b310c23d18bd2/layer.tar
[INFO] Upload started
[INFO] Starting pushing layer 27bcf483885811cb8ee15fd784cf8115910a78505fd98f54d013b14cf7b0b923/layer.tar
[INFO] Upload started
[INFO] Starting pushing layer 4d19ecbd1fbd26168147a173d7085150b661957dd7244fe154f1f48efd53302f/layer.tar
[INFO] Upload started
Pushing... 100.0%  
Pushing... 100.0%  
Pushing... 100.0%  


Pushing... 100.0%  
Pushing... 100.0%  
Pushing... 100.0%  
Pushing... 100.0%  
10:43:00 [Q] ERROR reincarnated worker Process-1:4 after death
10:43:00 [Q] INFO Process-1:11 ready for work at 50

I tried attempt_count = 1 and ack_failures = True but still getting error in worker

0

There are 0 best solutions below