I write a python program to analyze my genome data. But because of the storage limitation, I use external hard disk to store my output data. However, the occasional broken connection make my premature task (I got five task to test, each task got about 8hr to process) abort. My equipment are Mac (M2 chip and 8 GB RAM), a 10 Gbps cable and a 4TB Seagate One touch hard disk.
Here is my task code:
bwa mem -t 16 -R '@RG\tID:test_pop\tPL:illumina\tLB:library\tSM:bulk2_2_16_6' /Volumes/One_Touch/User/test/Out/10_ref/Sbicolor_454_v3.0.1.fa 2_16_6.1.fastq.gz 2_16_6.2.fastq.gz | samtools fixmate -m -@ 16 - - | samtools sort -m 500M -@ 16 | samtools markdup -r -s -d 2500 -@ 16 -f /Volumes/One_Touch/User/test/Out/20_bam/bulk2_2_16_6.0.dupstat - - | samtools view -b -f 2 -@ 16 -F 2048 -o /Volumes/One_Touch/User/test/Out/20_bam/bulk2_2_16_6.0.bam >> /Volumes/One_Touch/User/test/Out/log/alignment.log 2>&1
Here is chatGPT solution, but it seems to not meet my needs, it can catch multi-task error, but not single big task error.
import os
import time
def is_disk_connected(path):
return os.path.exists(path)
def long_running_task():
try:
# your task here
pass
except (IOError, OSError):
print("IOError or OSError occurred. Disk might be disconnected.")
while True:
if is_disk_connected('/path/to/your/disk'):
try:
long_running_task()
break
except (IOError, OSError):
print("Disk disconnected during task, waiting...")
time.sleep(5) # wait for 5 seconds before trying again
else:
print("Disk not connected, waiting...")
time.sleep(5) # wait for 5 seconds before checking again