How does AWS S3 determine which files need .md5sum files generated for them

61 Views Asked by At

I'm uploaded folders with large files to AWS S3.

In the folder, there are many large files. Below are some that AWS generated .md5sums for.

A0688.bam   bam 42.8 GB
A0688.bam.md5sum    md5sum  

A0688.vcf.gz    gz  398.9 MB
A0688.vcf.gz.md5sum md5sum  

A0688.hard-filtered.vcf.gz  gz  396.3 MB
A0688.hard-filtered.vcf.gz.md5sum   md5sum  

A0688.cnv.vcf.gz    gz  45.7 KB
A0688.cnv.vcf.gz.md5sum

I'm confused why some files, which are as large or larger as some with md5sums, don't seem to get the same treatment.

A0688.tn.tsv.gz gz  144.6 MB
A0688.target.counts.gz  gz  105.1 MB

No .md5sums

Ideally, I'd like a md5sum for all files... in the event the data needs to be transferred elsewhere, and I can use to do a data integrity check. But struggling to understand how AWS handles these sort of events.

1

There are 1 best solutions below

0
Fidi Naj On

I was mistaken. AWS doesn't create such files, it was my local software automatically producing them.