I have read many questions about aria2c here on SO, e.g.
Automatically Retry aria2c download for using in a script
aria2c save download result table to a file
to construct the following command:
aria2c -j 10 --https-proxy=http://localhost:9080 -i links1.txt --save-session links2.txt
I have about 4000 links in a file (links1.txt) that I want to download using this command. Most of the downloads are successful but when I look at the contents of links2.txt it is identical to links1.txt. I thought the --save-session (see link 1 above) only would save failed downloads?
Another weird thing is that no files are downloaded. It doesn't matter, these 4000 links are "Click link to confirm that you want to receive our newsletter"-type of links, and I receive confirmation by e-mail "Welcome to our newsletter", in spite of nothing is downloaded, so apparently they are downloaded, kind of.
My question is: I have a list for 4000 links I want to download. Some of these downloads will fail (because my proxy randomly but intentionally rejects connections). I want a log of the failed downloads after my aria-session finishes so I can redownload the failed links. How do I achieve that?
Alternatively, how can I feed a list of 4000 links into something that can download them in parallel and supports a proxy (preferably SOCKS5)?