How to complete scrape a web page with resources?

145 Views Asked by At

I want total scrape a web page with all resources (.css, .html, .favicon, ,js etc...)

Currently, I'm using this command to do that.

wget -E -H -k -K -e robots=off -p https://example.com -P ./myDir

However, on some pages, the downloaded folder has missing files.

For example, this is a website template and there's no .html file.

wget -E -H -k -K -e robots=off -p https://nicepage.com/html-templates/preview/zero-waste-166766?device=desktop -P ./myDir

Do you know any better way to scrape web pages? I'm using node.js so you can advise me of some node.js packages too.

0

There are 0 best solutions below