JS Nightmare not loading specific URL?

228 Views Asked by At

I have started small scraping project and stop myself on the start. While I can do the parsing part, I have problems to load the page. I've reduced the code to as little as possible and still not working for requested URL

// Required Libraries
const cheerio = require("cheerio");
const Nightmare = require("nightmare");

const scraping = (url) => {
  url = "https://www.portaldrazeb.cz/drazba/144ex02170-20-62-rb3p1";
  // All scraping will be done here
  let nightmare = new Nightmare({ show: true });
  const nightmarePromise = new Promise((resolve, reject) => {
    nightmare
      .goto(url)
      .wait("main")
      .evaluate(() => {})
      .end()
      .then((response) => {
        var json = {
          title: "",
          picture: "",
          price: "",
          starting_price: "",
          description: "",
          address: "",
          map: "",
          customer: { name: "", email: "", phone: "" },
        };
        resolve(json);
      });
  });

  return nightmarePromise.then((jsonObj) => {
    nightmare = null;
    return jsonObj;
  });
};

module.exports = {
  scraping: scraping,
};

While the code works reliably for whatever URL it is not working for the URL requested. The output is correct only in 30% calls, in the rest the call starts Nightmare browser, the page starts loading, then several resets occurs in a second and the browser gets closed and following error is displayed

(node:18468) UnhandledPromiseRejectionWarning: Error: navigation error at unserializeError (C:\Users\Tomas\Dropbox (Personal)\Server\ScrapReality\node_modules\nightmare\lib\ipc.js:162:13) at EventEmitter. (C:\Users\Tomas\Dropbox (Personal)\Server\ScrapReality\node_modules\nightmare\lib\ipc.js:89:13) at Object.onceWrapper (events.js:422:26) at EventEmitter.emit (events.js:315:20) at ChildProcess. (C:\Users\Tomas\Dropbox (Personal)\Server\ScrapReality\node_modules\nightmare\lib\ipc.js:49:10) at ChildProcess.emit (events.js:315:20) at emit (internal/child_process.js:876:12) at processTicksAndRejections (internal/process/task_queues.js:85:21) (node:18468) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 3)

0

There are 0 best solutions below