Puppeteer timeout on waitforselector

1.5k Views Asked by At

So I created my first web scrapper with Puppeteer and it works fine. I call the scrapper function in my server at 5mins time intervals but after calling it a couple times about 10 to 15 times, i begin to get this timeout error TimeoutError: waiting for selector `body > div.page-window.market-watch.compact > div > div.b > div.page-block > div > table > tbody > tr:nth-child(1) > td.symbol` failed: timeout 30000ms exceeded and I am sure the selector path is correct, the function even works well until after about 10 calls, then it breaks and would only work again if I restart the server. Please how can i fix this problem? My code bellow

puppeteer
    .launch({
      headless: true,
      args: ["--no-sandbox", "--disable-setuid-sandbox"],
    })
    .then(async (browser) => {
      console.log("Puppeteer Has launched...");
      const page = await browser.newPage();
      await page.setDefaultNavigationTimeout(120000); // set default timeout to 2mins
      try {
        await page.goto(baseUrl, { waitUntil: "load", timeout: 120000 });
        // wait for price section to mount
        await page.waitForSelector(
          "body > div.page-window.market-watch.compact > div > div.b > div.page-block > div > table > tbody > tr:nth-child(1) > td.symbol"
        );
      } catch {
         console.log(error)
      }

also as you can see in my code, i set the default timeout to be on 2mins but i get a timeout error of 30secs. Can anyone please explain why too?

0

There are 0 best solutions below