XPath inside HTTP request

Question

I want to scrape daily value changes from public page:

[1] http://www.example.com/page.html

I've got a full xpath:

[2] /html/body/div[5]/table[1]/tbody/tr[4]/td[2]/@data-val

or command that works to get that value thru Chrome console:

[3] $x("string(/html/body/div[5]/table[1]/tbody/tr[4]/td[2]/@data-val)")

But i'm stuck how to make/encode [1] + [2]/[3], that could retrive that data-val using just a http request? (i'm using integromat, to make http request, but failed to find any reasonable examples).

Answer 1

You will have to make a get request to load the document. Afterwards you can use a library to extract the value by xpath.

Please Provide more Info on which Language/Framework you are on. Here is an example in python for a refrence:

from scrapy.selector import Selector
from scrapy.http import HtmlResponse
response = HtmlResponse(url='http://example.com', body=body)
Selector(response=response).xpath('//span/text()').get()