Linkedin Scraper, Parsel Selector undefined

108 Views Asked by At

getting this error i am frustrated now, somebody help!

for linkedin_url in linkedin_urls:
    driver.get(linkedin_url)
    sleep(5)
    Sel=Selector(text=driver.page_source)

name = Sel.xpath("//div[starts-with(@class, 'text-heading-xlarge inline t-24 v-align-middle break-words')]/text()")
driver.execute_script("arguments[0].extract_first();",name)
if name:
    name = name.strip()

job_title = Sel.xpath('//div[starts-with(@class,"pv-top-card-section__headline")]/text()')
driver.execute_script("arguments[0].extract_first();",job_title)
if job_title:
    job_title = job_title.strip()

Error:

Traceback (most recent call last):
  File "D:\PycharmProjects\LinkedInScraper\main.py", line 49, in <module>
    name = Sel.xpath("//div[starts-with(@class, 'text-heading-xlarge inline t-24 v-align-middle break-words')]/text()")
NameError: name 'Sel' is not defined
1

There are 1 best solutions below

0
Prophet On

I guess your problem is with indentation.
Try this:

for linkedin_url in linkedin_urls:
    driver.get(linkedin_url)
    sleep(5)
    Sel=Selector(text=driver.page_source)

    name = Sel.xpath("//div[starts-with(@class, 'text-heading-xlarge inline t-24 v-align-middle break-words')]/text()")
    driver.execute_script("arguments[0].extract_first();",name)
    if name:
        name = name.strip()
    job_title = Sel.xpath('//div[starts-with(@class,"pv-top-card-section__headline")]/text()')
    driver.execute_script("arguments[0].extract_first();",job_title)
    if job_title:
        job_title = job_title.strip()