Requests library in python keep working and don't response anything

233 Views Asked by Soroush At 25 January 2024 at 19:48

I try to send a request to a website, but after I run my code, it's running forever and getting no response. Please someone help me, here is my code:

import requests

req = requests.session()
url = "https://www.stepstone.de/"
response = req.get(url)
print(response.status_code)

I try put some headers in it, but it doesn't work for me. If you have any advice, I would appreciate it if you could share it with me.

Original Q&A

There are 1 best solutions below

Andrej Kesely On 25 January 2024 at 19:52 BEST ANSWER

By setting User-Agent header I was able to obtain response from the server:

import requests

headers = {
    "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
}

url = "https://www.stepstone.de/"

html_text = requests.get(url, headers=headers).text
print(html_text)

Prints:

<!DOCTYPE html>
<html lang="de">
  <head>
    
    <title>Jobs sind unser Job | Stepstone</title>
    
    <script type="application/ld+json">
    {
      "@context": "https://schema.org",
      "@type": "Organization",
      "url": "https://www.stepstone.de",
      "logo": "https://www.stepstone.de/sharedassets/logo/core/stepstone/logo.svg"
     }
    </script> 
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta charset="utf-8" />

...

Requests library in python keep working and don't response anything

There are 1 best solutions below

Related Questions in PYTHON

Related Questions in PYTHON-3.X

Related Questions in WEB-SCRAPING

Related Questions in PYTHON-REQUESTS

Related Questions in PYTHON-REQUESTS-HTML

Trending Questions

Popular # Hahtags

Popular Questions