Python requests to Pixabay

Question

I'm trying to send a request to pixabay.

Here's my code

import requests

url = 'https://pixabay.com'
header = {
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.100 Safari/537.36'
}

req = requests.get(url, headers=header)

print(req.status_code)
print(req.headers)
print(req.text)

It won't work(403 error). How can I do to work?

Answer 1

Pixabay has Cloudflare security that requires you to solve a captcha if you connect from a blacklisted IP.

In order to bypass this, you have to first connect via a browser and then copy the headers and cookies into your python script. This works for me, but you have to replace the parts like __cfduid which is your cloudflare fingerprint in order to access the website. Also check that your User-Agent is correct.

import requests

url = 'https://pixabay.com/'
header = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3835.0 Safari/537.36',
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Accept-Language': 'en-US,en;q=0.5',
    'Accept-Encoding': 'gzip, deflate',
    'Upgrade-Insecure-Requests': '1',
    'Host': 'pixabay.com'
}

cookie = {
    '__cfduid': '<redacted>',
    'cf_clearance': '<redacted>',
    'anonymous_user_id': '<redacted>',
    '_sp_ses.aded': '*',
    '_sp_id.aded': '<redacted>',
    'is_human': '1',
    'client_width':'1540'
}
req = requests.get(url, headers=header, cookies=cookie)

print(req.status_code)
print(req.headers)