Selenium Wire webdriver cannot browse site

54 Views Asked by At

I am using Python and Selenium (selenium-wire) to automatically control a browser for two sites. Browsing most sites like: https://www.google.com
https://platform.cmcmarkets.com/
using a chromedriver under seleniumwire works fine.

However when browsing to: https://app.plus500.com/

A Chrome error page is displayed with the message:

This site can’t be reached  
The web page at https://app.plus500.com/ might be temporarily down or it may have moved permanently to a new web address.
ERR_HTTP2_PROTOCOL_ERROR

selenium-wire is a python module that extends selenium by allowing the capture of requests and responses. The site can be reached using plain selenium webdrivers, so it seems the issue lies with the selenium-wire package in use.

Importing the Root Authority certificate issued by selenium-wire into Chrome allows browsing HTTPS sites without the blocking "Your connection is not private" screen.

What is odd is that while certificate viewer for google.com (and other sites) shows the Selenium Wire CA:

Common Name (CN)    www.google.com
Organisation (O)    <Not part of certificate>
Organisational Unit (OU)    <Not part of certificate>
Common Name (CN)    Selenium Wire CA
Organisation (O)    <Not part of certificate>
Organisational Unit (OU)    <Not part of certificate>

app.plus500.com does not:

Common Name (CN)    *.plus500.com
Organisation (O)    Edgio, Inc.
Organisational Unit (OU)    <Not part of certificate>
Common Name (CN)    DigiCert Global G2 TLS RSA SHA256 2020 CA1
Organisation (O)    DigiCert Inc
Organisational Unit (OU)    <Not part of certificate>

The plus500.com site then reports that I am offline, possible because some app assets failed to load as a result. However it does report the certificate is valid so that does not seem to be the issue.

I've tried toggling the mitmproxy backend in the selenium-wire options also with no success.

Packet captures to the server on port 443 show the failed connection sequence corresponds with some extra TCP RST from the server, but I cannot interpret much else from them.

Please, any help diagnosing this would be great!

1

There are 1 best solutions below

0
user1330734 On

Disabling the mitm_http2 flag within seleniumwire_options, as recommended by the seleniumwire author for similar issues, did not work for me:

seleniumwire_options = {
    'mitm_http2': False,
}

I include it here as other users have reported success with this for other sites.

Disabling HTTP2 in my chromedriver instance is the workaround I settled for:

chrome_options.add_argument('--disable-http2')