Web scraping using Selenium (not working)

65 Views Asked by At

I'm a total beginner to webscraping using Selenium. I am trying to open a specific google profile (since all the sites would be already signed in). I'm happy that the code is able to open the specific profile window (profile being default). However, it fails to open the site in the window and start scraping. Here's the code:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time

# Installed chromedriver.exe

# Changing some arguments to set Chrome profile
options = webdriver.ChromeOptions()
# options.add_argument("--headless")
options.add_argument("--no-sandbox")
options.add_argument("--disable-gpu")

# Location where Chrome stores profiles
# options.add_arguments = {"user-data-dir": r"C:\Users\Kavipriyan\AppData\Local\Google\Chrome\User Data\Default"}
options.add_argument(r"--user-data-dir=C:\Users\Kavipriyan\AppData\Local\Google\Chrome\User Data")

# Profile name
options.add_argument(r"--profile-directory=Default")

service = Service(executable_path="chromedriver.exe")
driver = webdriver.Chrome(options=options, service=service) 

driver.get("https://twitter.com/home")
time.sleep(3)

driver.close()

This code on the side gave this huge error in the console too (I'm not sure which parts are the exactly required ones so I pasted whatever it printed out):

Opening in existing browser session.
Traceback (most recent call last):
  File "d:\COMPUTER FILES\My Stuff\Code Programs - Python\Twitterscraping 2.0\Twitterscraper 3.2.py", line 23, in <module>
    driver = webdriver.Chrome(options=options, service=service)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Kavipriyan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\selenium\webdriver\chrome\webdriver.py", line 45, in __init__
    super().__init__(
  File "C:\Users\Kavipriyan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\selenium\webdriver\chromium\webdriver.py", line 61, in __init__
    super().__init__(command_executor=executor, options=options)
  File "C:\Users\Kavipriyan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\selenium\webdriver\remote\webdriver.py", line 208, in __init__
    self.start_session(capabilities)
  File "C:\Users\Kavipriyan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\selenium\webdriver\remote\webdriver.py", line 292, in start_session     
    response = self.execute(Command.NEW_SESSION, caps)["value"]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Kavipriyan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\selenium\webdriver\remote\webdriver.py", line 347, in execute
    self.error_handler.check_response(response)
  File "C:\Users\Kavipriyan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\selenium\webdriver\remote\errorhandler.py", line 229, in check_response 
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: exited normally.
  (session not created: DevToolsActivePort file doesn't exist)
  (The process started from chrome location C:\Program Files\Google\Chrome\Application\chrome.exe is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
        GetHandleVerifier [0x00007FF667BC7062+63090]
        (No symbol) [0x00007FF667B32CB2]
        (No symbol) [0x00007FF6679CEC65]
        (No symbol) [0x00007FF667A00777]
        (No symbol) [0x00007FF6679FB2F4]
        (No symbol) [0x00007FF667A40BFB]
        (No symbol) [0x00007FF667A40830]
        (No symbol) [0x00007FF667A36D83]
        (No symbol) [0x00007FF667A083A8]
        (No symbol) [0x00007FF667A09441]
        GetHandleVerifier [0x00007FF667FC25CD+4238301]
        GetHandleVerifier [0x00007FF667FFF72D+4488509]
        GetHandleVerifier [0x00007FF667FF7A0F+4456479]
        GetHandleVerifier [0x00007FF667CA05A6+953270]
        (No symbol) [0x00007FF667B3E57F]
        (No symbol) [0x00007FF667B39254]
        (No symbol) [0x00007FF667B3938B]
        (No symbol) [0x00007FF667B29BC4]
        BaseThreadInitThunk [0x00007FFDF5047344+20]
        RtlUserThreadStart [0x00007FFDF56426B1+33]

Thanks in advance!

0

There are 0 best solutions below