how to make automation google search in python

Question

i have a 100 list in spreadsheet that content many keywords. i want to search in google automatically. i have try this code but does not work. what should i do ?

    from selenium import webdriver 

# Taking input from user 
search_string = input("Input the URL or string you want to search for:") 

# This is done to structure the string 
# into search url.(This can be ignored) 
search_string = search_string.replace(' ', '+') 

# Assigning the browser variable with chromedriver of Chrome. 
# Any other browser and its respective webdriver 
# like geckodriver for Mozilla Firefox can be used 
browser = webdriver.Chrome('chromedriver') 

for i in range(1): 
    matched_elements = browser.get("https://www.google.com/search?q=" +
                                    search_string + "&start=" + str(i)) 

when i try this code, there was error :

 File "C:\Users\iyyaj\script.py", line 13, in <module>
    browser = webdriver.Chrome('chromedriver')
  File "C:\Users\iyyaj\anaconda3\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 45, in __init__
    super().__init__(
  File "C:\Users\iyyaj\anaconda3\lib\site-packages\selenium\webdriver\chromium\webdriver.py", line 51, in __init__
    self.service.path = DriverFinder.get_path(self.service, options)
  File "C:\Users\iyyaj\anaconda3\lib\site-packages\selenium\webdriver\common\driver_finder.py", line 40, in get_path
    msg = f"Unable to obtain driver for {options.capabilities['browserName']} using Selenium Manager."
AttributeError: 'str' object has no attribute 'capabilities'

i just want to make in one code and the input are in excel or csv file

Answer 1

The first thing you need to fix that that instead of passing string object as an argument to webdriver.Chrome() you should pass of an instance of Options. Using Options is better and will make it easier for you to automate things when passing headless options as shown below.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

search_string = input("Input the URL or string you want to search for:")

search_string = search_string.replace(' ', '+')

# Chrome Options - This is how you're going to add options and location to the binary.

chrome_options = Options()

# Path to your Chrome WebDriver executable
chrome_driver_path = 'images/chromedriver_mac64'  # I've used mine for now.

# Set the executable path directly in options
chrome_options.binary_location = chrome_driver_path

# Assigning the browser variable with chromedriver of Chrome using Chrome Options
browser = webdriver.Chrome(options=chrome_options)

for i in range(1):
    matched_elements = browser.get("https://www.google.com/search?q=" +
                                  search_string + "&start=" + str(i))

This does work for me. Additionally, you might want to look into headless option which you can add using the below line of code. Headless option will allow you to run it without opening browser window as it may not be feasible for 100s of search results.

chrome_options.add_argument("--headless")

The ability to pass the Chrome WebDriver path directly to webdriver.Chrome() as a string argument (webdriver.Chrome(executable_path=chrome_driver_path)) was introduced in Selenium version 3.8.0. Before this version, the WebDriver path was set using a different method.

Link -> https://github.com/SeleniumHQ/selenium/blob/trunk/py/CHANGES

You might also get blocked for too many frequent requests if you do this often. You should give proper spacing and delay to not be blocked.