How to get full title of website in google custom search api?

Question

This is my code.

def google_search(query, **kwargs):
    service = build("customsearch", "v1", developerKey=app.config.get("GOOGLE_API_KEY"))
    res = service.cse().list(q=query, cx=app.config.get("GOOGLE_CSE_KEY"),**kwargs).execute()
    return res['items']

response = google_search(query, num=pageSize, start=start)
for item in response:
    print('Title: ', item['title'])

The result is like the following:

As you can see in the screenshot, one of the titles ends with "...". It is omitted. But I want to get full title from the result.

I tried to get full title using selenium. But it takes long time.

def get_full_title(title, url):
    if title.endswith("..."):
        driver = webdriver.Chrome(options=options)
        driver.get(url)
        fullTitle = driver.title
        driver.quit()
        return fullTitle
    return title

So I thought, selenium is not the solution.

How to get full title? Is there any setting to get full title in programmable search engine?

Answer 1

If you have the URL available in the item list, you can use requests and bs4 (BeautifulSoup) to get the title pretty quickly:

Install requirements:

pip install requests
pip install bs4

Code:

import requests
from bs4 import BeautifulSoup

# If you need to fake a browser request
HEADERS = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}


def get_page_title(url: str) -> str:
  response = requests.get(url, headers=HEADERS)
  soup = BeautifulSoup(response.content, features="html.parser")
  return soup.title.string

url = "https://stackoverflow.com/questions/77901995/how-to-get-full-title-of-website-in-google-custom-search-api"

title = get_page_title(url)

print(title)

Running for the current SO page returns python - How to get full title of website in google custom search api? - Stack Overflow (no ellipsis).