How can I scrape phone numbers and website adresses using beautiful soup

Question

import requests
from bs4 import BeautifulSoup

url = 'https://www.yellowpages.ca/search/si/1/coffee/Toronto+ON'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

for listing in soup.find_all('div', class_='listings'):
    phone = listing.find('div', class_='phone')
    website = listing.find('div', class_='url')
    if phone:
        print(phone.text)
    if website:
        print(website.text)

I am trying to scrape phone numbers and website addresses from this site https://www.yellowpages.ca/search/si/1/coffee/Toronto+ON but my code returns no results so I can't understand where the error is

Answer 1

The problem is that you are not finding the right tags inside the response. I have modified your code:

import requests
import re
from urllib.parse import unquote
from bs4 import BeautifulSoup

url = 'https://www.yellowpages.ca/search/si/1/coffee/Toronto+ON'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

for item in soup.find_all('div', {"class": "listing__content__wrap--flexed"}):
    phone = item.find('a', {"title": "Get the Phone Number"}).get('data-phone')
    website = item.find('a', {"title": re.compile(r'Business Website')}).get('href')
    website = unquote(unquote(website.split('redirect=')[1]))
    print(f"phone: {phone}")
    print(f"website: {website}")

And the output is:

phone: 416-531-7755
website: http://sicilianicecream.com/sicilian-sidewalk-cafe
phone: 416-849-1499
website: http://carolescheesecakecafeyorkville.com/
phone: 647-388-9726
website: https://www.starbucks.ca/store-locator/store/1014678
phone: 416-363-8555
website: https://www.starbucks.ca/store-locator/store/1005445