MODUL grequests, How GET simple print URL and Response?

271 Views Asked by At

anyone can please explain me how i can split results for get just simple url and response? I have try so many time but nothing, for now i can print just like:

50
0.4110674999999999
........, [<Response [200]>], [<Response [200]>], [<Response [200]>]]
[......, ['http://example.com.com/catalogue/page-48.html'], ['http://example.com.com/catalogue/page-49.html'], ['http://example.com.com/catalogue/page-50.html']]

I need like

<Response [200]>
https://example.com/

Thanks so much.

Ps. Also why after installing module grequests I get this message on the console

C:\P3\lib\site-packages\grequests.py:22: MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['urllib3.util.ssl_ (C:\\P3\\lib\\site-packages\\urllib3\\util\\ssl_.py)', 'urllib3.util (C:\\P3\\lib\\site-packages\\urllib3\\util\\__init__.py)'].
  curious_george.patch_all(thread=False, select=False)

How I can fix it ? Uninstall complete python, install some patch or what ? Thanks!

import grequests
from bs4 import BeautifulSoup
import time

def get_urls():
    urls = []
    for x in range(1,51):
        urls.append(f'http://books.toscrape.com/catalogue/page-{x}.html')
    return urls

def get_data(urls):
    reqs = [grequests.get(link) for link in urls]
    resp = grequests.map(reqs)
    return resp


if __name__ == '__main__':
    start = time.perf_counter()
    urls = get_urls()
    url = len(get_urls())
    resp = get_data(urls)
    respo = len(get_data(urls))
    fin = time.perf_counter() - start

    resp_list = resp
    chunked_resp = list()
    chunk_size = respo
    urls_list = urls
    chunked_url = list()
    chunk_size = url

    print(urls)
    print(url)
    print(resp)
    print(respo)
    print(fin)  

    resp_list = resp
    chunked_resp = list()
        chunk_size = 1


    for i in range(0, len(resp_list), chunk_size):
        chunked_resp.append(resp_list[i:i+chunk_size])
    print(chunked_resp) 

    urls_list = urls
    chunked_url = list()
        chunk_size = 1


    for i in range(0, len(urls_list), chunk_size):
        chunked_url.append(urls_list[i:i+chunk_size])
    print(chunked_url)      
1

There are 1 best solutions below

5
Montauro s.r.o On

OK i have get a solution only for print url:

def get_data(urls):
    reqs = [grequests.get(link) for link in urls]
    resp = grequests.get(reqs)
    return resp
    
    
if __name__ == '__main__':
    start = time.perf_counter()
    urls = get_urls()
    resp = get_data(urls)
    resp = '\n'.join(resp)
    url = '\n'.join(urls)

http://books.toscrape.com/catalogue/page-48.html
http://books.toscrape.com/catalogue/page-49.html
http://books.toscrape.com/catalogue/page-50.html

     resp = '\n'.join(resp)
TypeError: can only join an iterable

But for a resp i get TypeError: can only join an iterable

Ps. i am started learn python max 1 month... :(