How do you handle requests in Pyodide / Stlite?

Question

From what I've read so far Pyodide does not work with requests library. Usage of pyodide.http.pyfetch is recommended. But as it is async it really trips me and I can't figure it out (pyodide.http.open_url won't work as I'm sending get request to an api and I need to add some headers).

So the question is - how do make a request to an api and stop the further execution of a function until coroutine finishes?

I'm trying to get the data from the API, and use it to create an object. In regular Streamlit it works flawlessly.

class FabmanData:
    LINKS: Final = {
        "members": "members?orderBy=name&order=asc",
        "resource": "resources?orderBy=name&order=asc",
        "bookings": "bookings?order=desc&limit=50&summary=false",
    }

    def __init__(self) -> None:
        self.members: pd.DataFrame = self.get_data("members")[
            ["id", "firstName", "lastName", "memberNumber"]
        ]
        self.resources: pd.DataFrame = self.get_data("resource")[
            ["id", "name", "state"]
        ]
        self.latest_bookings: pd.DataFrame = self.get_data("bookings")[
            ["id", "resource", "fromDateTime", "untilDateTime", "member"]
        ]

    @staticmethod
    def get_data(category) -> pd.DataFrame:
        url = f"{BASEURL}{FabmanData.LINKS[category]}"
        return pd.DataFrame(requests.get(url=url, headers=HEADERS).json())

    def get_resources_dict(self):
        return {
            resource: resource_id
            for resource, resource_id in zip(
                self.resources["name"], self.resources["id"]
            )
        }

But I can't figure it out with pyfetch . I get TypeError: coroutine object is not subscriptable . Thus I assume that get_data method returns coroutine and I don't know what to do to make it return a value. Currently it looks like this:

    @staticmethod
    async def get_data(category) -> pd.DataFrame:
        url = f"{BASEURL}{FabmanData.LINKS[category]}"
        response = await pyodide.http.pyfetch(url=url, headers=HEADERS)
        await asyncio.wait_for(response, timeout=10)
        data = pd.DataFrame(response.json())
        return data

Any suggestions?

Answer 1

Check out the pyodide-http package. It monkey-patches requests and urllib to work in a (mostly) browser-friendly way.

Answer 2

pyodide.http.pyfetch is the solution as you wrote.

You have to use it with await as the Pyodide document says because it's an async function and it returns a coroutine object that must be awaited to get the actual result. Note that the awaited result is a FetchResponse object, so you also have to use its method such as .string() or .bytes() to get the actual content (See the doc). Here is an example:

from pyodide.http import pyfetch

url = "https://example.com/some_file"
res = await pyfetch(url)
data = await res.bytes()

stlite allows to use await at the top level, so you can write code like above.

Also, as usual, you can use pyodide.http.pyfetch in an async function with await. In that case, you would await the outer async function with a top level await like this:

from pyodide.http import pyfetch

async def main():
    url = "https://example.com/some_file"
    res = await pyfetch(url)
    data = await res.bytes()

await main()

This section of stlite's README explains these things with the example of pyfetch.