Fluent interfaces with pipelining or method chaining in Python

Question

I am coming to Python from F# and Elixir, and I am struggling heavily in terms of cleanly coding data transformations. Every language I have ever used has had a concept of a pipeline operator and/or method chaining, and so with Python, I am confused on finding an easy way to accomplish this that doesn't stray away from so-called "Pythonic" code.

Here's a simple collection of some processing functions I might have:

def convert_int_to_bool(integer: int) -> bool:
    match integer:
        case 0:
            return False
        case 1:
            return True
        case _:
            ValueError(
                f"Integer value must be either 0 or 1 to be converted to `bool`. Given: {integer}"
            )

def convert_string_to_characters(string: str) -> list[str]:
    return [character for character in string]

In Python, I can do something like:

def test(response: str) -> <some type>:
    [a, b, c, d] = map(convert_int_to_bool, map(int, convert_string_to_characters(response)))
    ...

But this is non-ideal, even in a simple case of mapping over the list twice. Well, then I know I could do something like:

[a, b, c, d] = [convert_int_to_bool(int(character)) for character in response]

That's okay, but it again doesn't scale all that well to a chain of processing functions, especially if there's a filter inside there. So what I'd like to do is something like:

[a, b, c, d] = response.convert_string_to_characters().map(int).map(convert_int_to_bool)

or

[a, b, c, d] = response |> convert_string_to_characters() |> map(int) |> map(convert_int_to_bool)

For the first proposed way with method chaining, it seems I could potentially do this by extending the built-in types of str and list, but then that has issues from I have read about not integrating well with the built-in literal constructions of those types.

---

Are there any libraries or ways of overloading/overriding built-in types or defining custom operators that would allow me to do this in a clean way? Thank you.

Answer 1

Are there any libraries or ways of overloading/overriding built-in types or defining custom operators that would allow me to do this in a clean way?

I suggest taking look at popular external package pandas or more precisely pandas.Series.apply. Simple example of applying 3 functions to Series of integers in succession.

import pandas as pd
def square(x):
    return x ** 2
def evenify(x):
    return 2 * (x // 2)
def add_one(x):
    return x + 1
s = pd.Series([1,2,3,4,5,6,7,8,9])
s2 = s.apply(square).apply(evenify).apply(add_one)
print(s2)

gives output

0     1
1     5
2     9
3    17
4    25
5    37
6    49
7    65
8    81
dtype: int64

You might use Series just like list in many respects (e.g. to retrieve 1st element from changed do s2[0]).

seems I could potentially do this by extending the built-in types of str and list, but then that has issues from I have read about not integrating well with the built-in literal constructions of those types.

If you want to avoid external dependency, but want list-like which allow chaining I suggest making use of collections.UserList e.g. by doing

import collections
class MyList(collections.UserList):
    def map(self, func):
        return MyList(map(func, self.data))
lst = MyList([1,2,3,4,5])
lst.map(square).map(evenify).map(add_one)