Go , Stripe, Iterators, and Abstraction

Question

GoLang newbie here. I'm working on a project that reaches out to stripe and pulls the contents of different stripe tables (Product and Plan, for example). Eventually, I will be pulling data from 40-50 tables at Stripe, so I'm looking for ways to keep things maintainable.

Currently, I have functions that pull all the data I need for each table. By and large the functions for each table are the same, with the differences being building iterators and getting the data from a particular table.

Here are the functions. I stripped out all the things not pertinent to the question for clarity.

func getAllCoupon(folderName string) {
    // do a bunch of setup stuff, init some vars, etc

    stripeIter := getCouponIter("")
    for okToContinue {
        lastId := ""
        for stripeIter.Next() && okToContinue {
            p := stripeIter.Coupon()
            
            // do some things with p

            }
        }

        if pageLineCount == 0 {
            okToContinue = false
        } else {
            stripeIter = getCouponIter(lastId)
        }
    }

    . . .
}

func getCouponIter(id string) *coupon.Iter {
    params := &stripe.CouponListParams{}

    params.Filters.AddFilter("limit", "", config.Config.Input.StripeMaxPageSize)
    if id != "" {
        params.Filters.AddFilter("starting_after", "", id)
    }
    return coupon.List(params)
}

So rather than have 40-50 copies of getAll(), I'd like to have one, and pass in a function that yields an iterator - getCouponIter in this case.

I have looked into the stripe package code, and attempted to change the getAll function signature to include a parameter:

func getAllCoupon(folderName string, getIter func(string) *stripe.Iter)

And with some tweaking, I can get everything to line up. Except the return type from getCouponIter. The return type from that func is the return type of *coupon.Iter.

So, I'm curious. If I want to wind up with just one GetAll function and just pass in a function that queries stripe and passes back an iterator, how should I do that?

Edit: hanzo rightfully pointed out that this isn't really related to stripe payments, other than I have to use their framework to iterate over their returned data. It's more related to how I abstract the iterators somehow so I dont have 40 copies of the same function floating around.

Edit: Burak Serdar asked if the iterators all have the same functions. They do. The issue is that I'd like to pass a function to the getAll function that generates those iterators when needed. The iterators are paged (for events, for example), and so I might need to regenerate an iterator during the course of execution.

So I have functions that generate the iterators that look like this:

func getCouponIter(id string) *coupon.Iter
func getCustomerIter(id string) *customer.Iter
etc

I'd like to pass one of these iterator generator functions to the function that pulls all the data. The issue I'm running into is that to define an interface for these, they need the same return type (if I'm wrong, please tell me) and I cannot force these into returning anything other than what they are returning.

Answer 1

Whenever you need to copy/paste code, you can probably use generics.

I believe stripe iterators can be expressed using:

type StripeIter interface {
  Next() bool
  Current() interface
}

Using this, you can create a generic type-safe iterator:

type GenericIter[T any] struct {
  StripeIter
}

func (g *GenericIter[T]) Item() T {
    return g.Current().(T)
}

Then, you can define GetAll as:

func getAll[T any](folderName string, get func() *GenericIter[T]) {

   iter:=get()
   // Iterate using iter and type T
}

Call it as:

getAll[*stripe.Something](folder, &GenericIter{getSometing(id)})

where getSomething returns a Stripe iterator.

There are probably missing pieces, but this should give you the general idea.

Answer 2

Based on the fact that all iterators have the same functions, you can make an interface that encapsulates those functions:

type Iter interface {
    // iteration functions
}

Then update getAll's function parameter to return that interface:

func getAll(getIter func(string) Iter) {
    iter := getIter("some string")
    // do something with iter, create return data, paginate, etc.
}

This way, no matter whether you are yielding a *coupon.Iter or a *customer.Iter, you can use the same getAll function.

Again, this is assuming all of the iterations have the same functions (same names, same types).

Answer 3

So the generics answer from @Burak was super helpful. I had trouble getting it working, so I started digging around in Stripe's code.

Stripe returns iterators of type *customer.Iter or *product.Iter, but these are just wrappers for Stripe's iterator with some helpful additional stuff: stripe.Iter.

So in my getCouponIter(string) *coupon.Iter code above, I changed the return value from: return coupon.List(params) to

tmp := coupon.List(params)
return tmp.Iter

This returns stripe.Iter, which they all share, and so now the signature is the same for all of the getIter functions.

So the signature for getAll is now:

func getAll(folderName string, getIter func(string) *stripe.Iter)

And that's what I needed. Thank you @burak