Background
I'm trying to create my own RSS feed reader, mainly drawing stories from Medium.
At the moment I'm testing and working off my own machine, and I can't access multiple IP addresses.
I have about 400 feeds I'm interested in reading from Medium, but when I try and read them in fast succession I get CloudFlare warnings. I presume this is because my current behaviour is unexpected/ unwanted.
I've read around and found that part of best practice is to wait 5-10 seconds between requests. Easy enough. I've also read it's best practice to provide information about what feed reader you are, contact information, subscriber number, and feed id. For example, a header like this:
{ user-agent: "my_feed_reader/1.0/ (+https://my_contact.com); feed-id:1; 1 - subscriber count" }
I'm happy to provide this also. But I presume I can do more to be a good internet citizen!
Question
What best practices to RSS feed readers follow to make sure they're not spamming websites?