How to prevent token-bucket based admission from going over target capacity

44 Views Asked by At

If I am allowed 10MB/s bandwidth utilization, and I model that with a token-bucket with this API

class TokenBucket {
 public:
  void setRefillRate(std::int64_t tokensPerMillisecond);
  bool tryConsume(std::int64_t tokens);
};

Where a token means a single byte. And bandwidth is allowed to be consumed whenever tryConsume() returns true. Then what rate of replenishing should I set for this to work? If I set 10,000 tokens per millisecond (to model 10MB/s), then isn't the following possible?

  1. At time t=0ms, bucket has 10,000,000 tokens (for 10MB)
  2. At time t=1ms, client requests 10,000,000 tokens, this is allowed
  3. At time t=999ms, client requests additional 9,980,000 tokens, this is allowed

At (3) the request is allowed because the token bucket fills up 9,980,000 tokens (based on the refill policy of 10,000 tokens per millsecond. And there have been 998ms between point (1) and (2) above). Now I've violated the 10MB/s limitation, and gone over by almost 2x the limit.

How do typical implementations get around this problem?

0

There are 0 best solutions below