Understanding Python's Decimal.quantize method

28 Views Asked by At

I am using the following approach to truncate my floating point values to X decimal places (12 in my case).

Decimal(val).quantize(Decimal(1).scaleb(-12), rounding=ROUND_UP)

I don't really care much about whether the value is rounded up or down but when I ran a few tests, with various values, I noticed a discrepancy with one use-case

With the following code, I understand each use case except for t4:

t1 = Decimal(-0.04).quantize(Decimal(1).scaleb(-2), rounding=ROUND_UP)
print(f"t1: {t1}")
t2 = Decimal(-0.05).quantize(Decimal(1).scaleb(-2), rounding=ROUND_UP)
print(f"t2: {t2}")
t3 = Decimal(-0.22).quantize(Decimal(1).scaleb(-2), rounding=ROUND_UP)
print(f"t3: {t3}")

t4 = Decimal(-0.092314564913).quantize(Decimal(1).scaleb(-12), rounding=ROUND_UP)
print(f"t4: {t4}")
t5 = Decimal(-0.092314564914).quantize(Decimal(1).scaleb(-12), rounding=ROUND_UP)
print(f"t5: {t5}")
t6 = Decimal(-0.09231456491391342377614447500).quantize(Decimal(1).scaleb(-12), rounding=ROUND_UP)
print(f"t5: {t6}")

The output is as follows:

t1: -0.05
t2: -0.06
t3: -0.23
t4: -0.092314564913
t5: -0.092314564915
t5: -0.092314564914

The output values for t1, t2, t3, t5, t6 made sense to me but can't understand why t4 outputs -0.092314564913 and does not round up to -0.092314564914 like the behavior of t1 to t3.

0

There are 0 best solutions below