double precision and single precision floating point numbers?

Question

I was wondering why double precision and single precision numbers are sometimes equal and sometimes not. For example when I have the following they are not equal:

import numpy as np

x=np.float64(1./3.)
y=np.float32(1./3.)

but the following are equal:

x=np.float64(3.)
y=np.float32(3.)

I understand why the first set of x and y is not equal but I am not quite sure as to why the second set is equal.

Answer 1

This answer assumes single is IEEE 754 32 bit binary floating point, and double is the corresponding 64 bit type.

Any value that can be represented exactly in a single can also be represented exactly as a double. That is the case for 3.0. The closest single and the closest double both have value exactly 3, and are equal.

If a number cannot be represented exactly in a single, the double is likely to be a closer approximation and different from the single. That is the case for 1.0/3.0. The closest single is 0.3333333432674407958984375. The closest double is 0.333333333333333314829616256247390992939472198486328125.

Both single and double are binary floating point. A number cannot be expressed exactly unless it is equal a fraction of the form A/(2**B), where A is an integer, B is a natural number, and "**" represents exponent. Numbers such as 0.1 and 0.2 that are terminating decimal fractions but not terminating binary fractions behave like 1/3.0. For example, the closest single to 0.1 is 0.100000001490116119384765625, the closest double is 0.1000000000000000055511151231257827021181583404541015625

Answer 2

Imagine you have to represent 1/3 in base 10 with a limited number of digits.

With 2 digits (let's call this single precision), it will be 0.33
With 4 digits (double precision) it will be 0.3333
So the two approximations are not equal.

Now transpose this to representing 1/5 in base 2. You also need an infinite number of bits (binary digits) - it's 0.001100110011....

With 24bits significand (IEEE 754 single precision) and 53 bits significand (double precision), the two floating point approximation will be different.

Same for 1/3...

If the number can be represented exactly without approximation in single precision, then both representation will be equal.

That is a numerator fitting in less than 25 bits (without the trailing zeros), and a denominator being a power of 2. (but not too high an exponent both in numerator nor in denominator...).

for example 1/2 3/2 5/2 ... 1/4 3/4 5/4 etc... will have equal representation.

2^24+1 won't have same representation.
But 2^60 will.

There are other case when representation will be inexact but approximation will be the same:
2^54+1 will have same float and double approximation.
so will 1+2^-60 for example.