are computations with large floats less accurate then with small floats

Question

Is this statement correct? :

Computations with large numbers is less accurate due to the logarithmic distribution of floating point numbers on the computer.

So that means computing with values around 1 is more accurate (because of rounding errors) then the same computations where each number has been scaled with 1e20 for example?

Answer 1

Short answer:

Yes the statement is correct, larger floating point numbers are less precise than smaller ones.

Details:

Floating point numbers have a fixed number of bits assigned to the mantissa. If the number that is being represented requires more bits than are in the mantissa then it will be rounded. So a smaller number can be represented more precisely.

To make this more concrete I wrote the following program that adds progressively smaller values to a large floating point number and a small one. Also to show the difference I included a double precision floating point that does not have rounding. But the double would experience the same problem if the mantissa were even larger.

#include <stdio.h>

int main() {
  float large_float, small_float, epsilon;
  double large_double, small_double;

  large_float = 1 << 20;
  small_float = 1;
  epsilon = 0.1;

  large_double = large_float;
  small_double = small_float;

  printf("large_float\t large_double\t small_float\t small_double\t epsilon\n");

  for(int i = 0; i < 10; i++) {
    printf("%f\t %f\t %f\t %f\t %f\n", large_float, large_double,small_float, small_double, epsilon);
    large_float += epsilon;
    large_double += epsilon;
    small_float += epsilon;
    small_double += epsilon;
    epsilon /= 2;        
  }

  return 0;
}

Running this program produces the following output:

large_float      large_double    small_float     small_double    epsilon
1048576.000000   1048576.000000  1.000000        1.000000        0.100000
1048576.125000   1048576.100000  1.100000        1.100000        0.050000
1048576.125000   1048576.150000  1.150000        1.150000        0.025000
1048576.125000   1048576.175000  1.175000        1.175000        0.012500
1048576.125000   1048576.187500  1.187500        1.187500        0.006250
1048576.125000   1048576.193750  1.193750        1.193750        0.003125
1048576.125000   1048576.196875  1.196875        1.196875        0.001563
1048576.125000   1048576.198438  1.198437        1.198438        0.000781
1048576.125000   1048576.199219  1.199219        1.199219        0.000391
1048576.125000   1048576.199609  1.199609        1.199609        0.000195

As you can see the large_float value is less precise than the small_float which can lead the final result to be less accurate as well.