Question

Biggest possible rounding error when computing floating-point numbers

score 2.2k · Answer 1 · 2015-06-17 05:30:04

2.2k

Views

Biggest possible rounding error when computing floating-point numbers

Published on 17 June 2015 at 05:30

score 4.3k · Answer 2 · 2015-06-25 19:28:06

4.3k

Views

Calculations being being rounded SQL Server 2012

Published on 25 June 2015 at 19:28

score 124 · Answer 3 · 2016-12-18 13:03:46

124

Views

Rounding, variable from a .txt in Python 3 and extra

Published on 18 December 2016 at 13:03

score 1k · Answer 4 · 2016-12-31 23:49:29

1k

Views

Different Rounding Results in Java 6 and Java 8

Published on 31 December 2016 at 23:49

score 155 · Answer 5 · 2014-11-28 12:10:09

155

Views

Matlab numerical error and how to get the correct answer

Published on 28 November 2014 at 12:10

score 901 · Answer 6 · 2017-08-29 01:21:25

901

Views

Rounding Error in pandas

Published on 29 August 2017 at 01:21

score 14.7k · Answer 7 · 2011-02-07 12:48:20

14.7k

Views

PHP rounding error

Published on 07 February 2011 at 12:48

score 3.8k · Answer 8 · 2011-01-28 17:02:08

3.8k

Views

Is there a way to truncate scientific notation numbers in Javascript?

Published on 28 January 2011 at 17:02

score 1.4k · Answer 9 · 2013-08-27 12:00:06

1.4k

Views

Fast round up/down double in fortran?

Published on 27 August 2013 at 12:00

score 53 · Answer 10 · 2013-08-29 21:00:45

53

Views

How to determine which formula is subject to which type of error

Published on 29 August 2013 at 21:00

score 3.2k · Answer 11 · 2013-09-11 16:56:19

3.2k

Views

Avoid rounding errors in SSRS for small values

Published on 11 September 2013 at 16:56

score 482 · Answer 12 · 2020-10-09 03:07:34

482

Views

Incorrect rounding on gcc and Ubuntu

Published on 09 October 2020 at 03:07

score 1.4k · Answer 13 · 2012-01-18 13:58:36

1.4k

Views

Dealing with hours with no minutes when rounding DateTime to the 30 minutes

Published on 18 January 2012 at 13:58

score 2k · Answer 14 · 2013-12-31 12:00:36

2k

Views

acos() bad results with two close points in C

Published on 31 December 2013 at 12:00

score 379 · Answer 15 · 2014-12-12 03:26:44

379

Views

Qt: Roundoff of 12.625 by 2 returns 12.62

Published on 12 December 2014 at 03:26

score 325 · Answer 16 · 2014-12-14 15:04:33

325

Views

rounding in matlab

Published on 14 December 2014 at 15:04

score 887 · Answer 17 · 2014-12-14 01:00:09

887

Views

Converting time_duration to DATE

Published on 14 December 2014 at 01:00

score 19 · Answer 18 · 2023-10-27 22:38:11

19

Views

Alpha blending on int colors exceeds byte ranges

Published on 27 October 2023 at 22:38

score 53 · Answer 19 · 2023-11-28 21:36:59

53

Views

Precision Error while converting dollars/gallon to dollars/liter and back

Published on 28 November 2023 at 21:36

score 17 · Answer 20 · 2023-11-29 00:48:41

17

Views

Discrepancy when rounding the answer of a division where the answer is .5 remainder

Published on 29 November 2023 at 00:48

List Question

Biggest possible rounding error when computing floating-point numbers

Calculations being being rounded SQL Server 2012

Rounding, variable from a .txt in Python 3 and extra

Different Rounding Results in Java 6 and Java 8

Matlab numerical error and how to get the correct answer

Rounding Error in pandas

PHP rounding error

Is there a way to truncate scientific notation numbers in Javascript?

Fast round up/down double in fortran?

How to determine which formula is subject to which type of error

Avoid rounding errors in SSRS for small values

Incorrect rounding on gcc and Ubuntu

Dealing with hours with no minutes when rounding DateTime to the 30 minutes

acos() bad results with two close points in C

Qt: Roundoff of 12.625 by 2 returns 12.62

rounding in matlab

Converting time_duration to DATE

Alpha blending on int colors exceeds byte ranges

Precision Error while converting dollars/gallon to dollars/liter and back

Discrepancy when rounding the answer of a division where the answer is .5 remainder

Trending Questions

Popular # Hahtags

Popular Questions