ARM Technical Support Knowledge Articles

ROUNDING PROBLEMS WITH FLOATING-POINT NUMBERS

Applies to: General Topics

Answer


Information in this article applies to:


QUESTION

I have seen a strange problem with floating-point to integer conversion. After multiplying the floating-point number by 100, the wrong value is copied into the integer variable. For example:

float fp1;
int   bad;

void main (void) {
  fp1 = 2.34;
  fp1 *= 100;
  bad = fp1;  // bad is 233 but should be 234

  fp1 = 4.68;
  fp1 *= 100;
  bad = fp1;  // bad is 467 but should be 468

  fp1 = 1.17;
  fp1 *= 100;
  bad = fp1;  // bad is 116 but should be 117
}

In these cases, I have noticed that the floating-point number ends with .9999999. Is there a solution to this problem?

ANSWER

Floating-point operations are not 100% accurate because some numbers cannot be represented in the IEEE format without the loss of precision. Therefore, the results of multiplications can end with .999999.

The ANSI Standard specifies the following for floating-point to integer cast operations:

When a finite value of real floating type is converted to an integer type other than _Bool, the fractional part is discarded (i.e., the value is truncated toward zero).

This implies that the result of the conversion of 1.999999 to int will result in the value 1. If you need rounding you may simply add the value 0.5 to the result before converting it to an integer value. For example:

float fp1;
int   good;

void main (void) {
  fp1 = 2.34;
  fp1 *= 100;
  good = fp1 + 0.5;  // gives correct result: 234
}

SEE ALSO

Article last edited on: 2005-03-20 12:12:31

Rate this article

[Bad]
|
|
[Good]
Disagree? Move your mouse over the bar and click

Did you find this article helpful? Yes No

How can we improve this article?

Link to this article
Copyright © 2011 ARM Limited. All rights reserved. External (Open), Non-Confidential