My name is Jim Hogg, a Program Manager in the Compilers team.<\/p>\n
We would like your feedback on a feature of the Visual C++ compiler that affects the code we generate for floating-point operations. Your answers will help determine what we do. You can vote via survey<\/a> — it should not take you more than a few minutes to fill out!<\/strong><\/p>\n The C and C++ languages let you declare variables of type float<\/span> or double<\/span>. We call these “floating-point” types. And the Visual C++ compiler lets you specify how it should treat calculations involving these floating-point variables. The options we discuss in this blog are \/fp:fast<\/span> and \/fp:precise<\/span>.<\/p>\n Today’s default is \/fp:precise<\/span>. This blog asks for your feedback on whether we should change the default to \/fp:fast<\/span>. This change would make your code run faster; but might reduce the accuracy of the results, depending upon the calculations involved.<\/p>\n There are many excellent articles that explain floating-point in detail. This blog, by contrast, includes an appendix providing a homely overview – enough for you to form an opinion on the issue of changing the default to \/fp:fast<\/span>. Readers who want to dig deeper may explore the links at the end of this post.<\/p>\n [Note that you have control either way: you can specify that the compiler should follow \/fp:fast<\/span> or \/fp:precise<\/span> down to the level of each .cpp file, or even each function]<\/p>\n Please let us know what you think, after reading this blog post, by filling out this short survey<\/a>.<\/p>\n This blog uses the notation 1.2E+34 as short-hand for 1.2 * 1034<\/sup>. If the “fraction” part is 1.0, we abbreviate further: so 1.0E+23 is shortened to E+23.<\/p>\n In C++, a float<\/span> can store a value in the 3 (approximate) disjoint ranges { [-E+38, -E-38], 0, [E-38, E+38] }. Each float<\/span> consumes 32 bits of memory. In this limited space, a float<\/span> can only store approximately 4 billion different values. It does this in a cunning way, where adjacent values for small numbers lie close together; while adjacent values for big numbers lie far apart. You can count on each float<\/span> value being accurate to about 7 decimal digits.<\/p>\n We all understand how a computer calculates with int<\/span>s. But what about float<\/span>s? One obvious effect is that if I add a big number and a small number, the small one may simply get lost. For example, E+20 + E-20 results in E+20 – there are not enough bits of precision within a float<\/span> to represent the precise\/exact\/correct value.<\/p>\n Similarly, each calculation using float<\/span>s has to round the precise result to fit within the space available (actually 23 bits). Depending upon the calculation, the result may differ a little, or a lot, from the mathematical result (the one you’d get if you had lots and lots of bits available).<\/p>\n Here is a simple example:<\/p>\n You would expect this program to add inc<\/span> (one-millionth) to sum<\/span>, one million times, resulting in an answer of 1.0. But one-millionth can only be represented approximately as a float<\/span> (actually 0x358637bd), so the result obtained is not 1.0, but 1.009039.<\/p>\n To scare ourselves even more, note that calculations with float<\/span>s do not obey all rules of algebra. For example, associativity of addition states that: (a + b) + c == a + (b + c). But floats don’t quite abide by that rule. For example:<\/p>\n So results can differ, depending upon the order in which we perform the operations.<\/p>\n Floating-point calculations don’t obey all the laws of algebra – but in many cases, it’s “close enough” to the mathematically precise answer. [Eg: if we calculate the stress on a bridge truss to be 1.2593 tonnes, but the accurate value is 1.2592 tonnes, we’re likely happy: the bridge won’t fall down]<\/p>\n By throwing the \/fp:fast<\/span> switch, you tell the compiler it should pretend that float<\/span>s (and double<\/span>s) obey the rules of simple algebra (associativity and distributivity). This allows the compiler to optimize your code so that it runs faster. It trades off accuracy for speed. (It also lets the compiler play a fast-and-loose with that sub-species of floats called NaN<\/span>s – “Not a Number” – see below)<\/p>\n Just how much speedup will you get by enabling \/fp:fast<\/span>? Here are results we found using a few common benchmarks:<\/p>\nOK, I’m Still Reading . . .<\/h3>\n
Notation<\/h3>\n
Floating Point Basics<\/h3>\n
Floating Point Calculations<\/h3>\n
int main() {
float inc = 0.000001, sum = 0.0;
for (int i = 1; i <= 1000000; ++i) sum += inc;
printf(\"Sum = %f \\n\", sum);
}<\/code><\/pre>\n\n
What Does \/fp:fast Do?<\/h3>\n
How Fast is \/fp:fast?<\/h3>\n