I'm looking up a book about CUDA.
On the chapter which explains the floating points of CUDA, I found something odd.
The book says that (1.00 * 1) + (1.00 * 1) + (1.00 * 0.01) + (1.00* 0.01) = 10. All the numbers are binaries. 0.01 refers to decimal 0.25.
So, in decimal serially adding 1 + 1 + 0.25 + 0.25 results in 2.
The book says why this happens ; after doing 1+1, it will ignore +0.25 since it's too small compared to the other operand(the result of 1+1, 2).
After this, they say that doing 0.25 + 0.25 + 1 + 1 will produce 2.5, since 0.5 is considered enough to be added with 1.
What is the meaning of this? How could the processor judge that 0.25 is too small compared to 2? Are there obvious standards for this?
Best How To :
The example implicitly declares a binary floating point format, which has an arbitrary precision exponent, but only 2 bits in the mantissa. All numbers are of format 1.xx * 2^n.
When one performs floating point addition, one must de-normalize or scale the arguments to have the same exponent.
0.25 = 1e-2 = 0.5e-1 = 0.25e0 = 0.125e1
2.00 = 1e1
But in the same base 0.125 = 0.001, which can't be represented with 2 bits of mantissa after the decimal.
Even if we add word length, it doesn't matter:
0.25 = 0.001000000000000000 (e=1)
2.00 = 1.000000000000000000 (e=1)
---------------------------------
2.25 = 1.001000000000000000 (e=1)
^^
The result will be those two bits after the decimal point, i.e. (1.00e1) = 2.
