Rounding question related to RoundNanoFractionTo

The offical for ZeosLib 7.3 Report problems, ask for help, post proposals for the new version of Zeoslib 7.3/v8
Quick Info:
-We made two new drivers: odbc(raw and unicode version) and oledb
-GUID domain/field-defined support for FB
-extended error infos of Firebird
-performance ups are still in queue
In future some more feature will arrive, so stay tuned and don't hassitate to help
Post Reply
MJFShark
Expert Boarder
Expert Boarder
Posts: 211
Joined: 04.06.2020, 13:59

Rounding question related to RoundNanoFractionTo

Post by MJFShark »

This may be something I don't understand, but RoundNanoFractionTo seems to give some unexpected results.

If you call:
x := RoundNanoFractionTo(123456789, 3);

The result is 124000000. Am I crazy or should that be 123000000? I'm aware of different rounding schemes (banker's rounding and such) but I can't figure out this one. I see that the way it works is to get the "rest" of the digits (456789) and then it compares it to 444445 and if it's greater it adds one which is where the 124 comes from.
It seems to be that
const HalfFractModulos: array [TFractionRoundToScale] of Cardinal = ( 444444445, 44444445, 4444445, 444445, 44445, 4445, 445, 45, 5,0);
should be:
const HalfFractModulos: array [TFractionRoundToScale] of Cardinal = ( 500000000, 50000000, 5000000, 500000, 50000, 5000, 500, 50, 5,0);

-Mark
Post Reply