If you’re a developer and I ask you about numeric rounding, you’d probably think of something like this:
- Jump to the decimal place that we are rounding to
- Look at the digit to the right of it
- Based on that digit and our rounding rule, make a decision
We can round up if we see non-zero in that spot (also known as “ceil”), or we can round down regardless of the digit (aka “floor” or truncate), or we can round >= 5 up and truncate if < 5 (banker’s rounding).
All pretty straightforward stuff, but Ruby’s BigDecimal class, often used for currencies, has a few other rounding options and if your expectation of how rounding works is what I just described above (as mine was) then the behaviour might be a bit surprising to you.
The journey begins
BigDecimal has a ROUND_HALF_DOWN option for BigDecimal#round. Let’s see what the documentation says:
ROUND_HALF_DOWN, :half_down
round towards the nearest neighbor, unless both neighbors are equidistant, in which case round towards zero.
Hmm, yeah, okay. But what does that mean? The actual behaviour of the method may be a bit counter-intuitive. Let’s start with the other rounding types and see how things stack up.
# require 'bigdecimal'
=> true
# BigDecimal.new(1.551, 10).round(1, :up).to_f
=> 1.6
# BigDecimal.new(1.551, 10).round(1, :down).to_f
=> 1.5
So far so good. We’re creating a BigDecimal here. I’ve given it a float precision of 10 digits but only so we don’t have to worry about it being an issue. Then we call #round. We’re rounding to one decimal place. Finally, I’m casting back to a float as I find the default 0.15e1 format less readable.
The plot thickens
Rounding up and down to one decimal place behave as one might expect. What about their half up/down siblings?
BigDecimal.new(1.551, 10).round(1, :half_up).to_f
=> 1.6
BigDecimal.new(1.551, 10).round(1, :half_down).to_f
=> 1.6
Hmm, not quite what I was expecting. But what if we change the number we’re using?
BigDecimal.new(1.55, 10).round(1, :half_up).to_f
=> 1.6
BigDecimal.new(1.55, 10).round(1, :half_down).to_f
=> 1.5
That’s more like it. But why is a rounding method called “half down” rounding 1.55 down, but 1.551 up? To find out, we can check out the actual source code for the #round method.
The part we care about is down around line 5200 (I just searched “half_down” to find it):
...
/* now fracf = does any positive digit exist under the rounding position?
now fracf_1further = does any positive digit exist under one further than the
rounding position?
now v = the first digit under the rounding position */
/* drop digits after pointed digit */
memset(y->frac + ix + 1, 0, (y->Prec - (ix + 1)) * sizeof(BDIGIT));
switch (f) {
case VP_ROUND_DOWN: /* Truncate */
break;
case VP_ROUND_UP: /* Roundup */
if (fracf) ++div;
break;
case VP_ROUND_HALF_UP:
if (v>=5) ++div;
break;
case VP_ROUND_HALF_DOWN:
if (v > 5 || (v == 5 && fracf_1further)) ++div;
break;
case VP_ROUND_CEIL:
if (fracf && BIGDECIMAL_POSITIVE_P(y)) ++div;
break;
case VP_ROUND_FLOOR:
if (fracf && BIGDECIMAL_NEGATIVE_P(y)) ++div;
break;
case VP_ROUND_HALF_EVEN: /* Banker's rounding */
...
If you’re not used to C this might be a bit jarring (in fact, even if you are used to C the Ruby codebase has its own style).
We have some comments at the top to help us understand what the variables represent. And we can ignore the memset line (the comment above indicates what it’s doing).
The meat of what we want to look at is in the switch-case statement. From the VP_ROUND_DOWN and VP_ROUND_UP cases we can deduce that doing nothing results in truncation, while ++div results in rounding-up.
In these simpler cases the rules are pretty clear - :down will always truncate no matter what. :up will round-up if there is a positive digit under the rounding position (i.e. not 0).
Armed with this knowledge, we can take a look at the :half_up and :half_down variants.
In the :half_up case we see that it rounds up if v (the digit one spot right of where we are rounding to) is >= 5. Our ‘rounding position’ is the first decimal place, so v is the second decimal digit. Indeed this matches what we see:
# BigDecimal.new(1.55, 10).round(1, :half_up).to_f
=> 1.6
# BigDecimal.new(1.54, 10).round(1, :half_up).to_f
=> 1.5
If the second digit is >= 5 then :half_up will round it up, otherwise it will truncate.
So what about :half_down?
We can see that if the digit is > 5 then it behaves the same as :half_up, but there is a special case if the digit is 5. If the digit is 5 then we will only round up if fracf_1further is true. This is true whenever there are any positive digits after the rounding position. Hopefully the behaviour we’ve seen is now a bit clearer:
# BigDecimal.new(1.56, 10).round(1, :half_down).to_f
=> 1.6
# BigDecimal.new(1.55, 10).round(1, :half_down).to_f
=> 1.5
# BigDecimal.new(1.55000001, 10).round(1, :half_down).to_f
=> 1.6
Our “rounding position” is 1, so the digit we look at is the second after the decimal place. When it is greater than 5 we round up (same as :half_up). When it’s 5 or less we truncate. But if it is 5 and there’s another digit further down (no matter how far down), then we round up.
If the way you thought about rounding was single-digit focused as I described at the start, then this behaviour is unexpected. For this :half_down rounding you have to think of the number, not just a series of digits. “Half down” means it will round down half (0.5) or less. 0.500001 is greater than 0.5, so it gets rounded up. Once I thought of it this way the behaviour made sense.
Summary
This was a fairly trivial example, but the general idea of jumping into the source code can save a lot of headaches, particularly if documentation is lacking. This is one of the many benefits of open source software, which we get to enjoy both as Ruby and Rails developers. And don’t be put off by being unfamiliar with the source language - I don’t understand most of the BigDecimal C code (or the Ruby codebase at all for that matter), but a keyword search and a little deduction may be all you need.