floating point

boaz · Joined: 04 Oct 2007 Posts: 259

Hi,

If I do 1/2 and get 0.4999999, is it casued by hardware or is it by the OS?

--
> There is no answer.
> There has not been an answer.
> There will not be an answer.
> That IS the answer!
> And I am screwed.
> Deadline was due yesterday.
>
> There is no point to life.
> THAT IS THE POINT.
> And we are screwed.
> We will run out of oil soon.

http://spaces.msn.com/bzDaCat

Archived from group: microsoft>public>vb>enterprise

Dmitriy Antonov · Joined: 04 Oct 2007 Posts: 431

"boaz" wrote in message %23GHA.3312@TK2MSFTNGP02.phx.gbl...
> Hi,
>
> If I do 1/2 and get 0.4999999, is it casued by hardware or is it by the
> OS?
>
>
> --
>> There is no answer.
>> There has not been an answer.
>> There will not be an answer.
>> That IS the answer!
>> And I am screwed.
>> Deadline was due yesterday.
>>
>> There is no point to life.
>> THAT IS THE POINT.
>> And we are screwed.
>> We will run out of oil soon.
>
> http://spaces.msn.com/bzDaCat
>
I can't reproduce it - tried with double, single and variant on two
computers - always 0.5. How do you get such result?

Dmitriy.

"Rick Rothstein \ · Joined: 04 Oct 2007 Posts: 1584

> If I do 1/2 and get 0.4999999, is it casued by hardware or is it by the
> OS?

If you do 1/2 where? I don't think that number will give you 0.4999999
unless it comes about from a floating point calculation. Did you perhaps
notice a problem with one of your divisions and attempt to "simplify" the
question for us? Anyway, there is a representational problem when dealing
with floating point number; and no, it is not caused by the hardware or
OS... the problem is most floating point numbers in a decimal system do not
have **exact** representations in the underlying binary number system that
computers use. These two links should give you a better insight into the
problem...

INFO: Visual Basic and Arithmetic Precision
http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/Q279/7/55.ASP&NoWebContent=1

(Complete) Tutorial to Understand IEEE Floating-Point Errors
http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/Q42/9/80.ASP&NoWebContent=1

Rick

boaz · Joined: 04 Oct 2007 Posts: 259

It is just an example. I don't get 0.4999999 with 1/2.

My boss was trying to substract one column from another column in ACCESS and
he didn't get what he was looking for. So, I told him to round the totals.

anyway... what you are saying is that it is a software problem; not a
hardware problem.

Bob Butler · Joined: 04 Oct 2007 Posts: 1325

"boaz" wrote in message$3k7pI%23GHA.1196@TK2MSFTNGP02.phx.gbl
> It is just an example. I don't get 0.4999999 with 1/2.
>
> My boss was trying to substract one column from another column in
> ACCESS and he didn't get what he was looking for. So, I told him to
> round the totals.
>
> anyway... what you are saying is that it is a software problem; not a
> hardware problem.

it's probably more accurate to call it an "expectation problem"

--
Reply to the group so all can participate
VB.Net: "Fool me once..."

Schmidt · Joined: 04 Oct 2007 Posts: 806

"Bob Butler" schrieb im Newsbeitrag%23GHA.4524@TK2MSFTNGP04.phx.gbl...

> > anyway... what you are saying is that it is a software problem; not a
> > hardware problem.
>
> it's probably more accurate to call it an "expectation problem"

Smile

Olaf

Bob O`Bob · Joined: 04 Oct 2007 Posts: 1456

Bob Butler wrote:
> "boaz" wrote in message
> $3k7pI%23GHA.1196@TK2MSFTNGP02.phx.gbl
>> It is just an example. I don't get 0.4999999 with 1/2.
>>
>> My boss was trying to substract one column from another column in
>> ACCESS and he didn't get what he was looking for. So, I told him to
>> round the totals.
>>
>> anyway... what you are saying is that it is a software problem; not a
>> hardware problem.
>
> it's probably more accurate to call it an "expectation problem"
>

Yeah, but that covers about 85% of all that goes on in these newsgroups...

boaz · Joined: 04 Oct 2007 Posts: 259

"Bob O`Bob" wrote in message %23GHA.1224@TK2MSFTNGP04.phx.gbl...
> Bob Butler wrote:
>> "boaz" wrote in message
>> $3k7pI%23GHA.1196@TK2MSFTNGP02.phx.gbl
>>> It is just an example. I don't get 0.4999999 with 1/2.
>>>
>>> My boss was trying to substract one column from another column in
>>> ACCESS and he didn't get what he was looking for. So, I told him to
>>> round the totals.
>>> anyway... what you are saying is that it is a software problem; not a
>>> hardware problem.
>>
>> it's probably more accurate to call it an "expectation problem"
>>
>
>
> Yeah, but that covers about 85% of all that goes on in these newsgroups...

This is strange to me. I have around 3000 clients. About 5-10 of them have
the floating points problem. So, if this is the implementation (software),
all of them should have the same problem. How come only 5-10 of them have
this problem?

Ralph · Joined: 04 Oct 2007 Posts: 4148

"boaz" wrote in message%23GHA.4268@TK2MSFTNGP02.phx.gbl...
>
> "Bob O`Bob" wrote in message
> %23GHA.1224@TK2MSFTNGP04.phx.gbl...
> > Bob Butler wrote:
> >> "boaz" wrote in message
> >> $3k7pI%23GHA.1196@TK2MSFTNGP02.phx.gbl
> >>> It is just an example. I don't get 0.4999999 with 1/2.
> >>>
> >>> My boss was trying to substract one column from another column in
> >>> ACCESS and he didn't get what he was looking for. So, I told him to
> >>> round the totals.
> >>> anyway... what you are saying is that it is a software problem; not a
> >>> hardware problem.
> >>
> >> it's probably more accurate to call it an "expectation problem"
> >>
> >
> >
> > Yeah, but that covers about 85% of all that goes on in these
newsgroups...
>
>
> This is strange to me. I have around 3000 clients. About 5-10 of them
have
> the floating points problem. So, if this is the implementation
(software),
> all of them should have the same problem. How come only 5-10 of them have
> this problem?
>

The exact same datatypes, using the exact same data, in the exact same
sequence of massage, using the exact same display format?

-ralph

boaz · Joined: 04 Oct 2007 Posts: 259

"Ralph" wrote in message @arkansas.net...
>
> "boaz" wrote in message
> %23GHA.4268@TK2MSFTNGP02.phx.gbl...
>>
>> "Bob O`Bob" wrote in message
>> %23GHA.1224@TK2MSFTNGP04.phx.gbl...
>> > Bob Butler wrote:
>> >> "boaz" wrote in message
>> >> $3k7pI%23GHA.1196@TK2MSFTNGP02.phx.gbl
>> >>> It is just an example. I don't get 0.4999999 with 1/2.
>> >>>
>> >>> My boss was trying to substract one column from another column in
>> >>> ACCESS and he didn't get what he was looking for. So, I told him to
>> >>> round the totals.
>> >>> anyway... what you are saying is that it is a software problem; not a
>> >>> hardware problem.
>> >>
>> >> it's probably more accurate to call it an "expectation problem"
>> >>
>> >
>> >
>> > Yeah, but that covers about 85% of all that goes on in these
> newsgroups...
>>
>>
>> This is strange to me. I have around 3000 clients. About 5-10 of them
> have
>> the floating points problem. So, if this is the implementation
> (software),
>> all of them should have the same problem. How come only 5-10 of them
>> have
>> this problem?
>>
>
> The exact same datatypes, using the exact same data, in the exact same
> sequence of massage, using the exact same display format?
>
> -ralph
>
>
>

same data type and formula but of course the numbers are different.

Ralph · Joined: 04 Oct 2007 Posts: 4148

"boaz" wrote in message
news:%23AxfPlT%23GHA.4740@TK2MSFTNGP03.phx.gbl...
>
> "Ralph" wrote in message
> @arkansas.net...
> >
> >>
> >
> > The exact same datatypes, using the exact same data, in the exact same
> > sequence of massage, using the exact same display format?
> >
> > -ralph
> >
>
> same data type and formula but of course the numbers are different.
>

There you have it.

Whenever you work with floating-point numbers, at some point you have to
determine a working 'scale', and occasionally trim yourself back. Or you
will get bit.

In the real world the 'scale' seldom has to be anywhere near the actual
ability of a PC to generate them. We went to the moon and back, and toured
the planets, with pi never calculated beyond 4 decimal places.

-ralph

boaz · Joined: 04 Oct 2007 Posts: 259

.... and yes... they are all in English... tons... pounds... etc...
hahaha!!!

"Ralph" wrote in message @arkansas.net...
>
> "boaz" wrote in message
> news:%23AxfPlT%23GHA.4740@TK2MSFTNGP03.phx.gbl...
>>
>> "Ralph" wrote in message
>> @arkansas.net...
>> >
>> >>
>> >
>> > The exact same datatypes, using the exact same data, in the exact same
>> > sequence of massage, using the exact same display format?
>> >
>> > -ralph
>> >
>>
>> same data type and formula but of course the numbers are different.
>>
>
> There you have it.
>
> Whenever you work with floating-point numbers, at some point you have to
> determine a working 'scale', and occasionally trim yourself back. Or you
> will get bit.
>
> In the real world the 'scale' seldom has to be anywhere near the actual
> ability of a PC to generate them. We went to the moon and back, and toured
> the planets, with pi never calculated beyond 4 decimal places.
>
> -ralph
>
>

Michael C · Joined: 04 Oct 2007 Posts: 1831

"Dmitriy Antonov" wrote in message
news:%23ErUbdI%
> I can't reproduce it - tried with double, single and variant on two
> computers - always 0.5. How do you get such result?

0.5 is 0.1 binary so can be stored exactly in a single or double.

Michael

Michael C · Joined: 04 Oct 2007 Posts: 1831

"boaz" wrote in message $3k7pI%23GHA.1196@TK2MSFTNGP02.phx.gbl...
> It is just an example. I don't get 0.4999999 with 1/2.
>
> My boss was trying to substract one column from another column in ACCESS
> and he didn't get what he was looking for. So, I told him to round the
> totals.
>
> anyway... what you are saying is that it is a software problem; not a
> hardware problem.

It's a hardware limitation most likely, not a hardware problem. A simplified
way to look at it is the floating point processor will store numbers as
being made up of 0.5, 0.25, 0.125 etc so cannot store every decimal value
with 100% accuracy. I believe even something as simple 0.1 cannot be stored
exactly. You need to take this into account in a financial app.

>
>

expvb · Joined: 04 Oct 2007 Posts: 525

Try using the Round() function to eliminate these problems.