 |
|
|
|
| Author |
Message |
footyplayer
Joined: 04 Oct 2007 Posts: 1
|
Posted: Tue Jun 13, 2006 5:13 pm Post subject: Problem with Dividing Number in VB6 |
|
|
Hi All,
I am currently experiencing the following issue:
If you take the value of 2275 and divide it by 11, (in the Windows
calculator), you are presented with an answer of 206.81818181...
However, if you divide the same value in VB, you are presented with 206.
I need to know how to get the proper value, up to two places after the
decimal. Then, I need to properly round the value - I need to get this right
as this number is used as a rate in figuring out dollar amounts for payroll
processing. I've checked MSDN help, as well as Google, but couldn't find any
help. I'm currently using Single as my data type in my variable that stores
the result. Any help/examples/paths to understanding would be greatly
appreciated!
Thanks in Advance,
Brad
Archived from group: microsoft>public>vb>syntax |
|
| Back to top |
|
 |
Karl E. Peterson
Joined: 04 Oct 2007 Posts: 4836
|
Posted: Tue Jun 13, 2006 5:52 pm Post subject: Re: Problem with Dividing Number in VB6 |
|
|
footyplayer wrote:
> Hi All,
>
> I am currently experiencing the following issue:
>
> If you take the value of 2275 and divide it by 11, (in the Windows
> calculator), you are presented with an answer of 206.81818181...
>
> However, if you divide the same value in VB, you are presented with
> 206.
Sounds like you're doing integer division. In the Immediate window:
?2275/11
206.818181818182
?2275\11
206
> I need to know how to get the proper value, up to two places after the
> decimal. Then, I need to properly round the value - I need to get
> this right as this number is used as a rate in figuring out dollar
> amounts for payroll processing. I've checked MSDN help, as well as
> Google, but couldn't find any help. I'm currently using Single as my
> data type in my variable that stores the result. Any
> help/examples/paths to understanding would be greatly appreciated!
?format(2275/11, "0.00")
206.82
--
Working without a .NET?
http://classicvb.org/ |
|
| Back to top |
|
 |
Rick Rothstein
Joined: 04 Oct 2007 Posts: 1589
|
Posted: Tue Jun 13, 2006 8:59 pm Post subject: Re: Problem with Dividing Number in VB6 |
|
|
> I am currently experiencing the following issue:
>
> If you take the value of 2275 and divide it by 11, (in the Windows
> calculator), you are presented with an answer of 206.81818181...
>
> However, if you divide the same value in VB, you are presented with 206.
>
> I need to know how to get the proper value, up to two places after the
> decimal. Then, I need to properly round the value - I need to get this
> right
> as this number is used as a rate in figuring out dollar amounts for
> payroll
> processing. I've checked MSDN help, as well as Google, but couldn't find
> any
> help. I'm currently using Single as my data type in my variable that
> stores
> the result. Any help/examples/paths to understanding would be greatly
> appreciated!
When you did your division, did you use a backward slash (\) or a forward
slash (/)? It kind of looks like you may have used the backward slash which
performs integer division; if you use a forward slash (normal division), you
should get the proper answer. So that you know, integer division seems to
have this equivalency with normal division...
A \ B = Int(Round(A) / Round(B))
where the Int function rounds decimal values downward to the nearest whole
number and Round implements "Banker's Rounding" (numbers ending in 5 when
rounded to the previous digit position, are rounded to the even digit; e.g.,
2.125 becomes 2.12 whereas 1.135 becomes 1.14) and not what I think you are
referring to as "proper" rounding (the Format function can be used to
produce "proper" rounding; e.g., Format(2.125, "0.00") returns 2.13 as I
think you might expect).
Rick |
|
| Back to top |
|
 |
footyplayer
Joined: 04 Oct 2007 Posts: 3
|
Posted: Tue Jun 13, 2006 10:03 pm Post subject: Re: Problem with Dividing Number in VB6 |
|
|
Karl and Rick,
Many thanks for explaining the issue to me - you guys were exactly right, I
was using integer division. Much appreciated!
Best Regards,
Brad
"footyplayer" wrote in message @microsoft.com...
> Hi All,
>
> I am currently experiencing the following issue:
>
> If you take the value of 2275 and divide it by 11, (in the Windows
> calculator), you are presented with an answer of 206.81818181...
>
> However, if you divide the same value in VB, you are presented with 206.
>
> I need to know how to get the proper value, up to two places after the
> decimal. Then, I need to properly round the value - I need to get this
> right
> as this number is used as a rate in figuring out dollar amounts for
> payroll
> processing. I've checked MSDN help, as well as Google, but couldn't find
> any
> help. I'm currently using Single as my data type in my variable that
> stores
> the result. Any help/examples/paths to understanding would be greatly
> appreciated!
>
> Thanks in Advance,
> Brad
|
|
| Back to top |
|
 |
|
|
| Related Topics: | plus number in V.Basic 6 Hi, I want to write a code in V.Basic 6 in order to plus every Month (+1) automatically when the Month change, Field's name is "Total", and is txt. For example: This Month(Aug) there is the number 5 in the Field (Total), Sept I want it changed to 6 auto,
Rounding Number How would I round a currency value to the nearest 0.25? I've looked at the ROUND function, but the documentation says it only rounds to a specified number of decimal places. TIA Dan
max number of arguments Is there a maximum number of arguments a Sub can have? When i try to code a Sub with more than 59 arguments, I get an error message - "Too many arguments". Is there any way to get around this? Cheers
format general Number I am looking for a way to format a previously calculated number that is stored as currency in a list. I need to parse out the comma delimeter. Any help is appreciated.
Large number to HEX conversion Hi all. How can I convert this number: 2227808324 To hex? (84C9A844) Using Hex$ in VB6 returns an error TIA Guy |
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|