473,324 Members | 2,254 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,324 software developers and data experts.

very confused....hex to decimal

hi all,

in the immediate window ( typing: ? Val("&H" & "4124EC7DE6666666") and in calc, if i
convert

4124EC7DE6666666 to decimal

then i get 4.6941367371074376E+18 (or 4694136737107437158).

i know ahead of time however that the decimal value should be 685630.95000000000. why is
it that i am getting the former number in VB and calc?

lance


Nov 23 '05 #1
7 1497
Lance,
Because "Hex" itself doesn't support decimal numbers per se, so 685630 would
be &H10BD3H

Are you thinking that 41 24 EC 7D E6 66 66 66 are the bytes for the Double
number 685630.95000000000?

If so, I would probably put the above bytes into a Byte array & use
BitConverter to convert the array of bytes to a Double.

Something like:

Dim bytes() As Byte = {&H41, &H24, &HEC, &H7D, &HE6, &H66, &H66,
&H66}
Array.Reverse(bytes) ' Get the bytes in the correct sequence
Dim d As Double = BitConverter.ToDouble(bytes, 0)

--
Hope this helps
Jay [MVP - Outlook]
..NET Application Architect, Enthusiast, & Evangelist
T.S. Bradley - http://www.tsbradley.net
"Lance" <nu***@business.com> wrote in message
news:uf**************@TK2MSFTNGP10.phx.gbl...
| hi all,
|
| in the immediate window ( typing: ? Val("&H" & "4124EC7DE6666666") and in
calc, if i
| convert
|
| 4124EC7DE6666666 to decimal
|
| then i get 4.6941367371074376E+18 (or 4694136737107437158).
|
| i know ahead of time however that the decimal value should be
685630.95000000000. why is
| it that i am getting the former number in VB and calc?
|
| lance
|
|
|
|
Nov 23 '05 #2
thank you!
"Jay B. Harlow [MVP - Outlook]" <Ja************@tsbradley.net> wrote in message
news:Oj**************@TK2MSFTNGP09.phx.gbl...
Lance,
Because "Hex" itself doesn't support decimal numbers per se, so 685630 would
be &H10BD3H

Are you thinking that 41 24 EC 7D E6 66 66 66 are the bytes for the Double
number 685630.95000000000?

If so, I would probably put the above bytes into a Byte array & use
BitConverter to convert the array of bytes to a Double.

Something like:

Dim bytes() As Byte = {&H41, &H24, &HEC, &H7D, &HE6, &H66, &H66,
&H66}
Array.Reverse(bytes) ' Get the bytes in the correct sequence
Dim d As Double = BitConverter.ToDouble(bytes, 0)

--
Hope this helps
Jay [MVP - Outlook]
.NET Application Architect, Enthusiast, & Evangelist
T.S. Bradley - http://www.tsbradley.net
"Lance" <nu***@business.com> wrote in message
news:uf**************@TK2MSFTNGP10.phx.gbl...
| hi all,
|
| in the immediate window ( typing: ? Val("&H" & "4124EC7DE6666666") and in
calc, if i
| convert
|
| 4124EC7DE6666666 to decimal
|
| then i get 4.6941367371074376E+18 (or 4694136737107437158).
|
| i know ahead of time however that the decimal value should be
685630.95000000000. why is
| it that i am getting the former number in VB and calc?
|
| lance
|
|
|
|

Nov 23 '05 #3
Hi,

4124EC7DE6666666
<<

Why do you know that this should be 685630.95000000000?

A Hex number can only be converted directly to an Integer, Long, etc. It
does not represent a non-integral value -- and is a MUCH larger number (as
the conversions that you observe indicate) than your "expected" result.

You must be attempting to convert the binary representation of a floating
point number to a floating point decimal representation -- This cannot be
done with the code that you are using, which, naturally, works only with
non-floating point (integral) values.

I think that if you Google, you will find code to convert binary FP numbers
to decimal FP. The "normal" reason for such conversion is that you are
receiving a binary data stream from some foreign device, and have to convert
it for use in your program. If this isn't so, then a more detailed
explanation is required.

Dick

Dick

--
Richard Grier (Microsoft Visual Basic MVP)

See www.hardandsoftware.net for contact information.

Author of Visual Basic Programmer's Guide to Serial Communications, 4th
Edition ISBN 1-890422-28-2 (391 pages) published July 2004. See
www.mabry.com/vbpgser4 to order.
Nov 23 '05 #4
you nailed it on all accounts, pretty much.

by the way, the reason i know it should be 685630.9500000000 is that i can use a few other
programs (that i didn't write, <g>) that correctly extract the information. it's a UTM x
coordinate. i'm using the other programs to confirm my results.

lance

"Dick Grier" <di**************@msn.com> wrote in message
news:Ot**************@TK2MSFTNGP12.phx.gbl...
Hi,

4124EC7DE6666666
<<

Why do you know that this should be 685630.95000000000?

A Hex number can only be converted directly to an Integer, Long, etc. It does not
represent a non-integral value -- and is a MUCH larger number (as the conversions that
you observe indicate) than your "expected" result.

You must be attempting to convert the binary representation of a floating point number
to a floating point decimal representation -- This cannot be done with the code that you
are using, which, naturally, works only with non-floating point (integral) values.

I think that if you Google, you will find code to convert binary FP numbers to decimal
FP. The "normal" reason for such conversion is that you are receiving a binary data
stream from some foreign device, and have to convert it for use in your program. If
this isn't so, then a more detailed explanation is required.

Dick

Dick

--
Richard Grier (Microsoft Visual Basic MVP)

See www.hardandsoftware.net for contact information.

Author of Visual Basic Programmer's Guide to Serial Communications, 4th Edition ISBN
1-890422-28-2 (391 pages) published July 2004. See www.mabry.com/vbpgser4 to order.

Nov 23 '05 #5
Out of curiosity, what is your source for this binary value?
Any particular reason you are working with the bytes themselves?

Gerald

"Lance" <nu***@business.com> wrote in message
news:OY**************@TK2MSFTNGP12.phx.gbl...
you nailed it on all accounts, pretty much.

by the way, the reason i know it should be 685630.9500000000 is that i can use a few other programs (that i didn't write, <g>) that correctly extract the information. it's a UTM x coordinate. i'm using the other programs to confirm my results.

lance

"Dick Grier" <di**************@msn.com> wrote in message
news:Ot**************@TK2MSFTNGP12.phx.gbl...
Hi,

4124EC7DE6666666
<<

Why do you know that this should be 685630.95000000000?

A Hex number can only be converted directly to an Integer, Long, etc. It does not represent a non-integral value -- and is a MUCH larger number (as the conversions that you observe indicate) than your "expected" result.

You must be attempting to convert the binary representation of a floating point number to a floating point decimal representation -- This cannot be done with the code that you are using, which, naturally, works only with non-floating point (integral) values.
I think that if you Google, you will find code to convert binary FP numbers to decimal FP. The "normal" reason for such conversion is that you are receiving a binary data stream from some foreign device, and have to convert it for use in your program. If this isn't so, then a more detailed explanation is required.

Dick

Dick

--
Richard Grier (Microsoft Visual Basic MVP)

See www.hardandsoftware.net for contact information.

Author of Visual Basic Programmer's Guide to Serial Communications, 4th Edition ISBN 1-890422-28-2 (391 pages) published July 2004. See www.mabry.com/vbpgser4 to order.


Nov 23 '05 #6
it's a tif file from a filestream. i'm checking for and detecting the values of GeoTIFF
tags.

lance

"Gerald Hernandez" <Cablewizard@sp*********@Yahoo.com> wrote in message
news:em*************@TK2MSFTNGP15.phx.gbl...
Out of curiosity, what is your source for this binary value?
Any particular reason you are working with the bytes themselves?

Gerald

"Lance" <nu***@business.com> wrote in message
news:OY**************@TK2MSFTNGP12.phx.gbl...
you nailed it on all accounts, pretty much.

by the way, the reason i know it should be 685630.9500000000 is that i can

use a few other
programs (that i didn't write, <g>) that correctly extract the

information. it's a UTM x
coordinate. i'm using the other programs to confirm my results.

lance

"Dick Grier" <di**************@msn.com> wrote in message
news:Ot**************@TK2MSFTNGP12.phx.gbl...
> Hi,
>
>>>
> 4124EC7DE6666666
> <<
>
> Why do you know that this should be 685630.95000000000?
>
> A Hex number can only be converted directly to an Integer, Long, etc. It does not > represent a non-integral value -- and is a MUCH larger number (as the conversions that > you observe indicate) than your "expected" result.
>
> You must be attempting to convert the binary representation of a floating point number > to a floating point decimal representation -- This cannot be done with the code that you > are using, which, naturally, works only with non-floating point (integral) values. >
> I think that if you Google, you will find code to convert binary FP numbers to decimal > FP. The "normal" reason for such conversion is that you are receiving a binary data > stream from some foreign device, and have to convert it for use in your program. If > this isn't so, then a more detailed explanation is required.
>
> Dick
>
> Dick
>
> --
> Richard Grier (Microsoft Visual Basic MVP)
>
> See www.hardandsoftware.net for contact information.
>
> Author of Visual Basic Programmer's Guide to Serial Communications, 4th Edition ISBN > 1-890422-28-2 (391 pages) published July 2004. See www.mabry.com/vbpgser4 to order. >
>



Nov 23 '05 #7
Ah, ok. I asked because I have been working on a bunch of things to iterop
with various CAD / GIS formats. Thought maybe I might have something already
to help you out. However, I have not had a need to work with GeoTIFF's
directly, well not yet. So sadly I do not have anything to help :-|

Gerald

"Lance" <nu***@business.com> wrote in message
news:uZ**************@TK2MSFTNGP12.phx.gbl...
it's a tif file from a filestream. i'm checking for and detecting the values of GeoTIFF tags.

lance

"Gerald Hernandez" <Cablewizard@sp*********@Yahoo.com> wrote in message
news:em*************@TK2MSFTNGP15.phx.gbl...
Out of curiosity, what is your source for this binary value?
Any particular reason you are working with the bytes themselves?

Gerald

"Lance" <nu***@business.com> wrote in message
news:OY**************@TK2MSFTNGP12.phx.gbl...
you nailed it on all accounts, pretty much.

by the way, the reason i know it should be 685630.9500000000 is that i
can use a few other
programs (that i didn't write, <g>) that correctly extract the

information. it's a UTM x
coordinate. i'm using the other programs to confirm my results.

lance

"Dick Grier" <di**************@msn.com> wrote in message
news:Ot**************@TK2MSFTNGP12.phx.gbl...
> Hi,
>
>>>
> 4124EC7DE6666666
> <<
>
> Why do you know that this should be 685630.95000000000?
>
> A Hex number can only be converted directly to an Integer, Long, etc.

It does not
> represent a non-integral value -- and is a MUCH larger number (as the

conversions that
> you observe indicate) than your "expected" result.
>
> You must be attempting to convert the binary representation of a

floating point number
> to a floating point decimal representation -- This cannot be done
with the code that you
> are using, which, naturally, works only with non-floating point

(integral) values.
>
> I think that if you Google, you will find code to convert binary FP

numbers to decimal
> FP. The "normal" reason for such conversion is that you are
receiving a binary data
> stream from some foreign device, and have to convert it for use in
your program. If
> this isn't so, then a more detailed explanation is required.
>
> Dick
>
> Dick
>
> --
> Richard Grier (Microsoft Visual Basic MVP)
>
> See www.hardandsoftware.net for contact information.
>
> Author of Visual Basic Programmer's Guide to Serial Communications,
4th Edition ISBN
> 1-890422-28-2 (391 pages) published July 2004. See

www.mabry.com/vbpgser4 to order.
>
>



Nov 23 '05 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

21
by: Batista, Facundo | last post by:
Here I send it. Suggestions and all kinds of recomendations are more than welcomed. If it all goes ok, it'll be a PEP when I finish writing/modifying the code. Thank you. .. Facundo
17
by: John Bentley | last post by:
John Bentley: INTRO The phrase "decimal number" within a programming context is ambiguous. It could refer to the decimal datatype or the related but separate concept of a generic decimal number....
6
by: Simon | last post by:
Hi all I am writing a small app that uses real numbers all over the place for calculations. But when it comes to displaying values it is better to only display an it, (especially when it...
4
by: postings | last post by:
Hi I have a decimal numbers showing up on my datagrid that is displaying lots of trailing zeros in the listbox. i.e. 1.000100 or 1.000000 I want to get rid of all the trailing zeros (I...
9
by: CsharpGuy | last post by:
Ok, here is my scenario, I have an asp web app that i'm converting to .net. The app reads files and loads the data into SQL db, now, in my file it has numbers like, 125.25, 3363.33, 69.00, and when...
3
by: Evan | last post by:
So I have two versions of a program (these are complete programs, not excerpts): Version 1: template <class T > void foo() { return bar( T() ); }
8
by: manmit.walia | last post by:
Hello Everyone, Long time ago, I posted a small problem I had about converting a VB6 program to C#. Well with the help with everyone I got it converted. But I overlooked something and don't...
13
by: VijaKhara | last post by:
Hi all In my codes, I have something a=b+(1/3); (a and b are two double variables) The result always is that (1/3)=0 and I got the wrong answer a=b. I have to replace it by a=b+0.333 to get...
2
by: IuliaS | last post by:
Hello everyone! I want to create a stored procedure, so I can more easily, and transparent retrieve data from db2. Long story short: when a user wants to put some data in the DB, he also creates...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.