473,287 Members | 1,926 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

Reading structures from a binary file

I'm trying to read some binary data from a file created by another
program. I know the binary file format but can't change or control the
format. The binary data is organised such that it should populate a
series of structures of specified variable composition.

I have the structures created OK, but actually reading the files is
giving me an error. Can I ask a simple question to start with:

I'm trying to read the file using the deserialize method. Since the
original file wasn't created with serialize am I wasting my time
trying to do this?

If I can use deserialize in this situation then I'll perservere with
looking for the error, but first I'd like to check that the approach
is valid in principle.

TIA
JGD
Nov 20 '05 #1
7 6005
* John Dann <ne**@prodata.co.uk> scripsit:
I'm trying to read some binary data from a file created by another
program. I know the binary file format but can't change or control the
format. The binary data is organised such that it should populate a
series of structures of specified variable composition.

I have the structures created OK, but actually reading the files is
giving me an error. Can I ask a simple question to start with:


Try to read the file using 'FileGet'.

--
Herfried K. Wagner [MVP]
<URL:http://dotnet.mvps.org/>
Nov 20 '05 #2
John,
In addition to the other comments.

Is the other program a .NET program or VB6 or something else?

If its VB6 or something else, I would consider using BinaryReader &
BinaryWriter to do my own binary serialization of the structures.

The following article offers using a BinaryReader & BinaryWriter to
serializing a class to a NetworkStream, you can use a FileStream and have
the same effect.

http://msdn.microsoft.com/library/de...rp09182003.asp

The example is in C#, however it should be easily converted to VB.NET, post
if you need help.

Hope this helps
Jay

"John Dann" <ne**@prodata.co.uk> wrote in message
news:03********************************@4ax.com...
I'm trying to read some binary data from a file created by another
program. I know the binary file format but can't change or control the
format. The binary data is organised such that it should populate a
series of structures of specified variable composition.

I have the structures created OK, but actually reading the files is
giving me an error. Can I ask a simple question to start with:

I'm trying to read the file using the deserialize method. Since the
original file wasn't created with serialize am I wasting my time
trying to do this?

If I can use deserialize in this situation then I'll perservere with
looking for the error, but first I'd like to check that the approach
is valid in principle.

TIA
JGD

Nov 20 '05 #3
On 24 Jun 2004 17:48:28 +0200, hi***************@gmx.at (Herfried K.
Wagner [MVP]) wrote:
Try to read the file using 'FileGet'.


I take it that this means that Deserialize won't work in this
situation, which I can understand if Serialize adds some extra
information to the binary file beyond its basic data content to help
in reading back via Deserialize. So for my erudition is this actually
the case?

One other comment: I think I used to use FileGetfor this purpose in a
previous VB6 incarnation of a related program. I was thinking I ought
to get more used to the (?) preferred .net way of doing things hence
why I was looking at deserialize and then failing that I was going to
try a binaryreader approach. But you seem to be implying that the
particular situation I'm tackling (ie reading structures from a
pre-existing binary file) is not best handled by one of the newer .net
input methods and FileGet is as good as anything despite being, well,
say older technology. Have I read this correctly?

JGD
Nov 20 '05 #4
John,
What do you mean by "Serialize" and "Deserialize".

If you used Binary or XML Serialization to Serialize the file, then you
should use Binary or XML Serializaiton to Deserialize the file respectively.

Note that its best if the classes that are being serialized & deserialized
as When you serialize an object to a file, one of the items serialized is
the assembly doing the serialization, when you deserialize the object, the
system checks the current assembly version against the version in the
serialized object. If these versions are different you see the error.

I would recommend you use a single class library that does the actual
serialization & deserialization, ensuring the assembly version is constant.

One of the following articles on serialization discusses how you can change
the version info read from an assembly, which is used more for version 2
program being able to read version 1 versions of the serialized data, not so
program 1 can read program 2 data per se (read, as I would recommend a
single assembly used by both programs).

http://msdn.microsoft.com/msdnmag/issues/02/04/net/
http://msdn.microsoft.com/msdnmag/issues/02/07/net/
http://msdn.microsoft.com/msdnmag/issues/02/09/net/

Hope this helps
Jay
"John Dann" <ne**@prodata.co.uk> wrote in message
news:4d********************************@4ax.com...
On 24 Jun 2004 17:48:28 +0200, hi***************@gmx.at (Herfried K.
Wagner [MVP]) wrote:
Try to read the file using 'FileGet'.


I take it that this means that Deserialize won't work in this
situation, which I can understand if Serialize adds some extra
information to the binary file beyond its basic data content to help
in reading back via Deserialize. So for my erudition is this actually
the case?

One other comment: I think I used to use FileGetfor this purpose in a
previous VB6 incarnation of a related program. I was thinking I ought
to get more used to the (?) preferred .net way of doing things hence
why I was looking at deserialize and then failing that I was going to
try a binaryreader approach. But you seem to be implying that the
particular situation I'm tackling (ie reading structures from a
pre-existing binary file) is not best handled by one of the newer .net
input methods and FileGet is as good as anything despite being, well,
say older technology. Have I read this correctly?

JGD

Nov 20 '05 #5
On 24 Jun 2004 17:48:28 +0200, hi***************@gmx.at (Herfried K.
Wagner [MVP]) wrote:

Try to read the file using 'FileGet'.


Actually, thinking about this, isn't FileGet just for random access
files, ie where the file consists of repeated records _all_ of the
same size. Whereas the situation I have, as per the original post, is
one where the records are not all of the same size. (In fact, they are
all of the same size except that the first record/structure is header
data and is of different size to the others. But I think this one
exception is enough to break the general approach of using FileGet.)

JGD
Nov 20 '05 #6
Jay, many thanks for your responses:
Is the other program a .NET program or VB6 or something else?
The other program is a proprietary program written I think in C. The
only information I have about it, beyond user documentation, is the
format of the binary file, which is described in terms of C
structures. I have previously read (an older version of) this file
successfully using VB6, so wasn't anticipating any problem with
VB.net[1].

However, I wasn't sure which of the newer .net IO routines to use.
Looking at a couple of books seemed to suggest that deserialize was a
good and efficient approach to reading structures from a binary file.
But what was never really clear was whether the binary file needed to
be created by serialize before it could be read by deserialize or
whether deserialize could be used with _any_ binary file of known
format, whatever its provenance. AFAICS the answer to the latter idea
now seems to be a definite no. In other words, serialize seems to add
additional information to the binary file beyond the actual binary
structures, information which is needed before the file can be read by
deserialize.

Maybe this is obvious to anyone who's worked with de/serialize before,
but it wasn't a point brought out in any of the books I had to hand.

Yes I'll certainly try BinaryReader as a next step, though I'm
slightly concerned that performance for reading thousands of records
may be an issue. But we'll see.

Thanks again.
JGD

[1] Actually there is one potential issue in a newer version of the
file format in that one parameter is saved as an unsigned short. I can
see that there's a vb.net method to read unsigned shorts from a
binary file, but I'm not quite sure how to handle it thereafter. But
that's perhaps a subject for a new thread if necessary.

If its VB6 or something else, I would consider using BinaryReader &
BinaryWriter to do my own binary serialization of the structures.

The following article offers using a BinaryReader & BinaryWriter to
serializing a class to a NetworkStream, you can use a FileStream and have
the same effect.

http://msdn.microsoft.com/library/de...rp09182003.asp

The example is in C#, however it should be easily converted to VB.NET, post
if you need help.

Hope this helps
Jay

"John Dann" <ne**@prodata.co.uk> wrote in message
news:03********************************@4ax.com.. .
I'm trying to read some binary data from a file created by another
program. I know the binary file format but can't change or control the
format. The binary data is organised such that it should populate a
series of structures of specified variable composition.

I have the structures created OK, but actually reading the files is
giving me an error. Can I ask a simple question to start with:

I'm trying to read the file using the deserialize method. Since the
original file wasn't created with serialize am I wasting my time
trying to do this?

If I can use deserialize in this situation then I'll perservere with
looking for the error, but first I'd like to check that the approach
is valid in principle.

TIA
JGD


Nov 20 '05 #7
John,
The other program is a proprietary program written I think in C. The
only information I have about it, beyond user documentation, is the
format of the binary file, which is described in terms of C
structures. I have previously read (an older version of) this file
successfully using VB6, so wasn't anticipating any problem with
VB.net[1]. I would use either a BinaryReader or a Stream & System.BitConverter. Seeing
as I would expect the BinaryReader is implemented in terms of
System.BitConverter or something lower I would use the BinaryReader and not
bother reimplementing the logic... Especially where my reimplementation may
not be as efficient as the BinaryReader...

I would follow the pattern identified in the article I posted earlier.
Yes I'll certainly try BinaryReader as a next step, though I'm
slightly concerned that performance for reading thousands of records
may be an issue. But we'll see. If the BinaryReader was a performance problem, then I would use a different
language (like C or C++ itself). As the BinaryReader is as "low level" as
you are going to get, short of using Win32 API directly.

In other words I would not expect the BinaryReader itself to be the
performance problem per se, but I could see mis-using the data returned from
the BinaryReader may be a performance problem.
[1] Actually there is one potential issue in a newer version of the
file format in that one parameter is saved as an unsigned short. I can
see that there's a vb.net method to read unsigned shorts from a
binary file, but I'm not quite sure how to handle it thereafter. But
that's perhaps a subject for a new thread if necessary. VB.NET does not really support unsigned Ints, I would read them as Signed
Ints, unless there is a chance they would be greater the Integer.MaxValue.
You can use BinaryReader to read the unsigned integer, however VB.NET does
not yet really allow you to do anything with it. No operators allowed, only
calling methods that return or accept an unsigned int as a parameter...

The one caveat to watch for with the BinaryReader is how strings are
encoded. BinaryReader.GetString expects length prefixed strings in the
System.Text.Encoding specified on the BinaryReader's constructor. The number
of bytes used for the length is variable length. If you have fixed length
strings or null terminated strings in the file, I would suggest a helper
function reading the string into a byte array, then using
System.Text.Encoding.GetString to convert this byte array into a String.

Hope this helps
Jay

"John Dann" <ne**@prodata.co.uk> wrote in message
news:u8********************************@4ax.com... Jay, many thanks for your responses:
Is the other program a .NET program or VB6 or something else?


The other program is a proprietary program written I think in C. The
only information I have about it, beyond user documentation, is the
format of the binary file, which is described in terms of C
structures. I have previously read (an older version of) this file
successfully using VB6, so wasn't anticipating any problem with
VB.net[1].

However, I wasn't sure which of the newer .net IO routines to use.
Looking at a couple of books seemed to suggest that deserialize was a
good and efficient approach to reading structures from a binary file.
But what was never really clear was whether the binary file needed to
be created by serialize before it could be read by deserialize or
whether deserialize could be used with _any_ binary file of known
format, whatever its provenance. AFAICS the answer to the latter idea
now seems to be a definite no. In other words, serialize seems to add
additional information to the binary file beyond the actual binary
structures, information which is needed before the file can be read by
deserialize.

Maybe this is obvious to anyone who's worked with de/serialize before,
but it wasn't a point brought out in any of the books I had to hand.

Yes I'll certainly try BinaryReader as a next step, though I'm
slightly concerned that performance for reading thousands of records
may be an issue. But we'll see.

Thanks again.
JGD

[1] Actually there is one potential issue in a newer version of the
file format in that one parameter is saved as an unsigned short. I can
see that there's a vb.net method to read unsigned shorts from a
binary file, but I'm not quite sure how to handle it thereafter. But
that's perhaps a subject for a new thread if necessary.

If its VB6 or something else, I would consider using BinaryReader &
BinaryWriter to do my own binary serialization of the structures.

The following article offers using a BinaryReader & BinaryWriter to
serializing a class to a NetworkStream, you can use a FileStream and have
the same effect.


http://msdn.microsoft.com/library/de...-s/dncscol/htm

l/csharp09182003.asp

The example is in C#, however it should be easily converted to VB.NET, postif you need help.

Hope this helps
Jay

"John Dann" <ne**@prodata.co.uk> wrote in message
news:03********************************@4ax.com.. .
I'm trying to read some binary data from a file created by another
program. I know the binary file format but can't change or control the
format. The binary data is organised such that it should populate a
series of structures of specified variable composition.

I have the structures created OK, but actually reading the files is
giving me an error. Can I ask a simple question to start with:

I'm trying to read the file using the deserialize method. Since the
original file wasn't created with serialize am I wasting my time
trying to do this?

If I can use deserialize in this situation then I'll perservere with
looking for the error, but first I'd like to check that the approach
is valid in principle.

TIA
JGD

Nov 20 '05 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Matt Laver | last post by:
Hi, I have a binary file that I'm currently reading byte by byte using code similiar to: string FileName = @"c:\myFile.dat"; FileStream fs = new FileStream(FileName, FileMode.Open,...
1
by: Manjunath sp via DotNetMonster.com | last post by:
Hi, How to effectively write and read structures from binary files in .Net? Currently I am using functions like ReadInt32 and the likes to read data from binary file into each elements of a...
3
by: Usenet User | last post by:
I am trying to read (and then save) a binary file which has certain data structures in it. (The file is in propritetary format produced by a 3rd party MFC application.) I know that those data...
23
by: Graham F French | last post by:
Hello, I can read text files into my application, but I cannot read in msg files as they seem to be in a proprietry format. Is there anyway of converting it on the fly or is there an...
3
by: Zeke Zinzul | last post by:
Hi Guys & Geeks, What's the most elegant way of dealing with binary data and structures? Say I have this (which I actually do, a woo-hoo): struct Struct_IconHeader { byte width; byte...
6
by: arne.muller | last post by:
Hello, I've come across some problems reading strucutres from binary files. Basically I've some strutures typedef struct { int i; double x; int n; double *mz;
13
by: swetha | last post by:
HI Every1, I have a problem in reading a binary file. Actually i want a C program which reads in the data from a file which is in binary format and i want to update values in it. The file...
4
by: Mastastealth | last post by:
I'm trying to create a program to read a certain binary format. I have the format's spec which goes something like: First 6 bytes: String Next 4 bytes: 3 digit number and a blank byte --- Next...
3
by: =?Utf-8?B?UmF5IE1pdGNoZWxs?= | last post by:
Hello, I'm trying to write an array of structures named myStructArray to a binary file and later on read it back. Although I could complete the entire project in C in about 2 minutes, I...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.