Problem reading a bin file

[efrenba]

Hi,

I came from delphi world and now I'm doing my first steps in C++. I'm
using C++builder because its ide is like delphi although I'm trying to
avoid the vcl.

I need to insert new features to an old program that I wrote in delphi
and it's a good opportunity to start with c++.
well, the problem is:

I need to read a binary file according to some structure, all
variables of its structure are of type "char" of different sizes.

The reading is made ok, but the value that I got of ManVer is his
value concatenated with Pieces and Weight and WeightUnit, the value of
Pieces is his value and also concatenated with Weight and WeightUnit.
This happens with the rest of variables until the last of them, that
only has his value.

Only when I see the structure with Evaluate/Modify (Ctrl +F7 on
windows) the result is the one that I expect to get

I hope that you can help me...
here's the code:
--------------

ifstream _fileStream;
TStructure_V250_Data _buffer250;

//open the file...
char* _fName = new char[_fileName.Length() +1];
strcpy(_fName, _fileName.c_str());

_fileStream.open(_fName, ios::binary);
delete [] _fName;
//read data...
_fileStream.read((char*)&_buffer250, sizeof(_buffer250));

---------------------------
_buffer250 structure is too long, i only show you a piece
---------------------------
typedef struct _Structure_V250_Data
{
char ManVer[4];
char Pieces[3];
char Weight[5];
char WeightUnit[2];
//...
//...
//...
} TStructure_V250_Data;

[Scott McPhillips [MVP]]

"efrenba" <ef*****@gmail.comwrote in message
news:d2**********************************@d21g2000 prf.googlegroups.com...

Hi,

I came from delphi world and now I'm doing my first steps in C++. I'm
using C++builder because its ide is like delphi although I'm trying to
avoid the vcl.

I need to insert new features to an old program that I wrote in delphi
and it's a good opportunity to start with c++.
well, the problem is:

I need to read a binary file according to some structure, all
variables of its structure are of type "char" of different sizes.

The reading is made ok, but the value that I got of ManVer is his
value concatenated with Pieces and Weight and WeightUnit, the value of
Pieces is his value and also concatenated with Weight and WeightUnit.
This happens with the rest of variables until the last of them, that
only has his value.

Only when I see the structure with Evaluate/Modify (Ctrl +F7 on
windows) the result is the one that I expect to get

I hope that you can help me...
here's the code:
--------------

ifstream _fileStream;
TStructure_V250_Data _buffer250;

//open the file...
char* _fName = new char[_fileName.Length() +1];
strcpy(_fName, _fileName.c_str());

_fileStream.open(_fName, ios::binary);
delete [] _fName;
//read data...
_fileStream.read((char*)&_buffer250, sizeof(_buffer250));

---------------------------
_buffer250 structure is too long, i only show you a piece
---------------------------
typedef struct _Structure_V250_Data
{
char ManVer[4];
char Pieces[3];
char Weight[5];
char WeightUnit[2];
//...
//...
//...
} TStructure_V250_Data;

Whatever you are using to display the first string is expecting a nul
terminated string. If you don't have a nul at the end of ManVer your
display function keeps going through memory until it finds a nul, no matter
how far it has to go. (This often results in an access violation.) So it
seems the program is working but you are not interpreting or displaying the
results correctly.

Do not attempt to display char strings that do not have a nul termination.

--
Scott McPhillips [VC++ MVP]

[Salt_Peter]

On Dec 22, 11:21 pm, efrenba <[e-mail]:

Hi,

I came from delphi world and now I'm doing my first steps in C++. I'm
using C++builder because its ide is like delphi although I'm trying to
avoid the vcl.

I need to insert new features to an old program that I wrote in delphi
and it's a good opportunity to start with c++.

well, the problem is:

I need to read a binary file according to some structure, all
variables of its structure are of type "char" of different sizes.

The reading is made ok, but the value that I got of ManVer is his
value concatenated with Pieces and Weight and WeightUnit, the value of
Pieces is his value and also concatenated with Weight and WeightUnit.
This happens with the rest of variables until the last of them, that
only has his value.

Only when I see the structure with Evaluate/Modify (Ctrl +F7 on
windows) the result is the one that I expect to get

I hope that you can help me...

here's the code:
--------------

ifstream _fileStream;
TStructure_V250_Data _buffer250;

//open the file...
char* _fName = new char[_fileName.Length() +1];
strcpy(_fName, _fileName.c_str());

_fileStream.open(_fName, ios::binary);
delete [] _fName;

//read data...
_fileStream.read((char*)&_buffer250, sizeof(_buffer250));

---------------------------
_buffer250 structure is too long, i only show you a piece
---------------------------
typedef struct _Structure_V250_Data
{
char ManVer[4];
char Pieces[3];
char Weight[5];
char WeightUnit[2];
//...
//...
//...

} TStructure_V250_Data;

Have you considered trying a different angle? Instead of storing data
in binary, why not store records with members seperated by spaces. If
you were to organize your class members with std::strings, integers,
doubles or whatever, then you could use operator<< and operator>to
stream your data in/out. you would use std::ifstream and std::getline
to extract your records, one line at a time, into a
std::vector<SomeTypeand use the operators to have the class stream
in your data (and therefore construct instances of that class).

The problem from our point of view is we just don't know what ManVer
represents. Is it a player, a person? Instead of having everything be
a char, why not deal with primitive types and strings as what they
really are instead of characters? Why not deal with a type instead of
a binary soup?

You'll probably say "i'ld rather not redesign the whole system from
scratch" but then thats the problem, isn't it? You have code that
isn't easily modifyable, let alone extendeable, without a considerable
amount of work. You don't have to live that life. There is a better
way.

Give us an idea of what the struct is meant to represent. What is a
ManVer?

[efrenba]

On Dec 23, 2:17 am, Salt_Peter <[e-mail]:

On Dec 22, 11:21 pm, efrenba <[e-mail]:

Hi,

I came from delphi world and now I'm doing my first steps in C++. I'm
using C++builder because its ide is like delphi although I'm trying to
avoid the vcl.

I need to insert new features to an old program that I wrote in delphi
and it's a good opportunity to start with c++.

well, the problem is:

I need to read a binary file according to some structure, all
variables of its structure are of type "char" of different sizes.

The reading is made ok, but the value that I got of ManVer is his
value concatenated with Pieces and Weight and WeightUnit, the value of
Pieces is his value and also concatenated with Weight and WeightUnit.
This happens with the rest of variables until the last of them, that
only has his value.

Only when I see the structure with Evaluate/Modify (Ctrl +F7 on
windows) the result is the one that I expect to get

I hope that you can help me...

here's the code:
--------------

ifstream _fileStream;
TStructure_V250_Data _buffer250;

//open the file...
char* _fName = new char[_fileName.Length() +1];
strcpy(_fName, _fileName.c_str());

_fileStream.open(_fName, ios::binary);
delete [] _fName;

//read data...
_fileStream.read((char*)&_buffer250, sizeof(_buffer250));

---------------------------
_buffer250 structure is too long, i only show you a piece
---------------------------
typedef struct _Structure_V250_Data
{
char ManVer[4];
char Pieces[3];
char Weight[5];
char WeightUnit[2];
//...
//...
//...

} TStructure_V250_Data;

Have you considered trying a different angle? Instead of storing data
in binary, why not store records with members seperated by spaces. If
you were to organize your class members with std::strings, integers,
doubles or whatever, then you could use operator<< and operator>to
stream your data in/out. you would use std::ifstream and std::getline
to extract your records, one line at a time, into a
std::vector<SomeTypeand use the operators to have the class stream
in your data (and therefore construct instances of that class).

The problem from our point of view is we just don't know what ManVer
represents. Is it a player, a person? Instead of having everything be
a char, why not deal with primitive types and strings as what they
really are instead of characters? Why not deal with a type instead of
a binary soup?

You'll probably say "i'ld rather not redesign the whole system from
scratch" but then thats the problem, isn't it? You have code that
isn't easily modifyable, let alone extendeable, without a considerable
amount of work. You don't have to live that life. There is a better
way.

Give us an idea of what the struct is meant to represent. What is a
ManVer?

Hi,

Unfortunately I cann't change the file structure because it belongs to
a wide system around the world. This system runs over unix os and has
several years online.

The reason that I wrote this application for the first time in delphi
is because we needed an interface that was capable to read those files
and to generate more friendly reports to analyze several situations at
work.

Extracted from a pdf sent by the company:
--------------------------------------------
variables file offset type values
-------------------------------------------------------
char ManVer[4]; //1-4 4AN 250U or 260U (data
struct version)
char Filler1[3]; //5-7 3AN SYS Reserved
char Pieces[3]; //8-10 3N Qty
char Weight[5]; //11-15 5.2D Some value
char WeightUnit[2]; //16-17 2A KG or LB, etc
char Orig_ProcHub[3]; //18-20 3A Origin
char FlightDest[3]; //21-23 3A Destiny
char PrintFlag; //24 1AN SYS Reserved
....
....
until 71 variables
A = Alphabetic, left-justified, blank-filled
AN = Alphanumeric, left justified, blank-filled
D = Decimal, right-justified, zero-filled (for example, 4.1D denotes
one implied decimal in a 4-digit string)
N = Numeric, right justified, zero-filled
I don't understand, why when the reading is made, the nul char is not
set and the end of each variables?

Also I have other doubt, if I change the size of each variables to set
the nul character at the end, the reading won't be made ok.
I need help... Thanks for your time...

[Jim Langston]

efrenba wrote:

On Dec 23, 2:17 am, Salt_Peter <[e-mail]:

>On Dec 22, 11:21 pm, efrenba <[e-mail]:

>>Hi,

>>I came from delphi world and now I'm doing my first steps in C++.
I'm using C++builder because its ide is like delphi although I'm
trying to avoid the vcl.

>>I need to insert new features to an old program that I wrote in
delphi and it's a good opportunity to start with c++.

>>well, the problem is:

>>I need to read a binary file according to some structure, all
variables of its structure are of type "char" of different sizes.

>>The reading is made ok, but the value that I got of ManVer is his
value concatenated with Pieces and Weight and WeightUnit, the value
of Pieces is his value and also concatenated with Weight and
WeightUnit. This happens with the rest of variables until the last
of them, that only has his value.

>>Only when I see the structure with Evaluate/Modify (Ctrl +F7 on
windows) the result is the one that I expect to get

>>I hope that you can help me...

>>here's the code:
--------------

>>ifstream _fileStream;
TStructure_V250_Data _buffer250;

>>//open the file...
char* _fName = new char[_fileName.Length() +1];
strcpy(_fName, _fileName.c_str());

>>_fileStream.open(_fName, ios::binary);
delete [] _fName;

>>//read data...
_fileStream.read((char*)&_buffer250, sizeof(_buffer250));

>>---------------------------
_buffer250 structure is too long, i only show you a piece
---------------------------
typedef struct _Structure_V250_Data
{
char ManVer[4];
char Pieces[3];
char Weight[5];
char WeightUnit[2];
//...
//...
//...

>>} TStructure_V250_Data;

Have you considered trying a different angle? Instead of storing data
in binary, why not store records with members seperated by spaces. If
you were to organize your class members with std::strings, integers,
doubles or whatever, then you could use operator<< and operator>to
stream your data in/out. you would use std::ifstream and std::getline
to extract your records, one line at a time, into a
std::vector<SomeTypeand use the operators to have the class stream
in your data (and therefore construct instances of that class).

The problem from our point of view is we just don't know what ManVer
represents. Is it a player, a person? Instead of having everything be
a char, why not deal with primitive types and strings as what they
really are instead of characters? Why not deal with a type instead of
a binary soup?

You'll probably say "i'ld rather not redesign the whole system from
scratch" but then thats the problem, isn't it? You have code that
isn't easily modifyable, let alone extendeable, without a
considerable amount of work. You don't have to live that life. There
is a better way.

Give us an idea of what the struct is meant to represent. What is a
ManVer?

Hi,

Unfortunately I cann't change the file structure because it belongs to
a wide system around the world. This system runs over unix os and has
several years online.

The reason that I wrote this application for the first time in delphi
is because we needed an interface that was capable to read those files
and to generate more friendly reports to analyze several situations at
work.

Extracted from a pdf sent by the company:
--------------------------------------------
variables file offset type values
-------------------------------------------------------
char ManVer[4]; //1-4 4AN 250U or 260U (data
struct version)
char Filler1[3]; //5-7 3AN SYS Reserved
char Pieces[3]; //8-10 3N Qty
char Weight[5]; //11-15 5.2D Some value
char WeightUnit[2]; //16-17 2A KG or LB, etc
char Orig_ProcHub[3]; //18-20 3A Origin
char FlightDest[3]; //21-23 3A Destiny
char PrintFlag; //24 1AN SYS Reserved
...
...
until 71 variables
A = Alphabetic, left-justified, blank-filled
AN = Alphanumeric, left justified, blank-filled
D = Decimal, right-justified, zero-filled (for example, 4.1D denotes
one implied decimal in a 4-digit string)
N = Numeric, right justified, zero-filled
I don't understand, why when the reading is made, the nul char is not
set and the end of each variables?

Also I have other doubt, if I change the size of each variables to set
the nul character at the end, the reading won't be made ok.
I need help... Thanks for your time...

Okay, here's the issue. Lets take for example the first three values in the
file.

char ManVer[4]; //1-4 4AN 250U or 260U (data
struct version)
char Filler1[3]; //5-7 3AN SYS Reserved
char Pieces[3]; //8-10 3N Qty

Now, the actual data in the file may look something like this:

2AXXX023...

Reading this into your structure it should put the characters into the right
fields, that is.

ManVer should contain the characters ' ' ' ' '2' 'A'
You really need to add a filler field to take up the next three chars, but
then
Pieces would contain '0' '2' '3'.

Notice, howiever, ManVer does NOT contain " 2A" because that indicates a
null terminated string. The file spec says that the fields are not null
terminated. Meaning you can not treat these as any type of c-style or c++
style strings as they sit. You could look at each individual character to
see what they contain.. I.E. something like

for ( int size_t i = 0; i < sizeof ManVer; ++i )
std::cout << ManVer[i];
std::cout << "\n";

Now, you really can't put space for a null terminator in your structure and
read the file directly into your structure because there is no room in the
file. For you to do anything with the fields you'll need to convert them to
a c or c++ style string or a number. There's a few ways to do that, using
strncpy with c-style strings, adding chars or using a std::string
constructor with std::string (I *think* there is a std::string constructor
taking iterators to the first and last char to add, not sure).

You are doing fine now, as long as you have the fields in your structure
correctly mapped to the fields in the flat file. Make sure you put in space
for that filler.

But now your problem becomes what to do with the fields. This is up to you.
You can continue to treat them as pure characters without null terminators
throughout your program, or you can convert the fields you need as you need
them, or you could convert them all to proper c++ variables after you read
them.

Here is a little program I threw together showing one way they could be
converted. This is just one of many ways you could do it and the best way
is up for you to decide.

Output of the program is:

ManVer: 2a
Pieces: 2
Weight: 4.6

#include <string>
#include <iostream>
#include <sstream>

struct TStructure_V250_Data
{
char ManVer[4];
char Filler[3];
char Pieces[3];
char Weight[5];
char WeightUnit[2];
/* etc... */
};

void PretendtoRead( TStructure_V250_Data& Data )
{
std::strncpy( Data.ManVer, " 2a", 4 );
std::strncpy( Data.Filler, "xxx", 3 );
std::strncpy( Data.Pieces, "002", 3 );
std::strncpy( Data.Weight, "00460", 5 );
std::strncpy( Data.WeightUnit, "KG", 2 );
}

std::string CharsToString( const char* Input, int Length )
{
return std::string(Input, &Input[Length] );
}

int main()
{
TStructure_V250_Data Data;
PretendtoRead( Data );

std::cout << "ManVer: " << CharsToString( Data.ManVer, sizeof
Data.ManVer ) << "\n";

int Pieces;
std::stringstream Convert;
Convert << CharsToString( Data.Pieces, sizeof Data.Pieces );
Convert >Pieces;
std::cout << "Pieces: " << Pieces << "\n";

double Weight = 0.0f;
Convert.clear();
Convert << Data.Weight;
Convert >Weight;
Weight /= 100;
std::cout << "Weight: " << Weight << "\n";
}
--
Jim Langston
ta*******@rocketmail.com

[efrenba]

On Dec 23, 6:22 pm, "Jim Langston" <[e-mail]:

efrenba wrote:

On Dec 23, 2:17 am, Salt_Peter <[e-mail]:

On Dec 22, 11:21 pm, efrenba <[e-mail]:

>Hi,

>I came from delphi world and now I'm doing my first steps in C++.
I'm using C++builder because its ide is like delphi although I'm
trying to avoid the vcl.

>I need to insert new features to an old program that I wrote in
delphi and it's a good opportunity to start with c++.

>well, the problem is:

>I need to read a binary file according to some structure, all
variables of its structure are of type "char" of different sizes.

>The reading is made ok, but the value that I got of ManVer is his
value concatenated with Pieces and Weight and WeightUnit, the value
of Pieces is his value and also concatenated with Weight and
WeightUnit. This happens with the rest of variables until the last
of them, that only has his value.

>Only when I see the structure with Evaluate/Modify (Ctrl +F7 on
windows) the result is the one that I expect to get

>I hope that you can help me...

>here's the code:
--------------

>ifstream _fileStream;
TStructure_V250_Data _buffer250;

>//open the file...
char* _fName = new char[_fileName.Length() +1];
strcpy(_fName, _fileName.c_str());

>_fileStream.open(_fName, ios::binary);
delete [] _fName;

>//read data...
_fileStream.read((char*)&_buffer250, sizeof(_buffer250));

>---------------------------
_buffer250 structure is too long, i only show you a piece
---------------------------
typedef struct _Structure_V250_Data
{
char ManVer[4];
char Pieces[3];
char Weight[5];
char WeightUnit[2];
//...
//...
//...

>} TStructure_V250_Data;

Have you considered trying a different angle? Instead of storing data
in binary, why not store records with members seperated by spaces. If
you were to organize your class members with std::strings, integers,
doubles or whatever, then you could use operator<< and operator>to
stream your data in/out. you would use std::ifstream and std::getline
to extract your records, one line at a time, into a
std::vector<SomeTypeand use the operators to have the class stream
in your data (and therefore construct instances of that class).

The problem from our point of view is we just don't know what ManVer
represents. Is it a player, a person? Instead of having everything be
a char, why not deal with primitive types and strings as what they
really are instead of characters? Why not deal with a type instead of
a binary soup?

You'll probably say "i'ld rather not redesign the whole system from
scratch" but then thats the problem, isn't it? You have code that
isn't easily modifyable, let alone extendeable, without a
considerable amount of work. You don't have to live that life. There
is a better way.

Give us an idea of what the struct is meant to represent. What is a
ManVer?

Hi,

Unfortunately I cann't change the file structure because it belongs to
a wide system around the world. This system runs over unix os and has
several years online.

The reason that I wrote this application for the first time in delphi
is because we needed an interface that was capable to read those files
and to generate more friendly reports to analyze several situations at
work.

Extracted from a pdf sent by the company:
--------------------------------------------
variables file offset type values
-------------------------------------------------------
char ManVer[4]; //1-4 4AN 250U or 260U (data
struct version)
char Filler1[3]; //5-7 3AN SYS Reserved
char Pieces[3]; //8-10 3N Qty
char Weight[5]; //11-15 5.2D Some value
char WeightUnit[2]; //16-17 2A KG or LB, etc
char Orig_ProcHub[3]; //18-20 3A Origin
char FlightDest[3]; //21-23 3A Destiny
char PrintFlag; //24 1AN SYS Reserved
...
...
until 71 variables

A = Alphabetic, left-justified, blank-filled
AN = Alphanumeric, left justified, blank-filled
D = Decimal, right-justified, zero-filled (for example, 4.1D denotes
one implied decimal in a 4-digit string)
N = Numeric, right justified, zero-filled

I don't understand, why when the reading is made, the nul char is not
set and the end of each variables?

Also I have other doubt, if I change the size of each variables to set
the nul character at the end, the reading won't be made ok.

I need help... Thanks for your time...

Okay, here's the issue. Lets take for example the first three values in the
file.

char ManVer[4]; //1-4 4AN 250U or 260U (data
struct version)
char Filler1[3]; //5-7 3AN SYS Reserved
char Pieces[3]; //8-10 3N Qty

Now, the actual data in the file may look something like this:

2AXXX023...

Reading this into your structure it should put the characters into the right
fields, that is.

ManVer should contain the characters ' ' ' ' '2' 'A'
You really need to add a filler field to take up the next three chars, but
then
Pieces would contain '0' '2' '3'.

Notice, howiever, ManVer does NOT contain " 2A" because that indicates a
null terminated string. The file spec says that the fields are not null
terminated. Meaning you can not treat these as any type of c-style or c++
style strings as they sit. You could look at each individual character to
see what they contain.. I.E. something like

for ( int size_t i = 0; i < sizeof ManVer; ++i )
std::cout << ManVer[i];
std::cout << "\n";

Now, you really can't put space for a null terminator in your structure and
read the file directly into your structure because there is no room in the
file. For you to do anything with the fields you'll need to convert them to
a c or c++ style string or a number. There's a few ways to do that, using
strncpy with c-style strings, adding chars or using a std::string
constructor with std::string (I *think* there is a std::string constructor
taking iterators to the first and last char to add, not sure).

You are doing fine now, as long as you have the fields in your structure
correctly mapped to the fields in the flat file. Make sure you put in space
for that filler.

But now your problem becomes what to do with the fields. This is up to you.
You can continue to treat them as pure characters without null terminators
throughout your program, or you can convert the fields you need as you need
them, or you could convert them all to proper c++ variables after you read
them.

Here is a little program I threw together showing one way they could be
converted. This is just one of many ways you could do it and the best way
is up for you to decide.

Output of the program is:

ManVer: 2a
Pieces: 2
Weight: 4.6

#include <string>
#include <iostream>
#include <sstream>

struct TStructure_V250_Data
{
char ManVer[4];
char Filler[3];
char Pieces[3];
char Weight[5];
char WeightUnit[2];
/* etc... */

};

void PretendtoRead( TStructure_V250_Data& Data )
{
std::strncpy( Data.ManVer, " 2a", 4 );
std::strncpy( Data.Filler, "xxx", 3 );
std::strncpy( Data.Pieces, "002", 3 );
std::strncpy( Data.Weight, "00460", 5 );
std::strncpy( Data.WeightUnit, "KG", 2 );

}

std::string CharsToString( const char* Input, int Length )
{
return std::string(Input, &Input[Length] );

}

int main()
{
TStructure_V250_Data Data;
PretendtoRead( Data );

std::cout << "ManVer: " << CharsToString( Data.ManVer, sizeof
Data.ManVer ) << "\n";

int Pieces;
std::stringstream Convert;
Convert << CharsToString( Data.Pieces, sizeof Data.Pieces );
Convert >Pieces;
std::cout << "Pieces: " << Pieces << "\n";

double Weight = 0.0f;
Convert.clear();
Convert << Data.Weight;
Convert >Weight;
Weight /= 100;
std::cout << "Weight: " << Weight << "\n";

}

--
Jim Langston
[e-mail]

Thanks to all, for your time, I will continue studying C++, 'cause it
seems to be that this language is not easy...

[Old Wolf]

On Dec 23, 5:21 pm, efrenba <[e-mail]:

>
//open the file...
char* _fName = new char[_fileName.Length() +1];
strcpy(_fName, _fileName.c_str());

_fileStream.open(_fName, ios::binary);
delete [] _fName;

You can replace all of that with:

_fileStream.open( _fileName.c_str(), ios::binary );

Also, in C++ it is not good to start identifiers with
the underscore (those names could be reserved to the
implementation. Better is just 'fileName', or if you
must, 'fileName_' .