Hi,
I currently writing a serialize/unserialize architecture. The read/write
function will read/write from a binary file.
My question is is there some sort on defined standart to use when
representing data type (int , int32, int64, double, string, etc....) ?
Thanks, 11 3111
Charles T. wrote: I currently writing a serialize/unserialize architecture. The read/write function will read/write from a binary file.
Why a binary file?
My question is is there some sort on defined standart to use when representing data type (int , int32, int64, double, string, etc....) ?
No. There is not even a "standard" for what order bytes go inside an int.
The least heinous data format is XML. You can write very simple or very
complex data structures in it, and you can read those structures in a text
editor.
But XML can be a little obese. Some data formats are compressed XML.
--
Phlip http://www.xpsd.org/cgi-bin/wiki?Tes...UserInterfaces
[snip] The least heinous data format is XML. You can write very simple or very complex data structures in it, and you can read those structures in a text editor.
But XML can be a little obese. Some data formats are compressed XML.
I would reccomend this, also.
The game Age of Mythology uses XML compressed with zLib compatible
compression, and it generates very compact but easily decoded files.
You can get zLib here: http://www.zlib.org
- Pete
Charles T. wrote: Hi,
I currently writing a serialize/unserialize architecture. The read/write function will read/write from a binary file.
There has been much discussion on Serialization and Persistence in
this newsgroup and news:comp.lang.c. Use a search engine and look
for some ideas.
My question is is there some sort on defined standart to use when representing data type (int , int32, int64, double, string, etc....) ?
There is no standard, from platform to platform. On some platforms,
there may be no standards between OS versions or compiler versions.
For better portability, write out the data in a consistent form
(i.e. uint64 == 64 bits, little endian) and let the programs convert
the data into the native representation.
Remember, when serializing, that the size of a structure may not
be the sum of the size of its members. Compilers are allowed to
add "padding bytes" between members.
Pointers don't store well. There is a very small probability
that an OS will allocate a variable in the same place for each
execution of a program.
Since pointers don't store well, don't store strings as pointers.
Store text as <quantity, text> or <text, sentinel character>.
See section [35] of the C++ FAQ (about serialization): http://www.parashift.com/c++-faq-lit...alization.html
Thanks,
--
Thomas Matthews
C++ newsgroup welcome message: http://www.slack.net/~shiva/welcome.txt
C++ Faq: http://www.parashift.com/c++-faq-lite
C Faq: http://www.eskimo.com/~scs/c-faq/top.html
alt.comp.lang.learn.c-c++ faq: http://www.raos.demon.uk/acllc-c++/faq.html
Other sites: http://www.josuttis.com -- C++ STL Library book
Phlip wrote: Charles T. wrote:
I currently writing a serialize/unserialize architecture. The read/write function will read/write from a binary file.
Why a binary file?
My question is is there some sort on defined standart to use when representing data type (int , int32, int64, double, string, etc....) ?
No. There is not even a "standard" for what order bytes go inside an int.
The least heinous data format is XML. You can write very simple or very complex data structures in it, and you can read those structures in a text editor.
But XML can be a little obese. Some data formats are compressed XML.
If you're talking about a real-time (streaming) system, the XML overhead
may be too much of a price to pay.
In 1999 I built a binary XML format that could be "parsed" in a fraction
of the time. But for some systems, even this one was too expensive.
Gianni Mariani wrote:
[snip] In 1999 I built a binary XML format that could be "parsed" in a fraction of the time. But for some systems, even this one was too expensive.
Would you mind posting your implementation? I would be interested in seeing
it.
Thanks!
- Pete
You might want to look (depending on your application area and on
whether you have time to learn it) at ASN.1, which is an ITU standard to
provide "a notation for defining data structures [and] a defined
(machine-independent) encoding for those data structures".
Have a glance at www-sop.inria.fr/rodeo/personnel/hoschka/asn1.html, www.asn1.org, or google will bring back lots of links.
Geoff Macartney
Charles T. wrote: Hi,
I currently writing a serialize/unserialize architecture. The read/write function will read/write from a binary file.
My question is is there some sort on defined standart to use when representing data type (int , int32, int64, double, string, etc....) ?
Thanks,
On Wed, 04 Feb 2004 12:21:16 -0500, Gianni Mariani wrote: In 1999 I built a binary XML format that could be "parsed" in a fraction of the time. But for some systems, even this one was too expensive.
No need to reinvent the wheel, have a look at ASN.1. Parsers abundand BTW.
M4
Martijn Lievaart wrote: On Wed, 04 Feb 2004 12:21:16 -0500, Gianni Mariani wrote:
In 1999 I built a binary XML format that could be "parsed" in a fraction of the time. But for some systems, even this one was too expensive.
No need to reinvent the wheel, have a look at ASN.1. Parsers abundand BTW.
ASN.1 is different - the binary format I'm talking about has a 1:1
correlation to XML. The format was simply more efficient to parse than
XML text - admitedly the XML parser I wrote was slower than molasses in
a blizzard ... :-)
Charles T. wrote: Hi,
I currently writing a serialize/unserialize architecture. The read/write function will read/write from a binary file.
My question is is there some sort on defined standart to use when representing data type (int , int32, int64, double, string, etc....) ?
I have an application in which the compactness of binary representation
(as compared with, say, XML) is important, but where portability of that
binary file, regardless of endianess, is also important. My solution is
very simple: I just choose an endianess and stick with it, and make sure
to write/read one byte at a time to construct/reconstruct the data. It
works fine. The binary file is as compact as if I didn't care about
portability, and it works with all kinds of endianess. The reading and
the writing in principle takes a little longer because of the
disassembling/assembling that takes place here, but in practice it is
not a problem at all because of buffering. I just read, say, 1k at a
time and the problem disappears. Also, there are usually layers of
buffering involved anyway, in the OS, in the disk etc.
/David
On Wed, 04 Feb 2004 19:50:36 -0500, Gianni Mariani wrote: Martijn Lievaart wrote: On Wed, 04 Feb 2004 12:21:16 -0500, Gianni Mariani wrote:
In 1999 I built a binary XML format that could be "parsed" in a fraction of the time. But for some systems, even this one was too expensive.
No need to reinvent the wheel, have a look at ASN.1. Parsers abundand BTW.
ASN.1 is different - the binary format I'm talking about has a 1:1 correlation to XML. The format was simply more efficient to parse than XML text - admitedly the XML parser I wrote was slower than molasses in a blizzard ... :-)
I think ASN.1 can easily handle binary-XML. Something like it's been a
while since I worked with ASN.1, so terminlogy is likely to be incorrect):
list xmlentitydef
list xmltagdef
utf8 xmltag
list xmlattrdef
utf8 attrkey
utf8 attrval
list xmlattr
utf8 attrkey
utf8 attrval
utf8 entitybody
Entity body could itself be a list with entities. Xmltag and xmlattrdef
could probably use binary tags if there are only a few possible tags, thus
saving greatly on space (and processing time).
I don't think you can get very much more efficient than that.
M4
Thank, for the response,
i will take a look at the asn1 stuff
"Geoff Macartney" <ge********************@openwavedotcom.nospam> wrote in
message news:E9**********************@newsfep2-win.server.ntli.net... You might want to look (depending on your application area and on whether you have time to learn it) at ASN.1, which is an ITU standard to provide "a notation for defining data structures [and] a defined (machine-independent) encoding for those data structures".
Have a glance at www-sop.inria.fr/rodeo/personnel/hoschka/asn1.html, www.asn1.org, or google will bring back lots of links.
Geoff Macartney
Charles T. wrote:
Hi,
I currently writing a serialize/unserialize architecture. The read/write function will read/write from a binary file.
My question is is there some sort on defined standart to use when representing data type (int , int32, int64, double, string, etc....) ?
Thanks, This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: J. Campbell |
last post by:
OK...I'm in the process of learning C++. In my old (non-portable)
programming days, I made use of binary files a lot...not worrying
about endian issues. I'm starting to understand why C++ makes...
|
by: geskerrett |
last post by:
In the '80's, Microsoft had a proprietary binary structure to handle
floating point numbers, In a previous thread, Bengt Richter posted
some example code in how to convert these to python floats;...
|
by: Tanuki |
last post by:
Hi All:
I encounter a programming problem recently. I need to read a binary
file. I need to translate the binary data into useful information. I
have the format at hand, like 1st byte = ID,...
|
by: Yeow |
last post by:
hello,
i was trying to use the fread function on SunOS and ran into some
trouble.
i made a simple test as follows:
i'm trying to read in a binary file (generated from a fortran code)
that...
|
by: alice |
last post by:
hi all,
Can anybody please tell the advantages which the binary files offers
over the character files.
Thanks,
Alice walls
|
by: PengYu.UT |
last post by:
Hi,
I write the content of a in file "data" (in Sun Machine). Then I read
"data" in both SunOS and linux. But the result is different. Do you
know how to make it binary data portable.
Best...
|
by: vim |
last post by:
hello everybody
Plz tell the differance between binary file and ascii
file...............
Thanks
in advance
vim
|
by: Default User |
last post by:
I work in software research and development and we're going to be doing
some investigations into message traffic. This is for embedded systems.
What we're looking at right now is XML encoded...
|
by: smith4894 |
last post by:
Hello all,
I'm working on writing my own streambuf classes (to use in my custom
ostream/isteam classes that will handle reading/writing data to a
mmap'd file).
When reading from the mmap...
|
by: elliotng.ee |
last post by:
I have a text file that contains a header 32-bit binary. For example,
the text file could be:
%%This is the input text
%%test.txt
Date: Tue Dec 26 14:03:35 2006...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
| |