By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
457,877 Members | 1,109 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 457,877 IT Pros & Developers. It's quick & easy.

Why is serialization so SLOW?

P: n/a
We have an app that computes a lot of numeric data. We would like to save to
disk about 1-2 gigabytes of computed data that includes ints, doubles,
strings and some complex objects that contain hashtables. We would like to
read this data back into the app and reuse those values and state to compute
more new data.

Up to this point we just write out comma separated ASCII values for
everything. And we read/parse the ascii back in. The ASCII input/output code
isn't very optimized, but it works. We would like to gain greater reuse and
increased performance by writing out data in binary format.

We did some testing and careful design, and implemented what we thought was
good quality binary output code. We used binary formatters with
serialization. The code is more carefully designed and tested than our
previous ASCII output code. However, at best, it is 14 times slower! It is
shockingly slow compared to something that we didn't think was that fast to
start with.

We've seen a lot of people posting similar experiences with .NET
serialization being slow. We also use it in our app for saving various
settings and enabling undo/redo. In those circumstances, it is also very
slow.

It seem slowest when dealing with complex objects. In our testing with just
doubles and strings, serialization in binary format was faster than ASCII
with parsing. But in the real life app with real data, serialization just
doesn't cut it.

Any tips or suggestions are appreciated. Is there another approach to
persisting complex program states to disk?
Nov 15 '05 #1
Share this Question
Share on Google+
1 Reply


P: n/a
Hello

Serialization can store much overhead data, like data types, member field
names, etc.

Try using System.IO.BinaryReader and System.IO.BinaryWriter classes,
although it is much harder than serialization, but it will outperform it.
You will have to implement saving a hashtable in binary format

The way you store you information is also important, with 1-2 GB of info,
you must have a way to index the data, so that you can get the data you need
without searching the whole file.

Also, have you considered storing the data in a database server, like SQL
server?

Best Regards

Sherif

"Mountain Bikn' Guy" <vc@attbi.com> wrote in message
news:Z4*******************@rwcrnsc52.ops.asp.att.n et...
We have an app that computes a lot of numeric data. We would like to save to disk about 1-2 gigabytes of computed data that includes ints, doubles,
strings and some complex objects that contain hashtables. We would like to
read this data back into the app and reuse those values and state to compute more new data.

Up to this point we just write out comma separated ASCII values for
everything. And we read/parse the ascii back in. The ASCII input/output code isn't very optimized, but it works. We would like to gain greater reuse and increased performance by writing out data in binary format.

We did some testing and careful design, and implemented what we thought was good quality binary output code. We used binary formatters with
serialization. The code is more carefully designed and tested than our
previous ASCII output code. However, at best, it is 14 times slower! It is shockingly slow compared to something that we didn't think was that fast to start with.

We've seen a lot of people posting similar experiences with .NET
serialization being slow. We also use it in our app for saving various
settings and enabling undo/redo. In those circumstances, it is also very
slow.

It seem slowest when dealing with complex objects. In our testing with just doubles and strings, serialization in binary format was faster than ASCII
with parsing. But in the real life app with real data, serialization just
doesn't cut it.

Any tips or suggestions are appreciated. Is there another approach to
persisting complex program states to disk?

Nov 15 '05 #2

This discussion thread is closed

Replies have been disabled for this discussion.