473,385 Members | 1,720 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Manual binary serialization of a dataset is slow for me - why?

hs
Hi
I am serializing a dataset using a binary formatter as follows:

IFormatter formater = new BinaryFormatter();
formatter.Serialize(stream, ds); // ds=DataSet, stream=MemoryStream
....
DataSet ds2 = (DataSet)formatter2.Deserialize(stream2);

For the size of my DataSet, its taking 0.8 seconds to serialize and 2.3
seconds to deserialize.

After reading some previous archive messages, I decided to try some manual
binary serialization using a custom DataSet (myDataSet). I have slightly
improved on my deserialization time (now takes 0.9 seconds) but my
serialzation has gone mad (takes 10 seconds). All this time is being spent
after the myDataSet.GetObjectData method. What is hapenning after this
method?
any advice on how to improve this time would be appreciated.
thanks

Code
------
IFormatter formater = new BinaryFormatter();
formatter.Serialize(stream, myds); // myds=myDataSet, stream=MemoryStream
....
myDataSet myds2 = (myDataSet)formatter2.Deserialize(stream2);

myDataSet class
----------------------
[Serializable]
public class myDataSet : DataSet, ISerializable
{
public myDataSet() : base()
{}
public myDataSet(string name) : base(name)
{}
protected myDataSet(SerializationInfo sinfo, StreamingContext context)
{
// Extract from the serialization info
ArrayList tables = new ArrayList();
tables = (ArrayList) sinfo.GetValue("Tables", typeof(ArrayList));

// foreach table name in tables create a DataTable and add to table
collection
for(int i=0;i<tables.Count; i++)
{
string tableName = (string)tables[i];
DataTable dt = new DataTable(tableName);

ArrayList colNames = new ArrayList();
ArrayList colTypes = new ArrayList();
ArrayList dataRows = new ArrayList();
colNames = (ArrayList) sinfo.GetValue("ColNames_"+tableName,
typeof(ArrayList));
colTypes = (ArrayList) sinfo.GetValue("ColTypes_"+tableName,
typeof(ArrayList));
dataRows = (ArrayList) sinfo.GetValue("DataRows_"+tableName,
typeof(ArrayList));

// Add columns
for(int iCol=0; iCol<colNames.Count; iCol++)
{
DataColumn col = new DataColumn(colNames[iCol].ToString(),
Type.GetType(colTypes[iCol].ToString() ));
dt.Columns.Add(col);
}

// Add rows
for(int iRows=0; iRows<dataRows.Count; iRows++)
{
DataRow row = dt.NewRow();
row.ItemArray = (object[]) dataRows[iRows];
dt.Rows.Add(row);
}

this.AcceptChanges();

// Add table to Tables collection
this.Tables.Add(dt);
}

void ISerializable.GetObjectData(SerializationInfo sinfo,
StreamingContext context)
{
// Add arrays
ArrayList tables = new ArrayList();

foreach(DataTable dt in this.Tables)
{
ArrayList colNames = new ArrayList();
ArrayList colTypes = new ArrayList();
ArrayList dataRows = new ArrayList();
string tableName = dt.TableName ;

// Insert the table name into worker array
tables.Add(tableName);

// Insert column information (names and types) into worker arrays
for(int iCols=0; iCols<dt.Columns.Count; iCols++)
{
DataColumn Col = dt.Columns[iCols];
colNames.Add(Col.ColumnName);
colTypes.Add(Col.DataType.FullName);
}

for(int iRows=0; iRows<dt.Rows.Count; iRows++)
{
dataRows.Add(dt.Rows[iRows].ItemArray);
}
// Add arrays to the serialization info structure
sinfo.AddValue("ColNames_"+tableName, colNames);
sinfo.AddValue("ColTypes_"+tableName, colTypes);
sinfo.AddValue("DataRows_"+tableName, dataRows);
}
sinfo.AddValue("Tables", tables);
}
}
Nov 15 '05 #1
4 8681

Hi
I add about more than 10000 records in database and use your code to
serialize it.
I met the same problem as you.
My serialization time is 19 seconds and my deserialization time is 3
seconds.

I will do some research and give you some suggestion.

Best regards,
Jeffrey Tan
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

--------------------
| Reply-To: "hs" <hs@nospam.syn>
| From: "hs" <hs@nospam.syn>
| Subject: Manual binary serialization of a dataset is slow for me - why?
| Date: Wed, 13 Aug 2003 16:28:43 +0100
| Lines: 119
| X-Priority: 3
| X-MSMail-Priority: Normal
| X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
| X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
| Message-ID: <Ov**************@TK2MSFTNGP10.phx.gbl>
| Newsgroups: microsoft.public.dotnet.languages.csharp
| NNTP-Posting-Host: mailgate.synergy-logistics.co.uk 62.49.130.162
| Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTN GP10.phx.gbl
| Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.languages.csharp:176176
| X-Tomcat-NG: microsoft.public.dotnet.languages.csharp
|
| Hi
| I am serializing a dataset using a binary formatter as follows:
|
| IFormatter formater = new BinaryFormatter();
| formatter.Serialize(stream, ds); // ds=DataSet, stream=MemoryStream
| ...
| DataSet ds2 = (DataSet)formatter2.Deserialize(stream2);
|
| For the size of my DataSet, its taking 0.8 seconds to serialize and 2.3
| seconds to deserialize.
|
| After reading some previous archive messages, I decided to try some manual
| binary serialization using a custom DataSet (myDataSet). I have slightly
| improved on my deserialization time (now takes 0.9 seconds) but my
| serialzation has gone mad (takes 10 seconds). All this time is being spent
| after the myDataSet.GetObjectData method. What is hapenning after this
| method?
| any advice on how to improve this time would be appreciated.
| thanks
|
| Code
| ------
| IFormatter formater = new BinaryFormatter();
| formatter.Serialize(stream, myds); // myds=myDataSet, stream=MemoryStream
| ...
| myDataSet myds2 = (myDataSet)formatter2.Deserialize(stream2);
|
| myDataSet class
| ----------------------
| [Serializable]
| public class myDataSet : DataSet, ISerializable
| {
| public myDataSet() : base()
| {}
| public myDataSet(string name) : base(name)
| {}
| protected myDataSet(SerializationInfo sinfo, StreamingContext context)
| {
| // Extract from the serialization info
| ArrayList tables = new ArrayList();
| tables = (ArrayList) sinfo.GetValue("Tables", typeof(ArrayList));
|
| // foreach table name in tables create a DataTable and add to table
| collection
| for(int i=0;i<tables.Count; i++)
| {
| string tableName = (string)tables[i];
| DataTable dt = new DataTable(tableName);
|
| ArrayList colNames = new ArrayList();
| ArrayList colTypes = new ArrayList();
| ArrayList dataRows = new ArrayList();
| colNames = (ArrayList) sinfo.GetValue("ColNames_"+tableName,
| typeof(ArrayList));
| colTypes = (ArrayList) sinfo.GetValue("ColTypes_"+tableName,
| typeof(ArrayList));
| dataRows = (ArrayList) sinfo.GetValue("DataRows_"+tableName,
| typeof(ArrayList));
|
| // Add columns
| for(int iCol=0; iCol<colNames.Count; iCol++)
| {
| DataColumn col = new DataColumn(colNames[iCol].ToString(),
| Type.GetType(colTypes[iCol].ToString() ));
| dt.Columns.Add(col);
| }
|
| // Add rows
| for(int iRows=0; iRows<dataRows.Count; iRows++)
| {
| DataRow row = dt.NewRow();
| row.ItemArray = (object[]) dataRows[iRows];
| dt.Rows.Add(row);
| }
|
| this.AcceptChanges();
|
| // Add table to Tables collection
| this.Tables.Add(dt);
| }
|
| void ISerializable.GetObjectData(SerializationInfo sinfo,
| StreamingContext context)
| {
| // Add arrays
| ArrayList tables = new ArrayList();
|
| foreach(DataTable dt in this.Tables)
| {
| ArrayList colNames = new ArrayList();
| ArrayList colTypes = new ArrayList();
| ArrayList dataRows = new ArrayList();
| string tableName = dt.TableName ;
|
| // Insert the table name into worker array
| tables.Add(tableName);
|
| // Insert column information (names and types) into worker arrays
| for(int iCols=0; iCols<dt.Columns.Count; iCols++)
| {
| DataColumn Col = dt.Columns[iCols];
| colNames.Add(Col.ColumnName);
| colTypes.Add(Col.DataType.FullName);
| }
|
| for(int iRows=0; iRows<dt.Rows.Count; iRows++)
| {
| dataRows.Add(dt.Rows[iRows].ItemArray);
| }
| // Add arrays to the serialization info structure
| sinfo.AddValue("ColNames_"+tableName, colNames);
| sinfo.AddValue("ColTypes_"+tableName, colTypes);
| sinfo.AddValue("DataRows_"+tableName, dataRows);
| }
| sinfo.AddValue("Tables", tables);
| }
| }
|
|
|

Nov 15 '05 #2
You may want to change the formatter type to XsdString, like below:. This
will make your serialization/deserialization much faster.

BinaryFormatter fmtobj = new BinaryFormatter();
IFormatter formatter = (IFormatter)fmtobj;
FormatterTypeStyle fts = fmtobj.TypeFormat;
fmtobj.TypeFormat = fts | FormatterTypeStyle.XsdString;

Best regards,
Xiao Xie
Microsoft Developer Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

Nov 15 '05 #3
You may want to change the formatter type to XsdString, like below:. This
will make your serialization/deserialization much faster.

BinaryFormatter fmtobj = new BinaryFormatter();
IFormatter formatter = (IFormatter)fmtobj;
FormatterTypeStyle fts = fmtobj.TypeFormat;
fmtobj.TypeFormat = fts | FormatterTypeStyle.XsdString;

Best regards,
Xiao Xie
Microsoft Developer Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

Nov 15 '05 #4
hs
Xiao and Jeffrey,

Thank you for your help. Your suggestion has worked.

Nov 15 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: phyzics | last post by:
I am porting an application from C++ to C#, and am having trouble finding a way to quickly and efficiently write structures to a binary file. In C++ this is trivial because all that is necessary is...
11
by: ajou_king | last post by:
I was running some tests on my Win32 1GHZ processor to see how long it would take to transmit objects numerous times via TCP/IP using C# ..NET Remoting vs the C++ trustworthy method of binary...
7
by: schoenfeld1 | last post by:
I've implemented IPC between two applications using named pipes and binary serialization, but have noticed that the binary formatter is rather slow. It seems that the binary formatter reflects...
2
by: Wild Wind | last post by:
Hello, I have an object which has various properties, including one which is of type DataSet. I need to serialize this object in binary format, but I've discovered that the BinaryFormatter...
1
by: Matt | last post by:
I have a web service that currently returns a dataset. Depending on the data being returned its size will be in megabytes (XML Document could be possibly 100 or more megabytes). To speed up the...
3
by: stockblaster | last post by:
Hello all.. Is it possible to convert a DataTable (i create the DataTable from a CSV file) into binary data and save it into an sql 2005 table (into binary field). After that I want to...
0
by: Bilz | last post by:
All, I have a rather complex object graph that I can use serialization to serialize to XML or Binary. I find that deserialiation is very slow in either case, but was REALLY suprised that...
7
by: Mike9900 | last post by:
If you do .NET Remoting and Binary Serilization with DataSet it does not work. If you set the Serialization type to Binary on DataSet, it does not seralize it correctly on windows xp. To do this...
1
by: kikisan | last post by:
I am developing a windows service which utilizes the following classes: interface IPersistable; abstract class PersistableObject : IPersistable;
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.