473,785 Members | 2,698 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Manual binary serialization of a dataset is slow for me - why?

hs
Hi
I am serializing a dataset using a binary formatter as follows:

IFormatter formater = new BinaryFormatter ();
formatter.Seria lize(stream, ds); // ds=DataSet, stream=MemorySt ream
....
DataSet ds2 = (DataSet)format ter2.Deserializ e(stream2);

For the size of my DataSet, its taking 0.8 seconds to serialize and 2.3
seconds to deserialize.

After reading some previous archive messages, I decided to try some manual
binary serialization using a custom DataSet (myDataSet). I have slightly
improved on my deserialization time (now takes 0.9 seconds) but my
serialzation has gone mad (takes 10 seconds). All this time is being spent
after the myDataSet.GetOb jectData method. What is hapenning after this
method?
any advice on how to improve this time would be appreciated.
thanks

Code
------
IFormatter formater = new BinaryFormatter ();
formatter.Seria lize(stream, myds); // myds=myDataSet, stream=MemorySt ream
....
myDataSet myds2 = (myDataSet)form atter2.Deserial ize(stream2);

myDataSet class
----------------------
[Serializable]
public class myDataSet : DataSet, ISerializable
{
public myDataSet() : base()
{}
public myDataSet(strin g name) : base(name)
{}
protected myDataSet(Seria lizationInfo sinfo, StreamingContex t context)
{
// Extract from the serialization info
ArrayList tables = new ArrayList();
tables = (ArrayList) sinfo.GetValue( "Tables", typeof(ArrayLis t));

// foreach table name in tables create a DataTable and add to table
collection
for(int i=0;i<tables.Co unt; i++)
{
string tableName = (string)tables[i];
DataTable dt = new DataTable(table Name);

ArrayList colNames = new ArrayList();
ArrayList colTypes = new ArrayList();
ArrayList dataRows = new ArrayList();
colNames = (ArrayList) sinfo.GetValue( "ColNames_"+tab leName,
typeof(ArrayLis t));
colTypes = (ArrayList) sinfo.GetValue( "ColTypes_"+tab leName,
typeof(ArrayLis t));
dataRows = (ArrayList) sinfo.GetValue( "DataRows_"+tab leName,
typeof(ArrayLis t));

// Add columns
for(int iCol=0; iCol<colNames.C ount; iCol++)
{
DataColumn col = new DataColumn(colN ames[iCol].ToString(),
Type.GetType(co lTypes[iCol].ToString() ));
dt.Columns.Add( col);
}

// Add rows
for(int iRows=0; iRows<dataRows. Count; iRows++)
{
DataRow row = dt.NewRow();
row.ItemArray = (object[]) dataRows[iRows];
dt.Rows.Add(row );
}

this.AcceptChan ges();

// Add table to Tables collection
this.Tables.Add (dt);
}

void ISerializable.G etObjectData(Se rializationInfo sinfo,
StreamingContex t context)
{
// Add arrays
ArrayList tables = new ArrayList();

foreach(DataTab le dt in this.Tables)
{
ArrayList colNames = new ArrayList();
ArrayList colTypes = new ArrayList();
ArrayList dataRows = new ArrayList();
string tableName = dt.TableName ;

// Insert the table name into worker array
tables.Add(tabl eName);

// Insert column information (names and types) into worker arrays
for(int iCols=0; iCols<dt.Column s.Count; iCols++)
{
DataColumn Col = dt.Columns[iCols];
colNames.Add(Co l.ColumnName);
colTypes.Add(Co l.DataType.Full Name);
}

for(int iRows=0; iRows<dt.Rows.C ount; iRows++)
{
dataRows.Add(dt .Rows[iRows].ItemArray);
}
// Add arrays to the serialization info structure
sinfo.AddValue( "ColNames_"+tab leName, colNames);
sinfo.AddValue( "ColTypes_"+tab leName, colTypes);
sinfo.AddValue( "DataRows_"+tab leName, dataRows);
}
sinfo.AddValue( "Tables", tables);
}
}
Nov 15 '05 #1
4 8711

Hi
I add about more than 10000 records in database and use your code to
serialize it.
I met the same problem as you.
My serialization time is 19 seconds and my deserialization time is 3
seconds.

I will do some research and give you some suggestion.

Best regards,
Jeffrey Tan
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

--------------------
| Reply-To: "hs" <hs@nospam.sy n>
| From: "hs" <hs@nospam.sy n>
| Subject: Manual binary serialization of a dataset is slow for me - why?
| Date: Wed, 13 Aug 2003 16:28:43 +0100
| Lines: 119
| X-Priority: 3
| X-MSMail-Priority: Normal
| X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
| X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
| Message-ID: <Ov************ **@TK2MSFTNGP10 .phx.gbl>
| Newsgroups: microsoft.publi c.dotnet.langua ges.csharp
| NNTP-Posting-Host: mailgate.synerg y-logistics.co.uk 62.49.130.162
| Path: cpmsftngxa06.ph x.gbl!TK2MSFTNG P08.phx.gbl!TK2 MSFTNGP10.phx.g bl
| Xref: cpmsftngxa06.ph x.gbl microsoft.publi c.dotnet.langua ges.csharp:1761 76
| X-Tomcat-NG: microsoft.publi c.dotnet.langua ges.csharp
|
| Hi
| I am serializing a dataset using a binary formatter as follows:
|
| IFormatter formater = new BinaryFormatter ();
| formatter.Seria lize(stream, ds); // ds=DataSet, stream=MemorySt ream
| ...
| DataSet ds2 = (DataSet)format ter2.Deserializ e(stream2);
|
| For the size of my DataSet, its taking 0.8 seconds to serialize and 2.3
| seconds to deserialize.
|
| After reading some previous archive messages, I decided to try some manual
| binary serialization using a custom DataSet (myDataSet). I have slightly
| improved on my deserialization time (now takes 0.9 seconds) but my
| serialzation has gone mad (takes 10 seconds). All this time is being spent
| after the myDataSet.GetOb jectData method. What is hapenning after this
| method?
| any advice on how to improve this time would be appreciated.
| thanks
|
| Code
| ------
| IFormatter formater = new BinaryFormatter ();
| formatter.Seria lize(stream, myds); // myds=myDataSet, stream=MemorySt ream
| ...
| myDataSet myds2 = (myDataSet)form atter2.Deserial ize(stream2);
|
| myDataSet class
| ----------------------
| [Serializable]
| public class myDataSet : DataSet, ISerializable
| {
| public myDataSet() : base()
| {}
| public myDataSet(strin g name) : base(name)
| {}
| protected myDataSet(Seria lizationInfo sinfo, StreamingContex t context)
| {
| // Extract from the serialization info
| ArrayList tables = new ArrayList();
| tables = (ArrayList) sinfo.GetValue( "Tables", typeof(ArrayLis t));
|
| // foreach table name in tables create a DataTable and add to table
| collection
| for(int i=0;i<tables.Co unt; i++)
| {
| string tableName = (string)tables[i];
| DataTable dt = new DataTable(table Name);
|
| ArrayList colNames = new ArrayList();
| ArrayList colTypes = new ArrayList();
| ArrayList dataRows = new ArrayList();
| colNames = (ArrayList) sinfo.GetValue( "ColNames_"+tab leName,
| typeof(ArrayLis t));
| colTypes = (ArrayList) sinfo.GetValue( "ColTypes_"+tab leName,
| typeof(ArrayLis t));
| dataRows = (ArrayList) sinfo.GetValue( "DataRows_"+tab leName,
| typeof(ArrayLis t));
|
| // Add columns
| for(int iCol=0; iCol<colNames.C ount; iCol++)
| {
| DataColumn col = new DataColumn(colN ames[iCol].ToString(),
| Type.GetType(co lTypes[iCol].ToString() ));
| dt.Columns.Add( col);
| }
|
| // Add rows
| for(int iRows=0; iRows<dataRows. Count; iRows++)
| {
| DataRow row = dt.NewRow();
| row.ItemArray = (object[]) dataRows[iRows];
| dt.Rows.Add(row );
| }
|
| this.AcceptChan ges();
|
| // Add table to Tables collection
| this.Tables.Add (dt);
| }
|
| void ISerializable.G etObjectData(Se rializationInfo sinfo,
| StreamingContex t context)
| {
| // Add arrays
| ArrayList tables = new ArrayList();
|
| foreach(DataTab le dt in this.Tables)
| {
| ArrayList colNames = new ArrayList();
| ArrayList colTypes = new ArrayList();
| ArrayList dataRows = new ArrayList();
| string tableName = dt.TableName ;
|
| // Insert the table name into worker array
| tables.Add(tabl eName);
|
| // Insert column information (names and types) into worker arrays
| for(int iCols=0; iCols<dt.Column s.Count; iCols++)
| {
| DataColumn Col = dt.Columns[iCols];
| colNames.Add(Co l.ColumnName);
| colTypes.Add(Co l.DataType.Full Name);
| }
|
| for(int iRows=0; iRows<dt.Rows.C ount; iRows++)
| {
| dataRows.Add(dt .Rows[iRows].ItemArray);
| }
| // Add arrays to the serialization info structure
| sinfo.AddValue( "ColNames_"+tab leName, colNames);
| sinfo.AddValue( "ColTypes_"+tab leName, colTypes);
| sinfo.AddValue( "DataRows_"+tab leName, dataRows);
| }
| sinfo.AddValue( "Tables", tables);
| }
| }
|
|
|

Nov 15 '05 #2
You may want to change the formatter type to XsdString, like below:. This
will make your serialization/deserialization much faster.

BinaryFormatter fmtobj = new BinaryFormatter ();
IFormatter formatter = (IFormatter)fmt obj;
FormatterTypeSt yle fts = fmtobj.TypeForm at;
fmtobj.TypeForm at = fts | FormatterTypeSt yle.XsdString;

Best regards,
Xiao Xie
Microsoft Developer Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

Nov 15 '05 #3
You may want to change the formatter type to XsdString, like below:. This
will make your serialization/deserialization much faster.

BinaryFormatter fmtobj = new BinaryFormatter ();
IFormatter formatter = (IFormatter)fmt obj;
FormatterTypeSt yle fts = fmtobj.TypeForm at;
fmtobj.TypeForm at = fts | FormatterTypeSt yle.XsdString;

Best regards,
Xiao Xie
Microsoft Developer Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

Nov 15 '05 #4
hs
Xiao and Jeffrey,

Thank you for your help. Your suggestion has worked.

Nov 15 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
7813
by: phyzics | last post by:
I am porting an application from C++ to C#, and am having trouble finding a way to quickly and efficiently write structures to a binary file. In C++ this is trivial because all that is necessary is to pack the structure to 1 byte boundries, and then just write out the structure directly to the File IO function pragma pack (1 typedef struct char var1 int var1 }MyStruc fwrite(&myStructure,sizeof(MyStruct),1,filepointer);
11
11961
by: ajou_king | last post by:
I was running some tests on my Win32 1GHZ processor to see how long it would take to transmit objects numerous times via TCP/IP using C# ..NET Remoting vs the C++ trustworthy method of binary streams. I ran the test for 50K, 100K, 500K iterations, where each iteration consists of sending an object from a client process to a server process, and the server process sends back an ack. Here are the results: .NET Remoting C++...
7
9559
by: schoenfeld1 | last post by:
I've implemented IPC between two applications using named pipes and binary serialization, but have noticed that the binary formatter is rather slow. It seems that the binary formatter reflects the entire type everytime it is invoked to serialize/deserialize an object of that type. Is there a way to prepare the binary formatter with a pre-defined type, such that it only reflects once but can be re-used to serialize/deserialize objects...
2
2167
by: Wild Wind | last post by:
Hello, I have an object which has various properties, including one which is of type DataSet. I need to serialize this object in binary format, but I've discovered that the BinaryFormatter will only format datasets in XML format. I've read various articles on how to address the problem, including the KB829740 article which talks about using a DataSetSurrogate
1
3697
by: Matt | last post by:
I have a web service that currently returns a dataset. Depending on the data being returned its size will be in megabytes (XML Document could be possibly 100 or more megabytes). To speed up the data transfer I'm looking at two options: 1. One is compress the XML generated by the Dataset and transfer it to the client as an array of bytes. (I was able to compress down to 2 MB) 2. The second option is to use binary serialization. I've...
3
7003
by: stockblaster | last post by:
Hello all.. Is it possible to convert a DataTable (i create the DataTable from a CSV file) into binary data and save it into an sql 2005 table (into binary field). After that I want to have the ability to add a row to the beginning of the to the binary data..
0
1144
by: Bilz | last post by:
All, I have a rather complex object graph that I can use serialization to serialize to XML or Binary. I find that deserialiation is very slow in either case, but was REALLY suprised that binary serialization was significantly slower than XML. I did not expect this. I know that the first attempt to deserialize is always the slowest, but
7
1947
by: Mike9900 | last post by:
If you do .NET Remoting and Binary Serilization with DataSet it does not work. If you set the Serialization type to Binary on DataSet, it does not seralize it correctly on windows xp. To do this set the serilization to Binary and use it in remoting on Windows XP. Is there a way around this? -- Mike
1
11104
by: kikisan | last post by:
I am developing a windows service which utilizes the following classes: interface IPersistable; abstract class PersistableObject : IPersistable;
0
9645
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9480
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10327
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10151
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8973
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
6740
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5511
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4053
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
3647
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.