473,695 Members | 2,577 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Manual binary serialization of a dataset is slow for me - why?

hs
Hi
I am serializing a dataset using a binary formatter as follows:

IFormatter formater = new BinaryFormatter ();
formatter.Seria lize(stream, ds); // ds=DataSet, stream=MemorySt ream
....
DataSet ds2 = (DataSet)format ter2.Deserializ e(stream2);

For the size of my DataSet, its taking 0.8 seconds to serialize and 2.3
seconds to deserialize.

After reading some previous archive messages, I decided to try some manual
binary serialization using a custom DataSet (myDataSet). I have slightly
improved on my deserialization time (now takes 0.9 seconds) but my
serialzation has gone mad (takes 10 seconds). All this time is being spent
after the myDataSet.GetOb jectData method. What is hapenning after this
method?
any advice on how to improve this time would be appreciated.
thanks

Code
------
IFormatter formater = new BinaryFormatter ();
formatter.Seria lize(stream, myds); // myds=myDataSet, stream=MemorySt ream
....
myDataSet myds2 = (myDataSet)form atter2.Deserial ize(stream2);

myDataSet class
----------------------
[Serializable]
public class myDataSet : DataSet, ISerializable
{
public myDataSet() : base()
{}
public myDataSet(strin g name) : base(name)
{}
protected myDataSet(Seria lizationInfo sinfo, StreamingContex t context)
{
// Extract from the serialization info
ArrayList tables = new ArrayList();
tables = (ArrayList) sinfo.GetValue( "Tables", typeof(ArrayLis t));

// foreach table name in tables create a DataTable and add to table
collection
for(int i=0;i<tables.Co unt; i++)
{
string tableName = (string)tables[i];
DataTable dt = new DataTable(table Name);

ArrayList colNames = new ArrayList();
ArrayList colTypes = new ArrayList();
ArrayList dataRows = new ArrayList();
colNames = (ArrayList) sinfo.GetValue( "ColNames_"+tab leName,
typeof(ArrayLis t));
colTypes = (ArrayList) sinfo.GetValue( "ColTypes_"+tab leName,
typeof(ArrayLis t));
dataRows = (ArrayList) sinfo.GetValue( "DataRows_"+tab leName,
typeof(ArrayLis t));

// Add columns
for(int iCol=0; iCol<colNames.C ount; iCol++)
{
DataColumn col = new DataColumn(colN ames[iCol].ToString(),
Type.GetType(co lTypes[iCol].ToString() ));
dt.Columns.Add( col);
}

// Add rows
for(int iRows=0; iRows<dataRows. Count; iRows++)
{
DataRow row = dt.NewRow();
row.ItemArray = (object[]) dataRows[iRows];
dt.Rows.Add(row );
}

this.AcceptChan ges();

// Add table to Tables collection
this.Tables.Add (dt);
}

void ISerializable.G etObjectData(Se rializationInfo sinfo,
StreamingContex t context)
{
// Add arrays
ArrayList tables = new ArrayList();

foreach(DataTab le dt in this.Tables)
{
ArrayList colNames = new ArrayList();
ArrayList colTypes = new ArrayList();
ArrayList dataRows = new ArrayList();
string tableName = dt.TableName ;

// Insert the table name into worker array
tables.Add(tabl eName);

// Insert column information (names and types) into worker arrays
for(int iCols=0; iCols<dt.Column s.Count; iCols++)
{
DataColumn Col = dt.Columns[iCols];
colNames.Add(Co l.ColumnName);
colTypes.Add(Co l.DataType.Full Name);
}

for(int iRows=0; iRows<dt.Rows.C ount; iRows++)
{
dataRows.Add(dt .Rows[iRows].ItemArray);
}
// Add arrays to the serialization info structure
sinfo.AddValue( "ColNames_"+tab leName, colNames);
sinfo.AddValue( "ColTypes_"+tab leName, colTypes);
sinfo.AddValue( "DataRows_"+tab leName, dataRows);
}
sinfo.AddValue( "Tables", tables);
}
}
Nov 15 '05 #1
4 8703

Hi
I add about more than 10000 records in database and use your code to
serialize it.
I met the same problem as you.
My serialization time is 19 seconds and my deserialization time is 3
seconds.

I will do some research and give you some suggestion.

Best regards,
Jeffrey Tan
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

--------------------
| Reply-To: "hs" <hs@nospam.sy n>
| From: "hs" <hs@nospam.sy n>
| Subject: Manual binary serialization of a dataset is slow for me - why?
| Date: Wed, 13 Aug 2003 16:28:43 +0100
| Lines: 119
| X-Priority: 3
| X-MSMail-Priority: Normal
| X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
| X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
| Message-ID: <Ov************ **@TK2MSFTNGP10 .phx.gbl>
| Newsgroups: microsoft.publi c.dotnet.langua ges.csharp
| NNTP-Posting-Host: mailgate.synerg y-logistics.co.uk 62.49.130.162
| Path: cpmsftngxa06.ph x.gbl!TK2MSFTNG P08.phx.gbl!TK2 MSFTNGP10.phx.g bl
| Xref: cpmsftngxa06.ph x.gbl microsoft.publi c.dotnet.langua ges.csharp:1761 76
| X-Tomcat-NG: microsoft.publi c.dotnet.langua ges.csharp
|
| Hi
| I am serializing a dataset using a binary formatter as follows:
|
| IFormatter formater = new BinaryFormatter ();
| formatter.Seria lize(stream, ds); // ds=DataSet, stream=MemorySt ream
| ...
| DataSet ds2 = (DataSet)format ter2.Deserializ e(stream2);
|
| For the size of my DataSet, its taking 0.8 seconds to serialize and 2.3
| seconds to deserialize.
|
| After reading some previous archive messages, I decided to try some manual
| binary serialization using a custom DataSet (myDataSet). I have slightly
| improved on my deserialization time (now takes 0.9 seconds) but my
| serialzation has gone mad (takes 10 seconds). All this time is being spent
| after the myDataSet.GetOb jectData method. What is hapenning after this
| method?
| any advice on how to improve this time would be appreciated.
| thanks
|
| Code
| ------
| IFormatter formater = new BinaryFormatter ();
| formatter.Seria lize(stream, myds); // myds=myDataSet, stream=MemorySt ream
| ...
| myDataSet myds2 = (myDataSet)form atter2.Deserial ize(stream2);
|
| myDataSet class
| ----------------------
| [Serializable]
| public class myDataSet : DataSet, ISerializable
| {
| public myDataSet() : base()
| {}
| public myDataSet(strin g name) : base(name)
| {}
| protected myDataSet(Seria lizationInfo sinfo, StreamingContex t context)
| {
| // Extract from the serialization info
| ArrayList tables = new ArrayList();
| tables = (ArrayList) sinfo.GetValue( "Tables", typeof(ArrayLis t));
|
| // foreach table name in tables create a DataTable and add to table
| collection
| for(int i=0;i<tables.Co unt; i++)
| {
| string tableName = (string)tables[i];
| DataTable dt = new DataTable(table Name);
|
| ArrayList colNames = new ArrayList();
| ArrayList colTypes = new ArrayList();
| ArrayList dataRows = new ArrayList();
| colNames = (ArrayList) sinfo.GetValue( "ColNames_"+tab leName,
| typeof(ArrayLis t));
| colTypes = (ArrayList) sinfo.GetValue( "ColTypes_"+tab leName,
| typeof(ArrayLis t));
| dataRows = (ArrayList) sinfo.GetValue( "DataRows_"+tab leName,
| typeof(ArrayLis t));
|
| // Add columns
| for(int iCol=0; iCol<colNames.C ount; iCol++)
| {
| DataColumn col = new DataColumn(colN ames[iCol].ToString(),
| Type.GetType(co lTypes[iCol].ToString() ));
| dt.Columns.Add( col);
| }
|
| // Add rows
| for(int iRows=0; iRows<dataRows. Count; iRows++)
| {
| DataRow row = dt.NewRow();
| row.ItemArray = (object[]) dataRows[iRows];
| dt.Rows.Add(row );
| }
|
| this.AcceptChan ges();
|
| // Add table to Tables collection
| this.Tables.Add (dt);
| }
|
| void ISerializable.G etObjectData(Se rializationInfo sinfo,
| StreamingContex t context)
| {
| // Add arrays
| ArrayList tables = new ArrayList();
|
| foreach(DataTab le dt in this.Tables)
| {
| ArrayList colNames = new ArrayList();
| ArrayList colTypes = new ArrayList();
| ArrayList dataRows = new ArrayList();
| string tableName = dt.TableName ;
|
| // Insert the table name into worker array
| tables.Add(tabl eName);
|
| // Insert column information (names and types) into worker arrays
| for(int iCols=0; iCols<dt.Column s.Count; iCols++)
| {
| DataColumn Col = dt.Columns[iCols];
| colNames.Add(Co l.ColumnName);
| colTypes.Add(Co l.DataType.Full Name);
| }
|
| for(int iRows=0; iRows<dt.Rows.C ount; iRows++)
| {
| dataRows.Add(dt .Rows[iRows].ItemArray);
| }
| // Add arrays to the serialization info structure
| sinfo.AddValue( "ColNames_"+tab leName, colNames);
| sinfo.AddValue( "ColTypes_"+tab leName, colTypes);
| sinfo.AddValue( "DataRows_"+tab leName, dataRows);
| }
| sinfo.AddValue( "Tables", tables);
| }
| }
|
|
|

Nov 15 '05 #2
You may want to change the formatter type to XsdString, like below:. This
will make your serialization/deserialization much faster.

BinaryFormatter fmtobj = new BinaryFormatter ();
IFormatter formatter = (IFormatter)fmt obj;
FormatterTypeSt yle fts = fmtobj.TypeForm at;
fmtobj.TypeForm at = fts | FormatterTypeSt yle.XsdString;

Best regards,
Xiao Xie
Microsoft Developer Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

Nov 15 '05 #3
You may want to change the formatter type to XsdString, like below:. This
will make your serialization/deserialization much faster.

BinaryFormatter fmtobj = new BinaryFormatter ();
IFormatter formatter = (IFormatter)fmt obj;
FormatterTypeSt yle fts = fmtobj.TypeForm at;
fmtobj.TypeForm at = fts | FormatterTypeSt yle.XsdString;

Best regards,
Xiao Xie
Microsoft Developer Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

Nov 15 '05 #4
hs
Xiao and Jeffrey,

Thank you for your help. Your suggestion has worked.

Nov 15 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
7805
by: phyzics | last post by:
I am porting an application from C++ to C#, and am having trouble finding a way to quickly and efficiently write structures to a binary file. In C++ this is trivial because all that is necessary is to pack the structure to 1 byte boundries, and then just write out the structure directly to the File IO function pragma pack (1 typedef struct char var1 int var1 }MyStruc fwrite(&myStructure,sizeof(MyStruct),1,filepointer);
11
11946
by: ajou_king | last post by:
I was running some tests on my Win32 1GHZ processor to see how long it would take to transmit objects numerous times via TCP/IP using C# ..NET Remoting vs the C++ trustworthy method of binary streams. I ran the test for 50K, 100K, 500K iterations, where each iteration consists of sending an object from a client process to a server process, and the server process sends back an ack. Here are the results: .NET Remoting C++...
7
9555
by: schoenfeld1 | last post by:
I've implemented IPC between two applications using named pipes and binary serialization, but have noticed that the binary formatter is rather slow. It seems that the binary formatter reflects the entire type everytime it is invoked to serialize/deserialize an object of that type. Is there a way to prepare the binary formatter with a pre-defined type, such that it only reflects once but can be re-used to serialize/deserialize objects...
2
2160
by: Wild Wind | last post by:
Hello, I have an object which has various properties, including one which is of type DataSet. I need to serialize this object in binary format, but I've discovered that the BinaryFormatter will only format datasets in XML format. I've read various articles on how to address the problem, including the KB829740 article which talks about using a DataSetSurrogate
1
3692
by: Matt | last post by:
I have a web service that currently returns a dataset. Depending on the data being returned its size will be in megabytes (XML Document could be possibly 100 or more megabytes). To speed up the data transfer I'm looking at two options: 1. One is compress the XML generated by the Dataset and transfer it to the client as an array of bytes. (I was able to compress down to 2 MB) 2. The second option is to use binary serialization. I've...
3
6997
by: stockblaster | last post by:
Hello all.. Is it possible to convert a DataTable (i create the DataTable from a CSV file) into binary data and save it into an sql 2005 table (into binary field). After that I want to have the ability to add a row to the beginning of the to the binary data..
0
1138
by: Bilz | last post by:
All, I have a rather complex object graph that I can use serialization to serialize to XML or Binary. I find that deserialiation is very slow in either case, but was REALLY suprised that binary serialization was significantly slower than XML. I did not expect this. I know that the first attempt to deserialize is always the slowest, but
7
1942
by: Mike9900 | last post by:
If you do .NET Remoting and Binary Serilization with DataSet it does not work. If you set the Serialization type to Binary on DataSet, it does not seralize it correctly on windows xp. To do this set the serilization to Binary and use it in remoting on Windows XP. Is there a way around this? -- Mike
1
11096
by: kikisan | last post by:
I am developing a windows service which utilizes the following classes: interface IPersistable; abstract class PersistableObject : IPersistable;
0
8559
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9112
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8975
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8826
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8818
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
5832
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4338
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
2996
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
1971
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.