XmlDataDocument\DataSet Memory Footprint, Feedback appreciated.

[Derrick]

I am reading in xml files that equate to sql tables, via XmlDataDocument,
and then operating on the DataSet. With the most simple app that just loads
the xml doc, I see the memory footprint of the app grow to roughly 5x the
size of the xml document after the xml doc is read. Can anyone comment on
this? Is this expected?

Below are two samples, the first hits northwind, and creates a 19meg
(roughly, on my machine) xml file. The second simply reads it in, with a
pause after reading so you can inspect the memory usage in task manager.

Any suggestions as to how I can get the footprint down much closer to the
size of the xml file would be greatly appreciated.

Thanks in advance!!

Derrick


Creation of sample xml file:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Xml;
namespace DataExtract
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
string forXml = " FOR XML AUTO, XMLDATA";
SqlConnection mySqlConnection = new
SqlConnection("server=127.0.0.1;database=northwind ;uid=sa;pwd=admin");
XmlTextReader myXmlReader = null;
SqlCommand mySqlCmd = new SqlCommand();
DataSet myDataSet = new DataSet();
mySqlConnection.Open();
mySqlCmd.Connection = mySqlConnection;
mySqlCmd.CommandText = "select * from orders cross join customers" + forXml;
myXmlReader = (XmlTextReader)mySqlCmd.ExecuteXmlReader();
myDataSet.ReadXml(myXmlReader, XmlReadMode.Fragment);
myXmlReader.Close();
myDataSet.WriteXml("c:\\lotsa_data.xml");
myDataSet.WriteXmlSchema("c:\\lotsa_data.xsd");
myDataSet.Dispose();
mySqlConnection.Close();
}
}
}

The reader:

using System.Data;
using System.Xml;
using System.IO;
namespace DataRead
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
Stream xsdStream = new StreamReader("c:\\lotsa_data.xsd").BaseStream;
Stream xmlStream = new StreamReader("c:\\lotsa_data.xml").BaseStream;
XmlDataDocument myXmlDoc = new XmlDataDocument();
myXmlDoc.DataSet.ReadXmlSchema(xsdStream);
myXmlDoc.Load(xmlStream);
Console.WriteLine("Loaded...");
Console.ReadLine();
}
}
}

[Sherif ElMetainy]

Hello

I suggest you use a memory profiler to track where your memory is consumed.
Here is a good one
http://www.scitech.se/memprofiler/
This software tells you the datatypes of allocated objects

Best regards
Sherif

"Derrick" <de*********@excite.com> wrote in message
news:Oo*************@tk2msftngp13.phx.gbl...

I am reading in xml files that equate to sql tables, via XmlDataDocument,
and then operating on the DataSet. With the most simple app that just loads the xml doc, I see the memory footprint of the app grow to roughly 5x the
size of the xml document after the xml doc is read. Can anyone comment on
this? Is this expected?

Below are two samples, the first hits northwind, and creates a 19meg
(roughly, on my machine) xml file. The second simply reads it in, with a
pause after reading so you can inspect the memory usage in task manager.

Any suggestions as to how I can get the footprint down much closer to the
size of the xml file would be greatly appreciated.

Thanks in advance!!

Derrick


Creation of sample xml file:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Xml;
namespace DataExtract
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
string forXml = " FOR XML AUTO, XMLDATA";
SqlConnection mySqlConnection = new
SqlConnection("server=127.0.0.1;database=northwind ;uid=sa;pwd=admin");
XmlTextReader myXmlReader = null;
SqlCommand mySqlCmd = new SqlCommand();
DataSet myDataSet = new DataSet();
mySqlConnection.Open();
mySqlCmd.Connection = mySqlConnection;
mySqlCmd.CommandText = "select * from orders cross join customers" + forXml; myXmlReader = (XmlTextReader)mySqlCmd.ExecuteXmlReader();
myDataSet.ReadXml(myXmlReader, XmlReadMode.Fragment);
myXmlReader.Close();
myDataSet.WriteXml("c:\\lotsa_data.xml");
myDataSet.WriteXmlSchema("c:\\lotsa_data.xsd");
myDataSet.Dispose();
mySqlConnection.Close();
}
}
}

The reader:

using System.Data;
using System.Xml;
using System.IO;
namespace DataRead
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
Stream xsdStream = new StreamReader("c:\\lotsa_data.xsd").BaseStream;
Stream xmlStream = new StreamReader("c:\\lotsa_data.xml").BaseStream;
XmlDataDocument myXmlDoc = new XmlDataDocument();
myXmlDoc.DataSet.ReadXmlSchema(xsdStream);
myXmlDoc.Load(xmlStream);
Console.WriteLine("Loaded...");
Console.ReadLine();
}
}
}

[Derrick]

Thanks, downloaded and tried it out, it basically looks like XML objects
(elements and attributes) are half the bag, strings the other half. Is
there a way to read in the XML as simple data, toss the xml associated
objects out, and end up with just native types in the DataSet tables?

The code that reads the xml file into a dataset is all core C#...

Thanks again!
"Sherif ElMetainy" <el*************@wayout.net.NOSPAM> wrote in message
news:%2*****************@TK2MSFTNGP11.phx.gbl...

Hello

I suggest you use a memory profiler to track where your memory is consumed. Here is a good one
http://www.scitech.se/memprofiler/
This software tells you the datatypes of allocated objects

Best regards
Sherif

"Derrick" <de*********@excite.com> wrote in message
news:Oo*************@tk2msftngp13.phx.gbl...

I am reading in xml files that equate to sql tables, via XmlDataDocument, and then operating on the DataSet. With the most simple app that just

loads

the xml doc, I see the memory footprint of the app grow to roughly 5x the size of the xml document after the xml doc is read. Can anyone comment on this? Is this expected?

Below are two samples, the first hits northwind, and creates a 19meg
(roughly, on my machine) xml file. The second simply reads it in, with a pause after reading so you can inspect the memory usage in task manager.

Any suggestions as to how I can get the footprint down much closer to the size of the xml file would be greatly appreciated.

Thanks in advance!!

Derrick


Creation of sample xml file:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Xml;
namespace DataExtract
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
string forXml = " FOR XML AUTO, XMLDATA";
SqlConnection mySqlConnection = new
SqlConnection("server=127.0.0.1;database=northwind ;uid=sa;pwd=admin");
XmlTextReader myXmlReader = null;
SqlCommand mySqlCmd = new SqlCommand();
DataSet myDataSet = new DataSet();
mySqlConnection.Open();
mySqlCmd.Connection = mySqlConnection;
mySqlCmd.CommandText = "select * from orders cross join customers" +

forXml;

myXmlReader = (XmlTextReader)mySqlCmd.ExecuteXmlReader();
myDataSet.ReadXml(myXmlReader, XmlReadMode.Fragment);
myXmlReader.Close();
myDataSet.WriteXml("c:\\lotsa_data.xml");
myDataSet.WriteXmlSchema("c:\\lotsa_data.xsd");
myDataSet.Dispose();
mySqlConnection.Close();
}
}
}

The reader:

using System.Data;
using System.Xml;
using System.IO;
namespace DataRead
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
Stream xsdStream = new StreamReader("c:\\lotsa_data.xsd").BaseStream;
Stream xmlStream = new StreamReader("c:\\lotsa_data.xml").BaseStream;
XmlDataDocument myXmlDoc = new XmlDataDocument();
myXmlDoc.DataSet.ReadXmlSchema(xsdStream);
myXmlDoc.Load(xmlStream);
Console.WriteLine("Loaded...");
Console.ReadLine();
}
}
}

[Sherif ElMetainy]

Hello

You can set the references to all xml objects to null, so that they are not
referenced anymore by your code.
Then they will be collected in the next garbage collection.

You can also force garbage collection to run immediately by calling
System.GC.Collect(), but many people thinks its better that you let the CLR
do garbage collection when it wants.

Best regards,
Sherif

"Derrick" <de*********@excite.com> wrote in message
news:eD**************@tk2msftngp13.phx.gbl...

Thanks, downloaded and tried it out, it basically looks like XML objects
(elements and attributes) are half the bag, strings the other half. Is
there a way to read in the XML as simple data, toss the xml associated
objects out, and end up with just native types in the DataSet tables?

The code that reads the xml file into a dataset is all core C#...

Thanks again!
"Sherif ElMetainy" <el*************@wayout.net.NOSPAM> wrote in message
news:%2*****************@TK2MSFTNGP11.phx.gbl...

Hello

I suggest you use a memory profiler to track where your memory is consumed.

Here is a good one
http://www.scitech.se/memprofiler/
This software tells you the datatypes of allocated objects

Best regards
Sherif

"Derrick" <de*********@excite.com> wrote in message
news:Oo*************@tk2msftngp13.phx.gbl...

I am reading in xml files that equate to sql tables, via XmlDataDocument, and then operating on the DataSet. With the most simple app that just

loads

the xml doc, I see the memory footprint of the app grow to roughly 5x the size of the xml document after the xml doc is read. Can anyone comment on

this? Is this expected?

Below are two samples, the first hits northwind, and creates a 19meg
(roughly, on my machine) xml file. The second simply reads it in,

with

a pause after reading so you can inspect the memory usage in task

manager.
Any suggestions as to how I can get the footprint down much closer to

the size of the xml file would be greatly appreciated.

Thanks in advance!!

Derrick


Creation of sample xml file:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Xml;
namespace DataExtract
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
string forXml = " FOR XML AUTO, XMLDATA";
SqlConnection mySqlConnection = new
SqlConnection("server=127.0.0.1;database=northwind ;uid=sa;pwd=admin");
XmlTextReader myXmlReader = null;
SqlCommand mySqlCmd = new SqlCommand();
DataSet myDataSet = new DataSet();
mySqlConnection.Open();
mySqlCmd.Connection = mySqlConnection;
mySqlCmd.CommandText = "select * from orders cross join customers" +

forXml;

myXmlReader = (XmlTextReader)mySqlCmd.ExecuteXmlReader();
myDataSet.ReadXml(myXmlReader, XmlReadMode.Fragment);
myXmlReader.Close();
myDataSet.WriteXml("c:\\lotsa_data.xml");
myDataSet.WriteXmlSchema("c:\\lotsa_data.xsd");
myDataSet.Dispose();
mySqlConnection.Close();
}
}
}

The reader:

using System.Data;
using System.Xml;
using System.IO;
namespace DataRead
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
Stream xsdStream = new StreamReader("c:\\lotsa_data.xsd").BaseStream;
Stream xmlStream = new StreamReader("c:\\lotsa_data.xml").BaseStream;
XmlDataDocument myXmlDoc = new XmlDataDocument();
myXmlDoc.DataSet.ReadXmlSchema(xsdStream);
myXmlDoc.Load(xmlStream);
Console.WriteLine("Loaded...");
Console.ReadLine();
}
}
}