473,322 Members | 1,352 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

SQLXML Bulk Loading or MSMQ

Hi,
I have a batch process that runs in the night on a daily basis.
Basically one of the folders gets a huge xml file which has multiple
nodes
of same type, as shown below :
<myParentNode>
<ChildNode>
<a>..</a>
<b>..</b>
<c>..</c>
</childNode>
<ChildNode>
<a>..</a>
<b>..</b>
<c>..</c>
</childNode>
<ChildNode>
<a>..</a>
<b>..</b>
<c>..</c>
</childNode>
</myParentNode>

Basically there could be around 50,000 childNodes carrying lot of info.
The batch Process as of today just BulkLoads the xml into a staging table
and then runs a Stored Proc that copies data from the Staging table to
the
appropriate Master tables.

I was just thinking what could be another alternative to this. How good
is
MSMQ in this scenario. I can put the xml into a queue and then take it,
break it into individual small xmls each containing one childNode xml and
send it to another queue that processes it and sends the data into the
Master tables.

Which is the better thing to do here ? Are there any other alternatives
for this ?
Thanks in advance.

Cheers,
Kris
Nov 12 '05 #1
0 1328

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Chris | last post by:
Any help would be appreciated. I am running a script that does the following in succession. 1-Drop existing database and create new database 2-Defines tables, stored procedures and functions...
2
by: David Sharp | last post by:
I've been doing some experiments with speeding up copying tables of approximately 1 million rows between databases using BCP and BULK INSERT. I noticed that the total time for removing the...
1
by: Mark Sisson | last post by:
Ok gurus, what would you do? I'm developing an app in C# and trying to use SqlXml but there seems to be a problem at every turn. I want to create a biz object that represents an order to sell a...
0
by: ram_kri | last post by:
Hi, I have a batch process that runs in the night on a daily basis. Basically one of the folders gets a huge xml file which has multiple nodes of same type, as shown below : <myParentNode>...
0
by: Paula DiTallo | last post by:
Hello Techies-- I am reading an XML document with many nested elements. Many of these elements go to such a variety of tables, that I am trying to simply the read of the document by dumping the...
3
by: moonriver | last post by:
Right now I develop an application to retrieve over 30,000 records from a binary file and then load them into a SQL Server DB. So far I load those records one by one, but the performance is very...
1
by: campwes | last post by:
Hey, all! We're having trouble displaying SQL Server data in a web site as XML, using a transform. The query is a simple SELECT statement. When running the query, we get the following error from...
4
by: amjad | last post by:
Hi i have dataset which i write as xml file like ds.WriteXmlSchema(XSDFilename); ds.WriteXml(XMLFilename,XmlWriteMode.IgnoreSchema); then i try to load that file into sql using below code...
4
by: vicpada | last post by:
Hi all, I am newbie in all the stuff about xml importing into sql server. What I try to do is simple. It is take an xml file and drop it into a table. I am using VS2005, SQLXML 4.0 and...
5
by: yeoj13 | last post by:
Hello, I have a db2load script I'm using to populate a large table. Ideally, my target table is required to have "Not Null" constraints on a number of different columns. I've noticed a ...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.