473,287 Members | 1,643 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

XML DOM, but in chunks

I have some very large XML files that are basically recordsets. I
would like to access each record, one-at-a-time, and I particularly
like the ElementTree library for accessing the data. Is there a way
to have ElementTree read only one record of the data at a time?
Alternatively, are there other ways that would allow one to parse out
a record at a time and maintain some nice ways of accessing the
elements within the record?

Thanks,
Sean

Oct 31 '07 #1
5 1167
On Wed, 31 Oct 2007 20:24:43 +0000, Sean Davis wrote:
I have some very large XML files that are basically recordsets. I
would like to access each record, one-at-a-time, and I particularly
like the ElementTree library for accessing the data. Is there a way
to have ElementTree read only one record of the data at a time?
Have you tried `iterparse()`?

Ciao,
Marc 'BlackJack' Rintsch
Oct 31 '07 #2

On 31 okt 2007, at 21.24, Sean Davis wrote:
I have some very large XML files that are basically recordsets. I
would like to access each record, one-at-a-time, and I particularly
like the ElementTree library for accessing the data. Is there a way
to have ElementTree read only one record of the data at a time?
Alternatively, are there other ways that would allow one to parse out
a record at a time and maintain some nice ways of accessing the
elements within the record?

Thanks,
Sean

--
http://mail.python.org/mailman/listinfo/python-list
You need a Python (or other language) implementation of the SAX API
----------------------------------
Skinheads are so tired of immigration, that they are going to move to
a country that don't accept immigrants!
Tommy Nordgren
to************@comhem.se

Oct 31 '07 #3

On 31 okt 2007, at 22.36, Tommy Nordgren wrote:
>
On 31 okt 2007, at 21.24, Sean Davis wrote:
>I have some very large XML files that are basically recordsets. I
would like to access each record, one-at-a-time, and I particularly
like the ElementTree library for accessing the data. Is there a way
to have ElementTree read only one record of the data at a time?
Alternatively, are there other ways that would allow one to parse out
a record at a time and maintain some nice ways of accessing the
elements within the record?

Thanks,
Sean

--
http://mail.python.org/mailman/listinfo/python-list
You need a Python (or other language) implementation of the SAX API
The module you need is xml.sax
----------------------------------
Skinheads are so tired of immigration, that they are going to move to
a country that don't accept immigrants!
Tommy Nordgren
to************@comhem.se

Nov 1 '07 #4
On Oct 31, 4:43 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.netwrote:
On Wed, 31 Oct 2007 20:24:43 +0000, Sean Davis wrote:
I have some very large XML files that are basically recordsets. I
would like to access each record, one-at-a-time, and I particularly
like the ElementTree library for accessing the data. Is there a way
to have ElementTree read only one record of the data at a time?

Have you tried `iterparse()`?

Ciao,
Marc 'BlackJack' Rintsch
Detailed docs at http://effbot.org/zone/element-iterparse.htm

George

Nov 1 '07 #5
Tommy Nordgren wrote:
>On 31 okt 2007, at 21.24, Sean Davis wrote:
>>I have some very large XML files that are basically recordsets. I
would like to access each record, one-at-a-time, and I particularly
like the ElementTree library for accessing the data. Is there a way
to have ElementTree read only one record of the data at a time?
Alternatively, are there other ways that would allow one to parse out
a record at a time and maintain some nice ways of accessing the
elements within the record?

Thanks,
Sean
You need a Python (or other language) implementation of the SAX API
The module you need is xml.sax
That's unlikely - few people "need" SAX. If they think they do, they just
don't know the alternatives. I think iterparse() is what he was looking for.

Stefan
Nov 1 '07 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Don | last post by:
Hi all, I need to read an uploaded file into chunks, so I can examine the acculative size between chunks to determine if a maximum size is being exceeded. I'm currently using the PHP function...
1
by: Guy Erez | last post by:
Hi, I'm running queries with MySql 4.0.17 that return thousands of records. Because I need to present them in GUI, I returieve the results in chunks using LIMIT, for example - get first 100,...
0
by: I Decker | last post by:
Hi all, Quite often I drag chunks of code onto the toolbox, so that I can reuse them later (even in different projects).I tend to drag them into a seperate tab that I created called code bits. I...
1
by: Nawaz Ijaz | last post by:
Hi All, Just need to know that how can it be possible to upload large file in chunks of bytes. Suppose i have a large file (100 MB) and i want to upload it in chunks (say 20 KB). How can this be...
1
by: Johan | last post by:
Here is my problem and I hope there is someone out-there who can help me. I want to search my domain for all users and groups. And since there are lot of them I want to read them back in chunks...
6
by: MackS | last post by:
Hello everyone I am faced with the following problem. For the first time I've asked myself "might this actually be easier to code in C rather than in python?", and I am not looking at device...
1
by: Al | last post by:
Searching for a way to upload binary files from client to server, I came across the following link: http://www.dotnetextreme.com/code/BinaryUpload.asp With this precaution: Editor's note -...
1
by: mdefoor | last post by:
I've written the following sample to split a string into chunks. Is this the right approach or is there perhaps a better way to accomplish getting chunks of a string? #include <stdio.h>...
1
by: JayDog | last post by:
I have a large data file that I split into smaller more manageable chunks (went from a 12.86 GB file to 500 MB - 1.6 GB chunks). I now want to add to the PERL script and go back through those more...
3
by: codemannh | last post by:
I have been trying to figure out the best way to parse some chunks of html code that contain tables. I've been trying to do this with HTML::Parser and HTML::TokeParser and HTML::TokeParser::Simple,...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.