473,379 Members | 1,491 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,379 software developers and data experts.

Sequential XML parsing with xml.sax

Hi hi.

I'm trying to do sequential decompression of a bzipped XML file and
feed it to a SAX parser with the following code.

remotefh = urllib.urlopen('file:///home/peter/catalog.rdf.bz2')
decompressor = bz2.BZ2Decompressor()
handler = CatalogueDocumentHandler(sys.stdout)
chunksize = 2048
data = remotefh.read(chunksize)
while data != '':
out = decompressor.decompress(data)
if out != '':
xml.sax.parseString(out, handler)
data = remotefh.read(chunksize)

This fails with the first chunk of decompressed data passed to
xml.sax.parseString. I'm suspecting because it's an incomplete fragment
of XML. I've tried with a number of different chunk sizes, putting the
break in different places, but it always fails on the first call. For
reference, the traceback looks like:

xml.sax.parseString(out, handler)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/__init__.py",
line 47, in parseString
parser.parse(inpsrc)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py",
line 109, in parse
xmlreader.IncrementalParser.parse(self, source)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/xmlreader.py",
line 125, in parse
self.close()
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py",
line 226, in close
self.feed("", isFinal = 1)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py",
line 220, in feed
self._err_handler.fatalError(exc)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/handler.py", line
38, in fatalError
raise exception
xml.sax._exceptions.SAXParseException: <unknown>:15132:63: no element
found

(line 15132 is the last, incomplete line feed to parseString. FWIW,
it's:
<pgterms:friendlytitle rdf:parseType="Literal">Searchlights o
)

The API reference isn't clear on whether parseString can only handle
discrete bits of valid XML, or if it's designed to be called in this
way. So I'm not sure if I'm misusing the function, or if I've done
something else wrong.

Any pointers?
Thanks,
--
Pete

Aug 23 '05 #1
2 3203
pe***@hardy.dropbear.id.au wrote:
The API reference isn't clear on whether parseString can only handle
discrete bits of valid XML


the documentation says that "parse" expects an XML document,
and that "parseString" is the same thing, but parses from a buffer.

it's probably easier to pass a BZ2File instance to "parse", but if you
insist on doing incremental SAX parsing, the IncrementalParser class
might be what you need:

http://www.python.org/doc/current/li...xmlreader.html
http://www.python.org/doc/current/li...r-objects.html

</F>

Aug 23 '05 #2
Hi.

Fredrik Lundh wrote:
pe***@hardy.dropbear.id.au wrote:
The API reference isn't clear on whether parseString can only handle
discrete bits of valid XML
the documentation says that "parse" expects an XML document,
and that "parseString" is the same thing, but parses from a buffer.


OK, so it sounded a lot more ambiguous at 4am. :-)
it's probably easier to pass a BZ2File instance to "parse",
It is easier to retrieve a remote file, and decompress and parse as
separate steps. But I've been wondering if it would be faster / more
efficient to do it without caching.
but if you
insist on doing incremental SAX parsing, the IncrementalParser class
might be what you need:


That'll do the trick nicely. Thanks.

Cheers,
--
Pete

Aug 24 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
by: GAVO. | last post by:
Hello every one I have a database with a form called "frmOrders" on that for I need to create a sequential number for each city apart from the original autonumber. So the table "tblorders" would...
2
by: Al Bahr | last post by:
H I am try to convert C# to vb.net I don’t know what would be an equivalent statements in VB.net any help will be appreciated. bmiColors ' RGBQUAD structs... Blue-Green-Red-Reserved, repeat.....
2
by: Bruce W.1 | last post by:
ASP.NET processes one request at a time, sequentially. If one process blocks or sleeps then it brings down the entire application for all users. This fact is hidden under the rug by Microsoft. ...
2
by: Mike Kingscott | last post by:
Hi all, I'd building an app that posts to a web service. One of the things that is required in the soap header is a sequential number appended to a ref, i.e. "IGI1001", "IGI1002", etc. ...
1
by: rayw | last post by:
I was wondering if there were some good on-line references to the various pros/cons with these types of file access at all? Ta rayw
11
by: Timofmars | last post by:
I'm try to Unload DB2 data from a table into a record sequential file on NT. I can an unload on Mainframe, but it doesn't seem to be an option in NT. In NT, all I can do is export/import. I can...
1
maxamis4
by: maxamis4 | last post by:
Hello folks, I have two forms a parent form and a subform. The parent form is an unbound form while the subform is a form that contains all a list of what I like to call 'in stock ' phone...
13
by: Chris Carlen | last post by:
Hi: Having completed enough serial driver code for a TMS320F2812 microcontroller to talk to a terminal, I am now trying different approaches to command interpretation. I have a very simple...
5
by: p3rk3le | last post by:
So, I'm about to do a sequential search on a table (n contents) of random numbers. I have to print the average between the number of comparisons and the contents of the table (n) and the...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.