471,831 Members | 821 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,831 software developers and data experts.

Sequential XML parsing with xml.sax

Hi hi.

I'm trying to do sequential decompression of a bzipped XML file and
feed it to a SAX parser with the following code.

remotefh = urllib.urlopen('file:///home/peter/catalog.rdf.bz2')
decompressor = bz2.BZ2Decompressor()
handler = CatalogueDocumentHandler(sys.stdout)
chunksize = 2048
data = remotefh.read(chunksize)
while data != '':
out = decompressor.decompress(data)
if out != '':
xml.sax.parseString(out, handler)
data = remotefh.read(chunksize)

This fails with the first chunk of decompressed data passed to
xml.sax.parseString. I'm suspecting because it's an incomplete fragment
of XML. I've tried with a number of different chunk sizes, putting the
break in different places, but it always fails on the first call. For
reference, the traceback looks like:

xml.sax.parseString(out, handler)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/__init__.py",
line 47, in parseString
parser.parse(inpsrc)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py",
line 109, in parse
xmlreader.IncrementalParser.parse(self, source)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/xmlreader.py",
line 125, in parse
self.close()
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py",
line 226, in close
self.feed("", isFinal = 1)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py",
line 220, in feed
self._err_handler.fatalError(exc)
File "/usr/lib/python2.4/site-packages/_xmlplus/sax/handler.py", line
38, in fatalError
raise exception
xml.sax._exceptions.SAXParseException: <unknown>:15132:63: no element
found

(line 15132 is the last, incomplete line feed to parseString. FWIW,
it's:
<pgterms:friendlytitle rdf:parseType="Literal">Searchlights o
)

The API reference isn't clear on whether parseString can only handle
discrete bits of valid XML, or if it's designed to be called in this
way. So I'm not sure if I'm misusing the function, or if I've done
something else wrong.

Any pointers?
Thanks,
--
Pete

Aug 23 '05 #1
2 3134
pe***@hardy.dropbear.id.au wrote:
The API reference isn't clear on whether parseString can only handle
discrete bits of valid XML


the documentation says that "parse" expects an XML document,
and that "parseString" is the same thing, but parses from a buffer.

it's probably easier to pass a BZ2File instance to "parse", but if you
insist on doing incremental SAX parsing, the IncrementalParser class
might be what you need:

http://www.python.org/doc/current/li...xmlreader.html
http://www.python.org/doc/current/li...r-objects.html

</F>

Aug 23 '05 #2
Hi.

Fredrik Lundh wrote:
pe***@hardy.dropbear.id.au wrote:
The API reference isn't clear on whether parseString can only handle
discrete bits of valid XML
the documentation says that "parse" expects an XML document,
and that "parseString" is the same thing, but parses from a buffer.


OK, so it sounded a lot more ambiguous at 4am. :-)
it's probably easier to pass a BZ2File instance to "parse",
It is easier to retrieve a remote file, and decompress and parse as
separate steps. But I've been wondering if it would be faster / more
efficient to do it without caching.
but if you
insist on doing incremental SAX parsing, the IncrementalParser class
might be what you need:


That'll do the trick nicely. Thanks.

Cheers,
--
Pete

Aug 24 '05 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

7 posts views Thread by GAVO. | last post: by
2 posts views Thread by Bruce W.1 | last post: by
2 posts views Thread by Mike Kingscott | last post: by
1 post views Thread by rayw | last post: by
13 posts views Thread by Chris Carlen | last post: by
NeoPa
reply views Thread by NeoPa | last post: by
reply views Thread by YellowAndGreen | last post: by
aboka
reply views Thread by aboka | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.