Parsing HTML?

[Benjamin]

I'm trying to parse an HTML file. I want to retrieve all of the text
inside a certain tag that I find with XPath. The DOM seems to make
this available with the innerHTML element, but I haven't found a way
to do it in Python.

[Daniel Fetchinson]

I'm trying to parse an HTML file. I want to retrieve all of the text

inside a certain tag that I find with XPath. The DOM seems to make
this available with the innerHTML element, but I haven't found a way
to do it in Python.

Have you tried http://www.google.com/search?q=python+html+parser ?

HTH,
Daniel

[benash]

BeautifulSoup does what I need it to. Though, I was hoping to find
something that would let me work with the DOM the way JavaScript can
work with web browsers' implementations of the DOM. Specifically, I'd
like to be able to access the innerHTML element of a DOM element.
Python's built-in HTMLParser is SAX-based, so I don't want to use
that, and the minidom doesn't appear to implement this part of the
DOM.

On Wed, Apr 2, 2008 at 10:37 PM, Daniel Fetchinson
<fe********@goo glemail.comwrot e:

I'm trying to parse an HTML file. I want to retrieve all of the text
inside a certain tag that I find with XPath. The DOM seems to make
this available with the innerHTML element, but I haven't found a way
to do it in Python.

Have you tried http://www.google.com/search?q=python+html+parser ?

HTH,
Daniel

[7stud]

On Apr 3, 12:39*am, [e-mail] m wrote:

BeautifulSoup does what I need it to. *Though, I was hoping to find
something that would let me work with the DOM the way JavaScript can
work with web browsers' implementations of the DOM. *Specifically, I'd
like to be able to access the innerHTML element of a DOM element.
Python's built-in HTMLParser is SAX-based, so I don't want to use
that, and the minidom doesn't appear to implement this part of the
DOM.

innerHTML has never been part of the DOM. It is however a defacto
browser standard. That's probably why you aren't having any luck
using a python module that implements the DOM.

[Stefan Behnel]

Benjamin wrote:

I'm trying to parse an HTML file. I want to retrieve all of the text
inside a certain tag that I find with XPath. The DOM seems to make
this available with the innerHTML element, but I haven't found a way
to do it in Python.

import lxml.html as h
tree = h.parse("somefi le.html")
text = tree.xpath("str ing( some/element[@condition] )")

http://codespeak.net/lxml

Stefan

[Benjamin]

On Apr 3, 9:10*pm, 7stud <bbxx789_0...@y ahoo.comwrote:

On Apr 3, 12:39*am, [e-mail] m wrote:

BeautifulSoup does what I need it to. *Though, I was hoping to find
something that would let me work with the DOM the way JavaScript can
work with web browsers' implementations of the DOM. *Specifically, I'd
like to be able to access the innerHTML element of a DOM element.
Python's built-in HTMLParser is SAX-based, so I don't want to use
that, and the minidom doesn't appear to implement this part of the
DOM.

innerHTML has never been part of the DOM. *It is however a defacto
browser standard. *That's probably why you aren't having any luck
using a python module that implements the DOM.

That makes sense.

[Benjamin]

On Apr 6, 11:03*pm, Stefan Behnel <stefan...@behn el.dewrote:

Benjamin wrote:

I'm trying to parse an HTML file. *I want to retrieve all of the text
inside a certain tag that I find with XPath. *The DOM seems to make
this available with the innerHTML element, but I haven't found a way
to do it in Python.

* * import lxml.html as h
* * tree = h.parse("somefi le.html")
* * text = tree.xpath("str ing( some/element[@condition] )")

http://codespeak.net/lxml

Stefan

I actually had trouble getting this to work. I guess only new version
of lxml have the html module, and I couldn't get it installed. lxml
does look pretty cool, though.

[Stefan Behnel]

Benjamin wrote:

On Apr 6, 11:03 pm, Stefan Behnel <stefan...@behn el.dewrote:

>Benjamin wrote:

>>I'm trying to parse an HTML file. I want to retrieve all of the text
inside a certain tag that I find with XPath. The DOM seems to make
this available with the innerHTML element, but I haven't found a way
to do it in Python.

import lxml.html as h
tree = h.parse("somefi le.html")
text = tree.xpath("str ing( some/element[@condition] )")

http://codespeak.net/lxml

Stefan

I actually had trouble getting this to work. I guess only new version
of lxml have the html module, and I couldn't get it installed. lxml
does look pretty cool, though.

Yes, the above code requires lxml 2.x. However, older versions should allow
you to do this:

import lxml.etree as et
parser = etree.HTMLParse r()
tree = h.parse("somefi le.html", parser)
text = tree.xpath("str ing( some/element[@condition] )")

lxml.html is just a dedicated package that makes HTML handling beautiful. It's
not required for parsing HTML and doing general XML stuff with it.

Stefan