HTML File Parsing

[Felipe De Bene]

I'm having problems parsing an HTML file with the following syntax :

<TABLE cellspacing=0 cellpadding=0 ALIGN=CENTER BORDER=1 width='100%'>
<TH BGCOLOR='#c0c0c 0' Width='3%'>User ID</TH>
<TH Width='10%' BGCOLOR='#c0c0c 0'>Name</TH><TH width='7%'
BGCOLOR='#c0c0c 0'>Date</TH>
and so on....

whenever I feed the parser with such file I get the error :

Traceback (most recent call last):
File "C:\Documen ts and Settings\Admini strator\My Documents\works pace
\thread\src\par ser.py", line 91, in <module>
p.parse(thechan ge)
File "C:\Documen ts and Settings\Admini strator\My Documents\works pace
\thread\src\par ser.py", line 16, in parse
self.feed(s)
File "C:\Python25\li b\HTMLParser.py ", line 110, in feed
self.goahead(0)
File "C:\Python25\li b\HTMLParser.py ", line 152, in goahead
k = self.parse_endt ag(i)
File "C:\Python25\li b\HTMLParser.py ", line 316, in parse_endtag
self.error("bad end tag: %r" % (rawdata[i:j],))
File "C:\Python25\li b\HTMLParser.py ", line 117, in error
raise HTMLParseError( message, self.getpos())
HTMLParser.HTML ParseError: bad end tag: "</TH BGCOLOR='#c0c0c 0'>", at
line 515, column 45

Googling around I've found a solution to a similar situation, over and
over again :
http://64.233.169.104/search?q=cache...ient=firefox-a

but coding :

you can disable proper parsing by setting the CDATA_CONTENT_E LEMENTS
attribute on the parser instance, before you start parsing. by
default, it is
set to
CDATA_CONTENT_E LEMENTS = ("script", "style")
setting it to an empty tuple disables HTML-compliant handling for
these
elements:
p = HTMLParser()
p.CDATA_CONTENT _ELEMENTS = ()
p.feed(f.read() )

didn't solve my problem. I've made a little modification then to
HTMLParser.py instead that solved the problem, as follows:
original: endtagfind = re.compile('</\s*([a-zA-Z][-.a-zA-Z0-9:_]*)?(.*)
\s*>')
my version : endtagfind = re.compile('</\s*([a-zA-Z][-.a-zA-Z0-9:_]*)
\s*>')

it worked ok for all the files I needed and also for a different file
I also parse using the same library. I know it might sound stupid but
I was just wondering if there's a better way of solving that problem
than just modifying the standard library. Any clue ?

thx in advance,
Felipe.

[Stefan Behnel]

Felipe De Bene wrote:

I'm having problems parsing an HTML file with the following syntax :

<TABLE cellspacing=0 cellpadding=0 ALIGN=CENTER BORDER=1 width='100%'>
<TH BGCOLOR='#c0c0c 0' Width='3%'>User ID</TH>
<TH Width='10%' BGCOLOR='#c0c0c 0'>Name</TH><TH width='7%'
BGCOLOR='#c0c0c 0'>Date</TH>
and so on....

whenever I feed the parser with such file I get the error :

HTMLParser.HTML ParseError: bad end tag: "</TH BGCOLOR='#c0c0c 0'>", at
line 515, column 45

Your HTML page is not HTML, i.e. it is broken. Python's HTMLParser is not made
for parsing broken HTML. However, you can use the parse of lxml.html to fix up
your HTML for you.

http://codespeak.net/lxml/

Stefan

[Felipe De Bene]

On Oct 28, 6:18 pm, Stefan Behnel <stefan...@behn el.dewrote:

Felipe De Bene wrote:

I'm having problems parsing anHTMLfile with the following syntax :

<TABLE cellspacing=0 cellpadding=0 ALIGN=CENTER BORDER=1 width='100%'>
<TH BGCOLOR='#c0c0c 0' Width='3%'>User ID</TH>
<TH Width='10%' BGCOLOR='#c0c0c 0'>Name</TH><TH width='7%'
BGCOLOR='#c0c0c 0'>Date</TH>
and so on....

whenever I feed the parser with such file I get the error :

HTMLParser.HTML ParseError: bad end tag: "</TH BGCOLOR='#c0c0c 0'>", at
line 515, column 45

YourHTMLpage is notHTML, i.e. it is broken. Python's HTMLParser is not made
for parsing brokenHTML. However, you can use the parse of lxml.htmlto fix up
yourHTMLfor you.

http://codespeak.net/lxml/

Stefan

Actually i fetch from an application that i thought it should act like
this and as I told you, the program is ready to be shipped so
rewriting an entire class that has public methods would be a real
pain. I really had to find a way to work this out by using the
python's parser instead of external libraries. But thanks anyway for
the clue, I might start working on a similar project next and this
library may be a good and a less painful path. Thanks :D
Felipe.