html 2 plain text

[robin]

hi,
i remember seeing this simple python function which would take raw html
and output the content (body?) of the page as plain text (no <..> tags
etc)
i have been looking at htmllib and htmlparser but this all seems to
complicated for what i'm looking for. i just need the main text in the
body of some arbitrary webbpage to then do some natural-language
processing with it...
thanks for pointing me to some helpful resources!

robin

[Faber]

robin wrote:

i remember seeing this simple python function which would take raw html
and output the content (body?) of the page as plain text (no <..> tags
etc)
i have been looking at htmllib and htmlparser but this all seems to
complicated for what i'm looking for. i just need the main text in the
body of some arbitrary webbpage to then do some natural-language
processing with it...
thanks for pointing me to some helpful resources!

Have a look at the Beautiful Soup library:
http://www.crummy.com/software/BeautifulSoup/

Regards

--
Faber
http://faberbox.com/
http://smarking.com/

A teacher must always teach to doubt his teaching. -- José Ortega y Gasset

[robin]

lucks yummy. merci beaucoup.

robin

[Ravi Teja]

> i remember seeing this simple python function which would take raw html

and output the content (body?) of the page as plain text (no <..> tags
etc)

http://www.aaronsw.com/2002/html2text/

[garabik-news-2005-05]

robin <ro*********@gmail.com> wrote:

hi,
i remember seeing this simple python function which would take raw html
and output the content (body?) of the page as plain text (no <..> tags
etc)
i have been looking at htmllib and htmlparser but this all seems to
complicated for what i'm looking for. i just need the main text in the
body of some arbitrary webbpage to then do some natural-language
processing with it...
thanks for pointing me to some helpful resources!

text=re.sub(r'(?s)\<.+?\>', '', html_text)
(this will keep html entities, though)

--
-----------------------------------------------------------
| Radovan GarabÃ*k http://kassiopeia.juls.savba.sk/~garabik/ |
| __..--^^^--..__ garabik @ kassiopeia.juls.savba.sk |
-----------------------------------------------------------
Antivirus alert: file .signature infected by signature virus.
Hi! I'm a signature virus! Copy me into your signature file to help me spread!

[Fredrik Lundh]

ga******************@kassiopeia.juls.savba.sk wrote:

text=re.sub(r'(?s)\<.+?\>', '', html_text)
(this will keep html entities, though)

here's a variation that handles that too:

http://effbot.org/zone/re-sub.htm#strip-html

</F>